I’ll be straight about something a lot of AI tools won’t: AI gets things wrong. Sometimes confidently. It can produce errors, bias, and convincing-sounding misinformation, and no amount of clever prompting fully removes that.
So when people ask whether any AI tool is truly “safe,” I don’t think the honest answer is a flat yes. AI is imperfect. The real question is whether a tool is designed around that reality, with constraints, review points, and a teacher in the loop to catch what slips through.
“Guardrails” is not a magic word
You’ll see a lot of AI tools offer guardrails as if that settles it. We use that word too, but I want to be careful with it, because it can imply a guarantee we can’t make.
Guardrails are real, and they help. They are not a force field. The honest framing is that constraints reduce the chance of a bad output. They don’t reduce it to zero.
What we actually do
CARL is intentionally constrained. It’s built on pedagogical rules, structured workflows, and review points rather than an open-ended prompt box. That structure helps narrow how far it can wander. CARL is also designed to flag uncertainty, avoid unnecessary assumptions, and prompt review at the points where getting something wrong matters most.
All of that is meant to reduce risk. None of it is meant to replace your judgment.
Inclusion guardrails are part of that design
One place this matters is representation. AI can default to narrow assumptions about who a student is, what a family looks like, who belongs in certain roles, or what a classroom community includes. We do not pretend CARL can remove bias completely, because no AI tool can honestly promise that.
What we can do is design against narrow defaults. CARL’s lesson and resource prompts are written to avoid assuming gender, race, family structure, sexual orientation, socioeconomic background, ability, language background, or culture when those details are not relevant. When those details do matter, CARL is meant to handle them carefully and in a teacher-reviewed way.
The same thinking applies to CARL’s image generation tools. The inclusion setting is designed to encourage broader, context-appropriate representation across race, skin tone, hair texture, ability, body size and shape, age, and visible classroom diversity. The goal is not to make every image about diversity. The goal is to stop the same narrow visual defaults from showing up again and again.
That still requires review. An image can look polished and still include a stereotype, an assumption, or a mismatch with the lesson context. That’s why generated images are drafts, not finished classroom materials.
For more on this part of the product, read Images as Diverse as Your Classroom.
The real safeguard is you
This is the part that matters. CARL’s outputs are starting points, not final answers. The platform is built so you review, edit, and decide what’s actually right for your class. You’re the one who catches the example that’s off, the claim that isn’t quite true, the framing that doesn’t fit your students, or the image that doesn’t reflect the classroom context well enough.
A tool that told you it was never wrong would be doing something worse than making mistakes. It would be teaching you to stop checking. We’d rather build something honest about its limits, because that keeps the right person in charge.
Some things we design CARL not to do
Reducing risk also means deciding where CARL shouldn’t go at all.
The clearest example is Indigenous content. We are not Indigenous, and CARL is designed not to generate or interpret Indigenous knowledge. Instead, the Indigenous Resource Pathways feature is built to point teachers toward Indigenous-authored and Indigenous-led sources where authorship is clear, and to name a gap rather than invent a connection when certainty isn’t possible. We don’t claim partnerships or consultation we haven’t undertaken. You can read the fuller version of how we think about this on our Commitments and Design Principles page.
That’s a limit by design, not a missing feature. Some things should come from relationship and consent, not a model.
The Bottom Line
AI is going to do what AI does. It will be useful, and it will sometimes be wrong.
Our job isn’t to pretend otherwise. It’s to constrain CARL sensibly, design against narrow defaults, flag uncertainty honestly, refuse the things that shouldn’t be automated, and keep a teacher in the loop to make the final call.
That’s not the flashiest promise. But it’s one we can actually keep.