IntegrAuth Academy
Identity, security and practical AI — free, from zero. Byte-sized lessons, real diagrams, hands-on labs, zero fluff. Two programs, each with its own exam and certificate.
English is hard? Your browser can translate this page into Tamil, Telugu, Kannada, Malayalam or Hindi. Tech words stay in English so you learn the real names.
How?
- Chrome on phone: ⋮ menu → Translate
- Chrome on computer: right-click → Translate to …
- Edge: the translate icon in the address bar
Guardrails and jailbreaks
Priya's team is launching a billing chatbot. Zara asks what stops someone from tricking it. Your goal: put checks around a chatbot and see what gets through.
🧪 Interactive lab — enable JavaScript to play with this one.
- A guardrail is a check around the model, not inside it.
- Check what goes in and what comes out.
- Every check can miss an attack or block a real customer.
- Some attacks will get through. So limit what the bot can do.
Around the model
Guardrails are checks that run before and after the model. Think of a shop with a bag check at the entrance and a bill check at the exit. A rule in your system prompt is only a request. An attacker can argue with it. A guardrail is separate code or a separate model. The chat can't argue with it.
An input guardrail stops bad requests early. An output guardrail checks the reply for mistakes and leaks.
Two kinds of mistake
A check can let an attack through. This is a false negative. It can also block a real customer. Say someone asks, "How do I kill a stuck payment?" Blocking that is a false positive. Tuning means choosing which mistake hurts more.
When an attack gets through
In a jailbreak, the person typing tries to talk the model out of its rules. New tricks keep appearing, so some will get through. So limit what a fooled bot can do. This is least privilege. A billing bot with no refund tool can't give a refund. It reads account data only for the signed-in customer.
Now tune two checks yourself.
🧪 Interactive lab — enable JavaScript to play with this one.
Go deeper: guardrail types
| Guardrail | What it checks | Where |
|---|---|---|
| Moderation | Harmful content, like abuse or violence | In and out |
| Jailbreak detector | Text that tries to override the rules | In |
| PII masking | Personal data, like card numbers and emails, hidden before the model or logs see it | In and out |
| Topic limit | Is this about billing at all? | In |
| Length and rate limits | Very long messages, or too many tries a minute | In |
| Schema check | Is the output valid JSON with allowed values? | Out |
| Groundedness check | Is each claim backed by the sources? | Out |
| Leak scan | Secrets, other people's data, or parts of the system prompt | Out |
Go deeper: building guardrails
Rules are patterns and lists of banned words. An example is "a number of 13 to 19 digits that passes the card checksum". Rules are fast, cheap and predictable. But attackers can reword a message to get past them.
A classifier is a small, fast model that labels text. Labels can be "safe" or "unsafe", or "on topic" or "off topic".
With an LLM as judge, a second model reads the chat and decides. It handles fuzzy rules, like "Is this financial advice?" But it is slower and costs more. It can be fooled, too.
You rarely build these from scratch. As of October 2026, open-source toolkits include NeMo Guardrails and Guardrails AI. Presidio finds and masks personal data. Cloud platforms offer ready-made checks, like Amazon Bedrock Guardrails and Azure AI Content Safety. AI-security companies sell guardrails, too. Examples are Lakera, now part of Check Point, and Enkrypt AI, now part of Anaconda.
Go deeper: jailbreak or injection?
Both try to make the model ignore its rules. The difference is who speaks. In a jailbreak, the person typing is the attacker. In prompt injection, the orders hide in content the model reads, like an email or a web page. The attacker never talks to the bot.
Jailbreak types to test for:
- Role-play. The attacker asks the model to act as an AI with no rules.
- Hiding the request. It is written in a code like Base64, or in another language. The filter may miss it, but the model can read it.
- Many-shot. A very long message holds many fake chats where the assistant breaks its rules. The model may follow the pattern.
- Slow drift. Many small, harmless steps lead to the bad goal. Each message alone looks fine. So check the whole chat.
- Automated search. Software tries thousands of small changes until one works. No word list is ever complete.
Go deeper: red-teaming
Red-teaming means attacking your own app on purpose, before someone else does. It is a loop:
- Write down what "bad" means for this app. Agree on it with the business.
- Build a set of attacks: the jailbreak types, hidden orders in each content source, and attacks for your field.
- Run them with open-source scanners, like garak, promptfoo or PyRIT. Then add people for creative attacks.
- Measure how often each attack type works, and how often normal messages get blocked.
- Fix what got through. Keep every attack that worked in a test set. Rerun it after each change, like evals.
The OWASP Top 10 for LLM Applications is a good checklist. Prompt injection is number one on its list.
Guardrails reduce risk. They never remove it. Let a person approve anything that can't be undone. Keep a fast way to turn the bot off (kill switch). Keep secrets out of the system prompt.
Play Lakera's free AI hacking games, like Gandalf. Each level is harder.