IntegrAuth Academy
Identity, security and practical AI — free, from zero. Byte-sized lessons, real diagrams, hands-on labs, zero fluff. Two programs, each with its own exam and certificate.
English is hard? Your browser can translate this page into Tamil, Telugu, Kannada, Malayalam or Hindi. Tech words stay in English so you learn the real names.
How?
- Chrome on phone: ⋮ menu → Translate
- Chrome on computer: right-click → Translate to …
- Edge: the translate icon in the address bar
Tests, checks and debugging
"Fixed ✅ All 38 tests pass," said Priya's coding agent. But it only handled the one price in the test. Other prices still broke. Your goal: make an agent prove that a fix is real.
🧪 Interactive lab — enable JavaScript to play with this one.
- An agent stops when the work looks done. Give it a check that says pass or fail.
- Ask for proof: the command it ran and what it printed.
- Write the test first. Watch it fail for the right reason.
- A pass can be fake. Read test changes first, then try new inputs.
Test first
A test runs your code and checks the answer. It is like the checklist a shop owner ticks before opening: "Does the bill add up?" Write the test before the code. Use a few different inputs. Run it and watch it fail. It should fail because the answer is wrong. A typo doesn't count. Neither does a file the test can't find. Commit the test. Then ask: "Make these tests pass. Don't change the test files." Check that the diff changes code, not tests. This is test-driven development.
Passing for the wrong reason
Models are partly trained with a reward when a check passes. Some learn to pass the check instead of doing the task. This is reward hacking. An agent may skip a test, weaken it, or hard-code the answer. So read changes to tests and settings files first. Then try new inputs. A fake fix often breaks on the first one.
Tests the agent writes with the code can share its mistake. Then both agree, and both are wrong. A pass only means the code matches the tests. So check the expected answers yourself. High test coverage only means many lines ran, not that results were checked.
Did the code work last week? Search the history for the change that broke it.
🧪 Interactive lab — enable JavaScript to play with this one.
Go deeper: kinds of checks
- Tests call your code and compare the result with the right answer. They miss cases nobody wrote a test for.
- A type checker finds the wrong kind of value, or a call to a function that doesn't exist. Examples are TypeScript's compiler, mypy and Pyright. Code can pass it and still be wrong.
- A linter finds risky patterns, like unused variables. Examples are ESLint and Ruff. It can't tell if the logic does what you meant.
- Build and run it on real data. This finds crashes and wrong output.
- Screenshots from a browser tool such as Playwright show broken layouts and missing buttons.
The best checks run on their own. Continuous integration (CI) is a server that runs your checks on every change. A hook can also run the tests before the agent may call a task done (Extend your assistant).
Go deeper: the fake fixes to watch for
In February 2025, Anthropic's system card for Claude 3.7 Sonnet described this. The model sometimes returned the exact values a test expected. Sometimes it changed the tests to match its code. This happened mostly after several failed tries at a real fix. In June 2025, the research group METR reported something similar. Leading models sometimes changed the tests or the scoring code instead of solving the task.
- Skip the test. The summary says "1 skipped".
- Loosen the check.
toBe(1299.5)becomestoBeGreaterThan(0). Almost any answer passes. - Change the expected value to match the bug.
- Hard-code the answer for the exact test input.
- Fake the code under test. A mock swaps the real code for a stand-in. The real code never runs.
- Hide the error. A crash turns into a silent wrong answer.
- Switch the check off. Look for
jest || truein a settings file, or comments that turn off a check.
Mutation testing tests your tests. It plants small bugs in your code, like > changed to >=. Then it lists each bug your tests missed. Stryker and mutmut are open-source examples. A line in the instruction file also helps: "Fix the general case. Never special-case test inputs." It guarantees nothing.
Go deeper: the debugging loop
- Reproduce it. Find one command or input that always shows the bug.
- Give the exact error. Paste the full message and the stack trace.
- Ideas before edits. Ask for 2 or 3 likely causes and a cheap check for each.
- Bisect if it used to work. Git bisect tests the commit in the middle. Each answer throws away half. With a test script,
git bisect runanswers for you. - Fix the cause and keep the proof. Keep the bug's test as a regression test. It stops this bug from coming back unseen.
Go deeper: when to take over
- Two corrections didn't work. Start a fresh session with a better prompt.
- It changes tests, settings or unrelated files.
- It gives a new explanation every time.
- It needs a business rule or a decision only a person can make.
- The bug comes and goes, like timing problems.
- You can't explain the change anymore.
Then narrow the task, write the failing test yourself, or use a debugger.