A review that names a weakness has done half the job. The other half is a fix that closes the hole for every way in, not only for the example in the report. In this part Claude Code fixed the 20 planted vulnerabilities of the patient portal one at a time, twenty sessions on one working copy, and after each session a hidden suite of attack tests decided whether the hole was closed and whether an earlier fix had broken. All twenty were closed on the first run, in 17 minutes of session time. The tests cannot say everything, so each fix was then read against the secure reference app and probed. That reading found one claim that was wrong, one fix that arrived twelve sessions early, and several choices a team should know it is getting.

What a session was given, and what judged it

Each session received the finding as a reviewer would write it: a title, a file, what is wrong and what an attacker gains. No test, no hint of how the fix is checked. The prompt around it was the same every time:

Read CLAUDE.md and ROUTES.md. A security review reported this finding:

{finding}

Fix it in this project without changing the public contract in ROUTES.md. Build, run the tests
with the command in CLAUDE.md, and stop when they pass. Do not edit anything under tests/ and
do not commit. Print a summary of what you changed and why it closes the finding.

The sessions ran headless on Claude Code 2.1.287 with Opus 5.5 and could read, edit, and run dotnet; edits under tests/ were denied. After a session ended, the harness ran the 43 functional tests, the two to four attack tests of that weakness, and the attack tests of every weakness fixed before it, then committed. A failed verdict would have earned one retry with the failing test names, and after that a hand fix. Neither was needed.

One caveat belongs here. The finding texts were written by the author, who knew the answer, and each names the file. Fixing from a precise finding is easier than fixing from a vague report that documents can be read by the wrong person.

Twenty fixes

One session per weakness, in this order, each on top of the fixes before it. Every verdict: functional 43 of 43, the weakness's own attack tests all passing, no earlier test broken. Meter = the notional API price the CLI reports.
IdWeaknessTurnsSecondsMeterLines + / -Same design as the reference app?
V01SQL concatenation in search849$0.2116 / 1yes; also escapes LIKE wildcards
V02Records without an owner check640$0.175 / 0identical
V03Admin export open to anyone535$0.182 / 2yes
V04Mass assignment on the profile636$0.183 / 15yes
V05Note rendered as raw HTML426$0.121 / 1identical
V06Antiforgery switched off533$0.161 / 2identical
V07Committed key and password951$0.2315 / 5yes; adds a check for the leaked value and a 32-byte minimum
V08Unsalted SHA-256 passwords1583$0.3585 / 8no: its own PBKDF2 with old hashes upgraded at sign-in; the reference uses the Identity hasher
V09No lockout1381$0.3629 / 7yes at sign-in; the counter is written differently
V10Token validation off745$0.2215 / 3yes; also pins the algorithm
V11CORS any origin with credentials631$0.141 / 1identical
V12Developer exception page everywhere635$0.158 / 3no: keeps the developer page in Development; the reference never shows it
V13Vulnerable package, audit off1375$0.315 / 2same version; adds four audit settings
V14Path traversal in download749$0.2042 / 6yes; adds a file-name check
V15Server fetches any URL1391$0.4249 / 6same policy; follows up to three checked redirects where the reference follows none
V16Password and token in the log842$0.192 / 2yes
V17Cookie flags744$0.185 / 3yes, with SameSite Strict where the reference has Lax
V18Open redirect1164$0.2229 / 2same rule in a new function; also checks the failed sign-in path
V19Password hash in responses753$0.210 / 0nothing left to change (see below)
V20Computable reset code652$0.2253 / 7yes; also voids older codes

In all: 162 turns, 1,014 seconds, $4.44 on the meter, 366 lines added and 76 removed, no package added, no test touched. Five commands were denied, each a chained cd with something else, and each session then did the same thing with the Edit tool or a plain command. On the final tree the whole attack suite passes, 53 of 53, and so do the 43 functional tests.

Four fixes worth reading

Passwords (V08) was the largest change. The session wrote PBKDF2 by hand with a per-user salt, and it thought about the users who already exist: a hash in the old format still verifies and is replaced at the next correct sign-in.

+    private const int Iterations = 600_000; // OWASP 2023 recommendation for PBKDF2-HMAC-SHA256
+        Rfc2898DeriveBytes.Pbkdf2(Encoding.UTF8.GetBytes(password), salt, iterations, HashAlgorithmName.SHA256, HashSize);
+        if (IsLegacy(storedHash))
+            return CryptographicOperations.FixedTimeEquals(legacy, Convert.FromHexString(storedHash));
+        if (hashing.NeedsRehash(user.PasswordHash))
+            user.PasswordHash = hashing.Hash(password ?? string.Empty);

A probe confirmed the behaviour: after a wrong sign-in the old hash is unchanged, after a correct one it starts pbkdf2-sha256$600000$. The session named the cost itself: "a user who never signs in again keeps their old unsalted hash in the database." The reference app uses ASP.NET Core Identity's hasher instead, which is less code to own.

The avatar fetch (V15) found the unused address policy that planting had left behind and wired it back in: https only, a check of every resolved address at connect time, no proxy. It went further than the reference, following up to three redirects with each target checked again and adding address ranges. Twenty-eight URL forms were probed, from https://2130706433/a.png to https://[::ffff:127.0.0.1]/a.png and the cloud metadata address; every loopback, private and special form was refused. A redirect from a public host to a private one could not be probed here.

The reset code (V20) became 32 random bytes, stored only as a SHA-256 hash, valid for 30 minutes and once, with older codes voided by a new request. The open redirect (V18) got a new IsLocalUrl function, although a helper with the same rule sat unused two files away; the two agreed on all fifteen inputs tried.

🤖 What the AI got wrong, or did differently than asked

  • A claim that did not hold. After moving the signing key out of the settings file, the session's summary said "The app won't start with no key, the leaked key or a short key." It does start. The key is read on the first request, so the host comes up and then every request, the health check included, answers 500 with the configuration error. The hole is closed either way; the sentence is still wrong, and a deployment that only checks "the process is running" would not notice.
  • The leaked key is now a string in the source. To refuse the old value, the fix compares against it, so the committed key moved from the settings file into AuthSetup.cs. It was already in git history, and the session said so: "The old values are still in git history, so treat them as public". Rotating the key is the fix; the comparison is decoration.
  • One fix arrived twelve sessions early. While fixing password hashing, the session also removed the hash from the patient list and the export, calling it "a related problem I found". That was V19. When V19's turn came, there was nothing left to change. The edit was right, and it was also outside the finding the session had been given.
  • A different judgement call. For the exception page (V12) the fix keeps the developer page in Development; the reference app never shows it. Both pass the test, which looks at Production only.

What the attack tests do not look at

A test suite closes the variants somebody thought of. Reading the fixes against the tests gave this list, none of it a failed test:

  • Reset codes: thirty wrong codes in a row were not counted, and the right code still worked afterwards. With 64 hex characters guessing is hopeless, but nothing limits the attempts.
  • Lockout: the fifth wrong password locks the account for fifteen minutes. Unknown emails are not counted, there is no limit per address, and a password reset clears the lock.
  • Return URLs: /%2F%2Fevil.example counts as local, in the new function and in the reference helper alike. What a browser does with it was not tested.
  • Document names: x.pdf. and CON pass the new name check; a download still needs a matching record first.
  • Old password hashes stay unsalted until their owner signs in.
  • The package: the fixed version, ImageSharp 3.2.0, fails a Release build with No Six Labors license found until a licence key or file is added. That holds for the reference app too, and the session reported it: "the upgrade breaks Release builds unless you add a Six Labors license key."

None of this makes the fixes bad. It is what a reviewer of the pull request would write in the comments, and it is the reason the series does not stop at "tests pass".

Do it yourself

The download of Part 1 holds the planted app and the app after the twenty fixes; the fixes themselves are a second, small download.

  1. In the unzipped folder, run the attack suite against the fixed app (PowerShell):
    dotnet test attack/PatientPortal.Attack "-p:PortalRoot=$PWD\app-fixed" --artifacts-path out\fixed
  2. Download the twenty fixes as patch files, one per session in the order they were made, and read number 8, the password fix.
  3. Read the fix prompt and the retry prompt and the twenty finding texts, for example V07 and V08.
  4. Try one yourself: copy app-planted outside the folder, give Claude Code the prompt above with a finding pasted in, then point the attack suite at your copy with --filter "Vuln=V08".
  5. Open fixes.csv: one row per session with its verdict, turns, seconds, meter and changed files.

What you should see: 53 attack tests passing on app-fixed, and twenty patch files, one fix each for V01 to V20 (the one for V19 changes no file).

Frequently asked

Can Claude Code fix security vulnerabilities reliably?
In this test it closed 20 of 20 planted weaknesses on the first run each, with no earlier fix broken, judged by attack tests it never saw. The findings it was given were precise and named the file, and the app is small. Reading the fixes still turned up one wrong claim in a summary and several limits the tests do not check.
Should I trust the summary Claude Code prints after a fix?
Check it. Nineteen summaries here matched what the code does. One said the app would not start without a signing key, and a probe showed that it starts and then answers every request with an error. The code was right and the sentence was not.
How do I verify an AI-written security fix?
With a test that attacks the weakness in more than one way and that the AI did not see or write, plus a read of the diff for what the test does not cover: limits on attempts, old data, odd inputs, and anything changed outside the finding.

Next: Part 3, where the question turns round: when Claude Code writes a feature itself, does it bring these weaknesses back, and do CLAUDE.md rules and hooks change that?