Case study one, method: what separates an engagement from a scan.
In the previous post we published the raw output of a Lory engagement: 38 findings across twelve deliberately vulnerable applications, 26 of them in a single 39-minute run.
The count is the least interesting number in that post. Anyone can raise a finding count. Lower the confidence threshold and the number goes up. What matters is how a finding was reached, and whether the tool understood what it was looking at or matched a string.
So here are four in full detail. Each one is a case where a signature-based scanner has a specific, explainable reason to produce the wrong answer.
The sanitiser that does not sanitise
A blog application's sign-in logic did the responsible-looking thing. It ran user input through a filter before building the query.
This is the trap. To a tool looking for the taint pattern "request variable flows into query string", the filter call sits in the middle of the path and looks like a sanitiser. Many static analysers treat a recognised filter function as a taint barrier and stop there. The finding is suppressed, and the suppression is the bug.
FILTER_SANITIZE_FULL_SPECIAL_CHARS encodes HTML special characters (angle brackets, ampersands, double quotes) into HTML entities. It is an output encoder for HTML contexts. It has nothing to say about SQL syntax, and critically, it does not escape the single quote in a way the database will not interpret. A payload that avoids <, >, & and " passes through completely untouched.
Lory reached the correct conclusion, that this particular filter does not prevent SQL injection, and then proved it rather than asserting it. She also found the identical pattern in two other files in the same application, signup-logic.php and add-user-logic.php, which is the part that turns a finding into a fix: you do not want to patch one call site and ship.
The general lesson. A sanitisation function is only meaningful relative to the context it defends. HTML encoding does not stop SQL injection. SQL escaping does not stop XSS. Neither stops command injection. The word "sanitize" in a function name is not a security property. It is a name.
The fix is not a better filter. It is not using a filter at all:
Two dull bugs that add up to remote code execution
Scanners evaluate findings independently. A check runs, returns a verdict, and knows nothing about what the other checks found. That model is why chains get missed, and chaining is most of what a human pentester actually does.
Here, a password-change endpoint did something unusual after a successful change: it built a base64-encoded shell command and redirected the browser to a logging endpoint, passing the command as a query parameter.
Taken separately: an open redirect (low), and an unauthenticated endpoint (a misconfiguration). Taken together: unauthenticated remote code execution, reachable directly by calling the logging endpoint with any command you like.
The detail that makes this a good example of reasoning rather than pattern matching is what Lory concluded from the redirect's existence. The redirect proves the execution endpoint is intended to work that way. It is not dead code, not a leftover, not a file someone forgot to delete. It is load-bearing, which changes both the severity and the remediation advice.
She also noted a consequence nobody designs for: because the command travels as a GET parameter, it lands in server access logs, in browser history, and in the Referer header sent to any third party the next page talks to. The command string leaks to places nobody thought about.
Remediation, as written in the finding: delete the execution endpoint entirely. Log password-change events with a server-side logging call, not by redirecting a user's browser through an endpoint that runs commands. Never build a logging workflow out of a user-facing URL that executes.
Point her at your own code
Set your scope per asset. Lory works the target and files findings into your dashboard.
Template injection that arrived through the database
Server-side template injection is usually hunted where it is easy to hunt: a URL parameter reflected into a template. Send a probe, look for evaluation, done. That approach finds SSTI in one request, which is why tooling is built around it.
This one was written in one request and evaluated in a different one.
Post bodies are stored, then later passed to Twig's dynamic template compiler at render time. Autoescaping is off, and the environment has a system function registered. Anyone who can create or edit a post, through the admin interface for example, can store a template expression that reaches OS-level command execution when someone else views the page.
Lory's write-up made a point worth repeating: even without the registered system function this is still critical. {{ dump() }} leaks internal application state, and the standard Twig sandbox escape techniques still apply. Removing the dangerous function would make the finding less immediately catastrophic without making it safe.
Finding this requires tracing a value from where it is written, through storage, to where it is finally evaluated, across separate requests, separate files, and a database round trip. There is no single request that demonstrates the bug.
A private key that mattered, and the file that proved it
Secret scanners find private keys constantly. Most of those alerts are noise: test fixtures, expired material, deliberately fake keys committed as examples. The signal problem in secret detection is not finding key-shaped strings. It is working out which ones matter.
Lory found an unencrypted OpenSSH RSA private key committed at blogger/src/ssh_keys_inject/id_rsa.bak. A pattern match gets you that far. She then did three more things:
- Found a plaintext note under an employee directory that identified whose key it was and stated where the corresponding public key had been installed.
- Found that the Dockerfile copies the key into the image at build time, so it ships with the container rather than merely sitting in history.
- Found that the same Dockerfile grants that user passwordless sudo:
jimmy ALL=(ALL) NOPASSWD:ALLon line 37.
Those four facts together are not "a private key was committed". They are: anyone with read access to this repository can authenticate as a specific user who has unrestricted root, on every container built from this image. The severity, the blast radius, and the urgency all come from the correlation, not from the key.
The remediation reflects that too, and in the right order: revoke the key from every authorized_keys it may have reached, then purge it from git history with git filter-repo or BFG, force-push and invalidate forks, then move to runtime secret injection, and separately remove the passwordless sudo grant. Rotating without purging leaves it in history. Purging without rotating leaves it working.
What these four have in common
None of them is exotic. Every one is a well-documented vulnerability class with a CWE identifier and a decade of literature. The difficulty is not the class. It is the context.
| Finding | Why a scanner gets it wrong |
|---|---|
| The sanitiser | A recognised filter function in the taint path reads as a barrier, so the true positive is suppressed. |
| The chain | Checks are evaluated independently. Neither half is severe alone, and nothing joins them. |
| Stored SSTI | The payload is written in one request and evaluated in another. No single request demonstrates it. |
| The key | Pattern matching finds key-shaped strings but cannot tell which ones are live or what they open. |
In each case the missing ingredient is the same: holding more than one fact in mind at once. That a filter exists and what that specific filter does. That a redirect exists and what it points at. That data is stored and where it is later evaluated. That a key exists and who it belongs to and what that account can do.
That is what we mean when we say Lory runs engagements rather than scans. Not that she is faster, but that she carries context across findings.
The honest caveat. These four came from deliberately vulnerable training applications, where the bugs were planted for exactly this kind of exercise. The reasoning is the point, not the difficulty. We also ran the same agent against our own production platform, where nothing was planted, and it found real issues there too.
See what she finds in your stack
You draw the scope, per asset, in and out. Lory works the target. A human reviews every finding before it reaches you.
Start an AI pentest Explore TalonThe Lory case study series
- Part one: 26 findings in 39 minutes, the full results
- Part two: Four findings a scanner would have walked past (you are here)
- Part three: We pointed it at our own platform
Notes on method
- All four findings are taken from our own engagement records. Descriptions, line references and remediation guidance are as written in the findings, condensed for readability.
- Code excerpts are simplified to isolate the relevant behaviour. Variable names and structure follow the originals.
- The targets were deliberately vulnerable training applications. See the caveat above.
