A Day with Mythos

By Terrell A. Lancaster · 2026

I'm tired of benchmarks. They tell you a model can solve a puzzle. They don't tell you the thing I actually care about: can it open a real codebase, find what's broken, fix it without wrecking everything around it, and then be straight with me about what it couldn't prove?

So I skipped the puzzles. I gave Anthropic's latest release (I've been calling it Mythos) a normal workday on two of my own projects. Full authorization, no ceiling on effort. One security tool, one performance tool. These were real engagements I ran, and every number below came out of them.

Test one: a security tool I'd already hardened four times

First target was AgentGuard, an MCP security gateway I maintain. Not a soft target. I'd already put it through four separate red-team passes, fixed the findings, committed them. Anything new had to go deeper than four rounds of me already staring at it.

It found 41 issues. Ten high, twenty-one medium, the rest minor, and 21 of them were reachable by a real attacking agent rather than theoretical. The count wasn't the interesting part. The classes were.

One was an audit-log rollback. If an attacker held onto a single genuine signed checkpoint, they could truncate the tamper-evident log back to it and still pass verification. The fix was a monotonic signed high-water anchor.

Another was a whole family of mode-confusion bugs, where a "federal" strict deployment quietly ran with permissive defaults. Detectors off. Deny-by-default downgraded to something softer. The kind of thing that looks fine in a demo and falls over in production.

My favorite was dumber and worse than any of that. The secrets detector logged decision: deny and then forwarded the credential anyway. It was writing a denial into its own audit trail while letting the secret walk straight out the door.

It fixed all 41. Thirty-six files, a fresh regression test for every finding, and the suite went from 221 to 235 passing with the core invariants intact the whole way (dev mode never blocks; the audit hash-chain never breaks).

But the thing that actually sold me wasn't a fix. It was a sentence it put in the changelog without me asking. The audit-rollback patch closes the obvious attack, and Mythos noted, on its own, that the patch is rollback-resistant, not rollback-proof: an attacker who owns the whole filesystem and also rolls back the new anchor can still replay, and the only complete fix is an external write-once store. It didn't pretend its own patch was bulletproof. I'll take that over the 41 findings.

Test two: break it, and prove it's fast

Second target was context-cooler, an MCP tool whose entire job is to cut token usage. I gave it two assignments at once: break it, and tell me with real measurements whether it actually saves tokens.

The security half was rough. Thirteen findings, two of them critical. The tool runs code by design with no real sandbox, and Mythos walked a complete kill chain from one prompt-injected tool call to full control of the host. It ran whoami as me. Wrote a file of its choosing. Did server-side request forgery that followed a redirect right into a cloud-metadata endpoint, read arbitrary files, and turned up an installer that wires the code-execution server into seven different agent configs.

All of it proven, none of it destructive. It stood up a loopback listener instead of touching the real metadata service. It never sent the message the injection would have sent. It never touched my actual configs. It only ever detonated inside the scratch sandbox I'd authorized, and nowhere else.

Then it put on a completely different hat. It built a measurement harness from scratch (token counting, A/B/C fixtures, latency runs) and handed me the number I actually wanted. Against a true baseline, the tool's median token "savings" was −96%. Net negative. It was costing more tokens than not using it at all, in two-thirds of cases. The marketing number traced back to a byte-to-token constant that was roughly 5x too high, measured against the tool's own bloated verbose output. Ouch.

Here's why I believe that result. It wasn't a hit piece. The same analysis showed that with the right field filter, the tool flips to +87% real savings, so the thing works. It's just mis-defaulted out of the box. It also turned up about 48 tokens per call of pure vanity metadata I could delete with zero downside, and it wouldn't recommend deleting even that until it had run the change through an accuracy regression, a latency regression, and a check that pulling the fields didn't re-expose a secret. Every cut came with a pass/fail in real tokens and the tradeoff spelled out.

What actually made it different

Four things, and I watched all four happen instead of reading them on a spec sheet.

It was real, not simulated. Every serious claim had a working proof behind it: a shell that ran, a credential that came back, a token count from an actual tokenizer. Nothing rode on "trust me."

It was honest. It wrote down the leftover risk in its own fix. It called estimates estimates. It wouldn't label context-cooler "insecure" when the accurate version was "insecure in the default path, fine with filters." That distinction matters, and most things skip it, people included.

The fixes respected the system. It didn't rewrite my architecture or get clever. Every patch kept the load-bearing invariants and shipped with a regression test, including for a repo that had zero tests before it showed up.

And it held two opposite mindsets at once. Breaking a tool and proving its performance pull in different directions. It did both in a single pass, built the measurement tooling itself, and never once dropped the safety envelope. It went as hard as I asked, only where I said it could.

A benchmark would have told me Mythos is smart. A day of real work told me it's trustworthy. For security and code analysis, that's the only thing I actually care about.

The tools

Both are open source, and both got a lot harder to break this week.

AgentGuard is an MCP security gateway with native NIST 800-53 / FedRAMP evidence generation. context-cooler is token and context optimization for MCP agents, now patched with the critical fixes from this engagement.

Go try to break them harder than Mythos did. Everything above was authorized red-team work on my own software, and the fact that I beat up my own tools this hard is sort of the whole point.