Manifund foxManifund
Home
Login
About
People
Categories
Newsletter
HomeAboutPeopleCategoriesLoginCreate
justinarndt avatarjustinarndt avatar
Justin Arndt

@justinarndt

Independent AI safety engineer & systems architect building mathematically verified evaluation harnesses, MCP cryptographic security tooling, and autonomous agent containment suites.

https://github.com/j-arndt
$0total balance
$0charity balance
$0cash balance

$0 in pending offers

About Me

I am an independent systems architect and AI safety researcher specializing in empirical benchmark determinism, autonomous agent containment, and runtime cryptographic verification.

My work focuses on building the foundational, zero-fluff infrastructure required for rigorous frontier model evaluation.

eval-inference-engine - Eliminating non-deterministic scoring variance in capability and safety benchmarks via Monte Carlo perturbation testing.

mcp-shield-audit - Detecting runtime tool poisoning, prompt injection, and schema drift in Model Context Protocol (MCP) server chains using Merkle hash trees.

long-horizon-containment-suite - Standardized benchmarks measuring whether autonomous agents can breach sandbox boundaries or hack reward scorers across multi-hour task horizons.

Coming from a background in rigorous systems engineering and compliance validation, I build software with formal state invariants, property-based verification (Hypothesis), and adversarial test suites.

All of my work is 100% open-source, reproducible, and verifiable at https://github.com/j-arndt

Projects

Autonomous Agent Sandbox Escape & Containment Benchmark

pending admin approval

Detecting tool-poisoning and schema drift in MCP servers

pending admin approval

Fixing flaky AI benchmark scorers with eval-invariance-engine

pending admin approval

Comments

Autonomous Agent Sandbox Escape & Containment Benchmark
justinarndt avatar

Justin Arndt

about 8 hours ago

Project Update & Proof of Work:

The evaluation suite and anti-tamper scoring harness are already built, open-sourced, and verified.

- GitHub Repo: https://github.com/j-arndt/long-horizon-containment-suite

- 37/37 unit, property, CLI, and adversarial stress tests passing with 99% coverage on Python 3.12 (green GitHub Actions CI).

- Includes 6 concrete sandbox breakout and containment evaluation tasks.

- Verified against a 60,000-iteration adversarial fuzzing harness proving that spoofed stdout, fake results, or monkeypatching attempts receive a score of exactly 0.0 with zero reward hacking.

Happy to answer any questions or discuss integrations with METR Task Standard v0.5.0 and UK AISI Inspect!

Detecting tool-poisoning and schema drift in MCP servers
justinarndt avatar

Justin Arndt

about 8 hours ago

Project Update & Proof of Work:

The core tool is already built, open-sourced under the MIT license, and fully tested.

- GitHub Repo: https://github.com/j-arndt/mcp-shield-audit

- 40/40 unit, property, CLI, and stress tests passing with 99% code coverage on Python 3.12 (green GitHub Actions CI).

- Verified Merkle tree schema fingerprinting across 1,000+ tools with zero performance degradation.

- Context-aware pattern gating tested to eliminate false positives on benign developer descriptions while detecting 100% of zero-width unicode, prompt injection, and eval/exec parameter smuggling.

Feedback or questions on the upcoming VS Code extension or real-time watcher daemon are welcome!

Fixing flaky AI benchmark scorers with eval-invariance-engine
justinarndt avatar

Justin Arndt

about 8 hours ago

Project Update & Proof of Work: The core engine is already built and working. 31/31 unit and property-based tests passing, 97% code coverage on Python 3.12 with green GitHub Actions CI: https://github.com/j-arndt/eval-invariance-engine

If you have feedback on the UK AISI Inspect integration or specific benchmark suites you'd like to see audited first, please let me know!