### What I want to do

Security agents can run scanners, inspect code, test an endpoint, and produce a confident report. The problem is that the report does not always match what actually happened. A failed command may be described as a success. A tool result may be interpreted too strongly. Sometimes an agent reports a vulnerability without enough evidence to support it.

I want to study this problem for five months with Prof. Ramesh Karri at NYU Tandon. He has agreed to supervise the work if I secure external funding and NYU approves the visit.

I will build a test system that records an agent's tool calls, terminal output, files, errors, approval decisions, and final report. I will then check each important claim against that record. The central question is simple: can the agent show evidence for what it says it found?

The work will focus on defensive tasks inside isolated test environments. Examples include reviewing source code, triaging a known vulnerability, checking a configuration, validating a patch, and testing whether a defensive rule works. I will label claims as supported, unsupported, contradicted, or unclear.

After measuring the baseline, I will test four changes:

1. Require evidence for each important finding.

2. Give the execution record to a separate verifier.

3. Restrict tool permissions and require approval before sensitive actions.

4. Make failed commands and missing evidence appear explicitly in the final report.

I care about whether these changes improve accuracy, but also what they cost. A safeguard is not very useful if it makes the agent too slow or unable to finish ordinary tasks. I will therefore track accuracy, task completion, unsafe actions, runtime, and model cost together.

### Why I am working on this

This project comes directly from problems I encountered while building security tools.

During my internship at Indusface, I worked on LLM-assisted workflows that adapted emerging CVE information into working internal tests and generated targeted ModSecurity rules. I also studied firewall evasions. That work made one weakness hard to ignore: a polished answer from an agent can hide a failed command or a weak chain of evidence.

I later built [Pentest Code](https://github.com/Arpitpm23/pentestcode), an open-source agentic penetration-testing CLI. It uses isolated Docker execution, approval controls, persistent state, recursive subagents, and multiple model providers. I have also built a larger private multi-agent web-testing system. Working on these systems is what led me to the research question in this proposal.

My other relevant experience includes placing 39th among more than 9,800 participants in OffSec's Echo Response Gauntlet and helping lead COEP CyberCell. I am currently a final-year Computer Science undergraduate at COEP Technological University in India.

I do not have an academic publication yet. That is a real weakness in my application. My evidence is the software I have built, my security work, and a project plan that starts with a small pilot before scaling up. Prof. Karri's supervision will also help me turn applied engineering experience into careful research.

### What I expect to produce

If the project works, I plan to publish:

- the evaluation code;

- a set of safe benchmark tasks;

- a labelled collection of agent claims and their supporting evidence;

- results comparing the four safeguards with the baseline; and

- a report or manuscript explaining what worked, what failed, and where the conclusions do not generalize.

The public material will be designed for defensive research. I will use deliberately vulnerable local services and sanitized traces. I will not publish operational details when doing so would create unnecessary misuse risk.

The schedule is:

- January: define the threat model, tasks, evidence format, and labeling rules;

- February: build the first version of the harness and manually audit a pilot run;

- March: run the baseline experiments and label the dataset;

- April: test the four safeguards;

- May: repeat the strongest experiments, analyze the results, and prepare the public release.

A negative result would still be useful. The safeguards may fail, or the errors may vary too much across tasks to support a broad conclusion. If that happens, I will publish the result and describe exactly where the approach broke down.

### Budget

I am asking for up to USD 22,770:

- USD 16,000 for five months of living expenses in New York;

- USD 1,300 for return travel from India;

- USD 1,500 for health insurance;

- USD 700 for visa, SEVIS, and administrative costs;

- USD 1,200 for model APIs and compute; and

- USD 2,070 as contingency.

The minimum useful amount is USD 10,000. With partial funding, I would combine non-overlapping support from another funder or shorten the stay in New York and complete the remaining work remotely. I will disclose every award and will not charge the same expense to two funders.

### Other funding

I have not received cash funding for this project. Microsoft for Startups Founders Hub gave my earlier project Ideate/Initial Tier benefits in October 2024, including up to USD 1,000 in Azure credits. That support was non-cash and falls outside the last twelve months.

Applications for this project are pending with EA Funds, Emergent Ventures, BlueDot Impact, OpenAI, the AI Alignment Foundation, Foresight Institute, and several other programmes. One BlueDot application requests USD 5,500 for a smaller set of expenses. If more than one application succeeds, I will agree on separate budget lines with the funders before accepting overlapping support.