Project summary

I ran a trading bot. I put in CAD 159. It made 26 trades, won 4, and lost about $6. The biggest trade was $70.54.

It saved a backtest for a test that never ran. The numbers matched nothing in its data, and it started recommending a live trade from that.

I built claimcheck to check an agent's numbers against the data the agent actually produced. It is MIT licensed, has no dependencies, and has 49 tests. The code is at https://github.com/meenadchisundaralinkamscooby-hash/claimcheck.

The money is to test that on real agent transcripts, publish how often agents invent results, and build a guard so invented results are not saved.

What are this project's goals? How will you achieve them?

1. Measure the problem. Months 1 to 4: collect public agent transcripts that come with the tool output or data behind them, and label the numeric claims as supported or unsupported, with paid contract labellers so the labels are not only mine.

2. Publish it. Months 5 to 8: release the labelled set as a public benchmark, with the error rates I find and how well claimcheck does on it.

3. Ship the guard. Months 9 to 12: release a memory guard that flags unsupported claims before an agent saves them, plus a written report and documentation.

All work is public from day one.

How will you know it worked?

The benchmark is public and other people can run it.

There is a published, reproducible number for how often agents state numeric results their own data does not support.

claimcheck has measured precision and recall on that benchmark, including the cases it misses.

The memory guard is released and at least one agent project other than mine has tried it.

How will this funding be used?

Full goal, US$150,000, 12 months full-time: my stipend 96,000 (including tax), contract labellers 24,000, compute and API costs 15,000, buffer 15,000.

At US$15,000: 6 months part-time. A smaller benchmark labelled by me, the error rates, and the memory guard.

At the minimum, US$7,500: about 3 months part-time. A small labelled set and a first write-up of the error rates.

Who is on your team? What's your track record on similar projects?

Just me. I live in Ontario, Canada. I am self-taught, with no degree, no paid developer job so far and no safety research background. I build with AI coding tools.

claimcheck exists and works on structured sources, but it is one commit with 0 stars and no users yet. I have no track record on a research project like this, so please weigh that.

This application was drafted with help from an AI assistant.

What are the most likely causes and outcomes if this project fails?

Public transcripts that include the data behind them may be hard to find, so the benchmark ends up small.

Labelling may take longer than planned.

claimcheck only checks numeric claims. If most invented results are not numeric, it catches a small share of the problem.

Nobody may adopt the guard.

In those cases there is still a public labelled set and an honest report on what did not work. Also, this reduces everyday agent errors more directly than it reduces catastrophic risk, so it may simply be a weak fit for safety funders.

How much money have you raised in the last 12 months, and from where?

US$0.

Pending, no decisions yet: the goose grant program (US$100,000, applied 2026-11-05, overlapping work: a goose extension that checks numeric claims against tool output), the EA Funds Transformative AI Fund (US$150,000, applied 2026-11-11, this same project), and a small Awesome Foundation Toronto application for an unrelated project.

If any of these is funded I will say so here right away and lower this ask.