1
AI safety for builders hackathon
ProposalGrant
Closes August 30th, 2026$0raised
$5,000minimum funding
$30,000funding goal
Offer to donate
31 daysleft to contribute
You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
Project summary
I am planning an online AI safety for builder hackathon in India - to build tools, products, etc.
One thing I observed was that there is lot of research & empirical work happening in AI safety, but much less focus on building products and tools.
This gap has been recognized by several organizations. For example:
- The case for AGI safety products (Apollo)
- RFP for safety-tooling founders (Halcyon)
- Design sketches for a more sensible world (Forethought)
- AI Safety Ideas — open project board (Apart)
- AI Safety software incubator (Surplus)
- Builders focussed fellowships like Generator Residency, MATS fieldbuilding streams, BlueDot fieldbuilding/incubator week, Gen Stream programme for generalists, etc
What are this project's goals? How will you achieve them?
There'll be three themes
1. AI tools for Accountability
2. AI tools for Epistemics
3. AI tools for India / Global South
See this list of proposals (collated after surveying various AI Safety + builders)
I think this will make AGI go well in 2 ways:
1. Bring AI safety work from research orgs / toy implementation to real world faster.
2. It is also a way to engage India where somehow AI safety ecosystem is very small even though it represents a significant share of the world's population and technical talent. Culturally Indian tech has a builders focus & it can be a huge momentum inducing event for them.
If this is successful - I (with friends / colleagues) plan to launch a full fledged AI safety for builders fellowship. See a proposal called AI kavach.
How will this funding be used?
[below is for the approved funds of $10,000; however if we get more funds, we'll be able to run a 8 weeks pilot cohort of AI safety for builders fellowship]
Compute, model APIs and software - $5,000
Organizer time - $2,000
Programme operations, platform and communications - $1,500
Accounting, payment and compliance reserve - $1,000
General contingency - $500
Who is on your team? What's your track record on similar projects?
Ankur Pandey is an AI/SaaS founder-builder and AI safety generalist. Involved in EA, rationality, and AI safety for 12+ years (going back to a MIRI workshop in 2013) and far more seriously over the past year across field building, education, projects, and fellowships. He started likely the first big freethinkers / rationalist community in India (in 2010).
Recent AI safety work spans education (BlueDot Impact, CAIS, Cambridge Digital Minds), fellowships like SPAR (working on AI reasoning failures), AI Safety Camp (working with Sahil, TJ, etc - integrity disorders in AI system), PIBBSS workshop.
Heavily involved with field building — organizing AI safety events in India, including an AI safety event at India AI Impact Summit 2026, building a comprehensive global directory of AI safety orgs. Consulted for Alice on AI security; currently consulting for Groundless.ai (Sahil K & team) on epistemic and AI-codependency risks.
Brings 15 years of applied AI and 3x founder. He had build tools reaching 100s of thousands of users - from fake news detection to general purpose AI agents - reaching 100s of thousands of users. Won an award by OpenAI (❗). Also has additional background in public-policy / humanities.
Soumya Jain is an AI safety researcher and research manager at CAISH working across technical AI safety and governance. She was previously a fellow at MARS. She has led interdisciplinary research, managed research programs, and supported multiple organizations with research management and grant development.
Previously she spent over five years in product management and data science across enterprise AI and fintech. She has led cross-functional teams building AI products, with experience spanning product strategy, AI systems, evaluations, governance, and deployment.
She recently judged Global South AI Safety Hackathon organised by Apart Research, mentored a researcher through the Paragon Policy Fellowship, and has been selected to mentor an agent governance project through the Sentient Futures Project Incubator. She also founded and facilitated Decoding the Mind, a discussion series exploring morality, phenomenology, self-reference and the structure of suffering.
What are the most likely causes and outcomes if this project fails?
1st two are more urgent
Teams don't finish. Some teams may not complete their projects or end up with rough, untested prototypes.
Scope is too ambitious.
Data or infrastructure isn't available.
Teams aren't matched well.
Participants drop out.
Mentorship is inconsistent.
Projects aren't useful. Teams build something, but it doesn't solve an important problem or isn't practical to use.
Projects are chosen for novelty instead of real need.
No clear end user.
Practical constraints are discovered too late.
Nothing gets adopted. A good prototype is built, demoed, and then forgotten.
No organisation takes ownership.
No one maintains or deploys it.
There's no clear path to continue after the hackathon.
How much money have you raised in the last 12 months, and from where?
The project has been funded 10,000 USD by grantmaking.ai, see link to the project page