Fund development of p(Doom)1

A nightmarishly hard AI safety strategy game about holding p(Doom) down. You can't win; you can only buy time.

Faithful Preference Learning : Cognitively-Aligned Post-Training

This project builds cognitively-aligned preference learning that interprets feedback the way human actually decide rather than as a reward to maximize.

Formalizing mathematical AI safety research in Lean

A hand-verified library of AI-safety theorem statements in Lean 4 with AI-generated proofs, building the skills to trust AI formalization.

Evaluating the Tacit Coordination Capacity and Decision Theory of LMs

A benchmark to empirically investigate: (i) the ability of models to tacitly coordinate with copies of themselves and (ii) which decision theory best explains t

Getting Split-Personality Training ready for deployment

Clearing barrieers to adoption for an existing ICML-published interpretability technique that can elicit latent knowledge from red teamed model organisms

Editing models inside their own geometry

One training-free geometry fitted to a model's residual-stream activations that reads a state, moves it, and tests whether the behaviour follows

BioRAG-Guard: Poisoning Evals for Biological AI Agents

An open-source benchmark and defense toolkit for testing whether corrupted biological databases can hijack retrieval-augmented AI agents used in genomics, prote

Help the Argentinian AI Safety Community Grow

The Argentinian AI Safety community (BAISH, baish.com.ar) is the largest in Latin-America. Support BAISH's growth, by providing funding for paying salaries.

Separatrix

Creating conditions for cooperative strategies to dominate adversarial ones among near-future AIs while we still can

Interpreting Latent Reasoning Models with Natural Language Autoencoders

Formally verified autoresearch for theoretical mech interp

LLM agents collaborate to discover and formally verify theorems about the internal computations of transformers, beginning with a simple pilot question: how man

Token taxes as a mechanism for reducing AI-driven power concentration.

A policy memo, co-authored with the Institute for Public Policy Research, resolving the open technical, economic, and legal questions blocking real-world implem

AI Safety Nepal

Five-week AI safety fellowship + 3-month project mentorship

Originality, Quantification, and Decomposition of Generalization.

Perform research to rigorously elucidate and quantify generalization versus memorization, and examine evidence of originality in LLMS.

Empirical econ - applied AI transition for quantitative econometrician

Fund demonstrated/rigorous quantitative researcher (already run reproduction/audit pipelines on published economics) for 6-month AI safety transition, shipping

What Factors Influence Chain-of-Thought Faithfulness?

Measuring whether CoT monitoring fails when an influence reaches an agent through a tool return rather than the user message.

Creating a better way to compile to transformer architectures

Shutdown evaluations for AI agents

Anthropology of Machines: Mechanistic Interpretability for Social Sciences

A hands-on book teaching social scientists to observe, intervene on, and validate what happens inside a language model.

LLM judges approve fabrications: 43–100% blind, 0–65% even with sources

The same grounding instruction leaves 0–65.5% leak depending on the judge; forcing per-claim counting closes it to 0.5%. 4 model families, 2,000 sealed calls.