Project summary

I am applying for funding for a 12-month postdoc at MIT CSAIL on mechanistic interpretability for AI safety.

Mechanistic interpretability is mostly used post-hoc to improve our understanding of AI models, which is important in itself. The question I want to explore in this project is slightly different and it's whether the mechanisms we discover can also be used constructively, by using them to build models that are more interpretable-by-design.

My recent paper Learning Concept Bottleneck Models from Mechanistic Explanations (ICLR 2026), developed during a visit at MIT CSAIL, explored a first small step in this direction. We extracted concepts from the internal representations of vision models and used them to build concept bottleneck models, replacing the original prediction head with an interpretable concept-based surrogate. The approach retained most of the original model performance at ImageNet scale.

During the postdoc, I would study how far this idea can be pushed. In particular, I want to test whether it can be extended at the scale of language models and whether interpretable components can replace computation deeper inside a model rather than only the final prediction layer.

The postdoc would be hosted by Lalana Kagal, Principal Research Scientist at MIT CSAIL, continuing a collaboration that started during my MIT CSAIL visiting period in 2025.

What are this project's goals? How will you achieve them?

The postdoc would focus on three main questions. These questions define the research agenda, and I do not necessarily expect to fully solve all three within 12 months. I will prioritize them based on the results of the earlier experiments.

(i) Scaling M-CBM to language models. The first question is whether the idea behind M-CBM can transfer from vision models to language models. M-CBM worked well up to ImageNet scale, but this does not tell us whether the same approach will work at LLM scale. The task is substantially more complex, so I see this mainly as an empirical research question rather than something that we can assume will scale.

(ii) Replacing computation deeper inside the model. M-CBM replaces only the final prediction layer. Everything before that remains a black box. I would study whether the same idea can be applied progressively deeper inside the model. Vision models provide a useful controlled setting for these experiments. I would initially focus on deeper layers, where representations tend to be more semantically meaningful, and use sparse autoencoders (or equivalent tools) to recover interpretable concept representations at these layers. Human-understandable descriptions for concepts can be provided via automated interpretability methods, but would be treated as hypotheses whose fidelity needs to be tested. Then to replace computation, the question is whether the transformation between them can be represented by an explicit, interpretable surrogate that can also capture non-linear relationships (e.g., inductive logic programming, decision trees).

(iii) Interventions and corrections over the interpretable component. After part of the original computation is replaced by an interpretable surrogate, I would try to study whether this also provides a useful interface for interventions. For example, if the surrogate shows unsafe concepts or spurious correlations, these could be corrected or removed, while studying the effect on model behavior.

I do not expect that an interpretable algorithm could replace an entire transformer block in the same way that M-CBM replaces the final prediction head, at least not within the scope of this project. More realistically, we expect that part of the computation may be captured relatively faithfully, while a residual signal remains fully neural.

This differs from standard circuit discovery as there the goal is to recover the original model’s computation as faithfully as possible, since the final objective is to understand what the model is doing. Here, the final goal is slightly different. I want to use what we learn from the original model to construct a more interpretable version that can replace a significant part of it. The replacement does not need to reproduce the exact internal algorithm of the original model, as long as it remains interpretable and preserves most of its capabilities and performance.

How will this funding be used?

The total amount I am requesting is $110,000 for a 12-month postdoctoral research period at MIT CSAIL.

The budget will be distributed approximately as follows:

Gross stipend: $85,000

MIT health insurance: $8,000

Visa and relocation costs: $2,000

Research laptop: $3,000

Two conference trips: $6,000

Contingency and additional research expenses: $6,000

I am not requesting a separate compute budget as it will be provided by the institution.

I am setting the minimum funding bar at $55,000 so that if it is reached, I would be able to fund the first 6 months of the project. During that period, I would continue trying to raise funding for the remaining 6 months. If additional funding proves unfeasible, I would adjust the research plan accordingly.

Who is on your team? What's your track record on similar projects?

Team. I would lead the project as the primary researcher and Lalana Kagal would be my postdoctoral host and main mentor and collaborator.

Track record. I have already completed a smaller and closely related version of this project structure. From March to November 2025, I was a visiting PhD student at MIT CSAIL supported by an external fellowship. During that visit, I developed the project on learning concept bottleneck models from mechanistic explanations that is the direct starting point for this proposal, which led to a first-author ICLR 2026 paper. [Paper] [Code] [MIT News]

Before that, my research during my master’s and PhD also focused on concept-based and mechanistic interpretability. During my master’s, I worked on human-in-the-loop interpretability for vision models, which led to a co-first-author IJCAI 2024 paper. During my PhD, I developed Visual-TCAV, a concept-based interpretability method that grounds concepts spatially and estimates their contribution to model predictions, resulting in a first-author TMLR paper. I also have other peer-reviewed publications on interpretability, including work appearing in CVPR Findings, ACL Findings and various workshops.

Google Scholar: https://scholar.google.com/citations?user=Xwx26YgAAAAJ

What are the most likely causes and outcomes if this project fails?

This project is intentionally a high-risk research bet. In my view, there are mainly three plausible ways in which it may fail. I still think it is worth giving it a shot because, if it works, it could provide a path toward systems that humans can better understand and correct, by making parts of their computation explicit and directly editable.

(i) One major risk is scaling. Within vision, M-CBM remained effective when moving from smaller datasets with hundreds of concepts to ImageNet with thousands of concepts. We do not yet know what happens at the scale of hundreds of thousands of concepts that may be required for language models.

(ii) A second risk is error accumulation. Replacing one component may work well, while composing several interpretable concept layers may cause small errors to accumulate. This could put a practical limit on how much of a model can be replaced while preserving performance.

(iii) Another risk is information leakage. An interpretable model could achieve high performance not because it has recovered the concepts actually relevant to the original computation, but because continuous concept activations or other intermediate representations still "secretly" encode task-relevant information unrelated to their intended semantics. The resulting model could then only appear interpretable.

If the project fails, the main outcome would be a clearer understanding of the practical limits and trade-offs in how much of a model’s computation can currently be made explicit without sacrificing too much performance. It could also provide a test of the fidelity of mechanistic explanations. If we claim that a mechanism represents a specific concept or function, then replacing it with an explicit implementation of that concept or function should preserve the relevant behavior. If this consistently fails, it would suggest that the original interpretation was incomplete, i.e., that the mechanism was carrying additional information that the explanation did not capture.

How much money have you raised in the last 12 months, and from where?

I have raised $0 in the last 12 months.