I lead the adversarial robustness team at Anthropic, where I’m hoping to reduce existential risks from AI systems. I helped to develop Retrieval-Augmented Generation (RAG), a widely used approach for augmenting large language models with other sources of information. I also helped to demonstrate that state-of-the-art AI safety training techniques do not ensure safety against sleeper agents. I received a best paper award at ICML 2024 for my work showing that debating with more persuasive LLMs leads to more truthful answers. I received my PhD from NYU under the supervision of Kyunghyun Cho and Douwe Kiela and funded by NSF and Open Philanthropy. Previously, I’ve spent time at DeepMind, Facebook AI Research, Montreal Institute for Learning Algorithms, and Google. I was also named one of Forbes’s 30 Under 30 in AI.

Projects

Outgoing donations

Comments

CaML - AGI alignment to nonhumans

Ethan Josean Perez

11 days ago

I'm excited about work on CompassionBench and similar evals, to help better understand how well models are doing on animal welfare -relevant axes

🔥2

Transluce: Fund Scalable Democratic Oversight of AI

Ethan Josean Perez

8 months ago

I'm a huge fan of Transluce's work on investigator agents, automated red teaming and jailbreaking, and introducing fresh angles in interpretability/explainability. Their work has consistently been thought-provoking and among the best safety research out there. They also seem to have quite a talented and productive team as well.

➕

Forethought

Ethan Josean Perez

8 months ago

Forethought’s work has been some of the biggest influence on my safety research work, raising several new areas of research I and some of my collaborators hadn’t previously considered or prioritized. Keep up the amazing work!

🔥