Per my own work, I feel very excited about means of evaluating AI systems that are more "in the wild"/not having the constraints of existing benchmark-style tasks (self-contained, algorithmically scorable, etc.). From this perspective "AI village" is a very exciting project; I would like for this donation to be internally earmarked for the AI village (though not legally restricted; I want to leave room for the team's judgement, just to let them know that this is my vote). (I think I'm personally relatively less excited about AI digest, even though it looks very well executed, basically on neglectedness grounds.) There's a decent chance I put more of my pot to this project later. COI: nothing serious; I'm personal friends with Misha Yagudin and Eli Lifland, two of the founders of the organization that AI Digest is a project of. Donated another $20k for $40k total. I think it's crazy that we live in a world where AIs are well on their way to saturating ~all challenging technical benchmarks and yet they can't pull off basic experiments. It seems to me that exploring this harder would be very informative for learning about the capability profile of AIs. Right now it's unclear what we should learn from AI village because (1) I don't know how much I should trust quality of the set-up/elicitation, and (2) the effective sample size is tiny. One blocker for this happening is limited staff time. Another blocker -- totally speculative on my part, but my guess is shared between both other grantmakers and Sage/AI digest staff themselves -- is viewing AI digest as primarily a comms play and not a science play. I think AI village should primarily be seen as a science play (that currently isn't doing a great job at science but has a tonne of potential). I know @Austin is a bit concerned that this grant opportunity is not Manifund's comparative advantage/can easily be considered by other funders. I disagree -- it seems to me that traditional funders (and perhaps the staff behind AI village themselves) are being far too unambitious/unexcited about this project (on the basis of it being a comms play with limited existing comms traction), and I'm thrilled to put some of my Manifund pot towards nudging this situation in a better direction. Recommended my remaining regrantor budget, $31.7k, for $71.7k total. I continue to stand behind my last comment (and hopefully will write more later). I think that something like AI village is one of the most interesting science (not comms!!!) plays in AI today. I'm disappointed that this continues to be a non-consensus pick.

This seems like solid work that I would love to see continued! Your "how fast is AI improving" explainer seems great and I would love an improved version to exist. The chain of thought faithfulness explainer also sounds great. So you seem to have good taste in projects! I'm just donating a token regrant for now since your desired budgets are large enough that I'd want to understand your funding situation in more detail before doing anything larger, but wanted to make a gesture of support.

I'm donating $10k, and I think there's a good chance I'll come back and end up donating more. I think that AI Digest has done great work and will be able to put marginal funding to good use. Stuff I'm particularly excited about in the past: - The Agent Village. I think getting real world experience seeing how AI agents will interact with tasks outside the narrow confines of a benchmark suite is a neglected form of capability evaluation. - Various explainers, especially the one of the METR graph. Main reservations: - Nothing jumps out to me. Conflicts of Interest: No COIs.