Manifund foxManifund
Home
Login
About
People
Orgs
Categories
Newsletter
HomeAboutPeopleCategoriesLoginCreate
🦄
🦄
Caroline Ellison

@carol

132

$8,019total balance
$8,019charity balance
$0cash balance

$0 in pending offers

Outgoing donations

Sunlight Fund
$10
11 days ago
By Caroline72Following38
Robustifying Alignment Techniques to Survive RLVR and Continual Learning
🦄

Caroline Ellison

2 days ago

oh btw if you haven't yet, might be worth filling out https://survey.chipsforhumanity.org/

Robustifying Alignment Techniques to Survive RLVR and Continual Learning
🦄

Caroline Ellison

2 days ago

Thanks for the thoughtful responses! This makes sense to me: "I see the primary role of independent researchers then as exploring somewhat-weird bets on alignment techniques and generating enough evidence to convince frontier labs to pursue them, and thereby moving those ideas inside-the-overton-window."

Given all this, I'd like to fund you for something through Sunlight! Happy to do either this project, or if you want to write up a proposal (can be very brief/low-effort since we've already talked about it and I don't want to take too much of your time) on a continuation of the data filtering work you've been doing on the BlueDot grant I can fund that instead. I lean slightly toward being more excited about things less likely to be covered by frontier labs (so less excited about the RLVR part of the proposal) but also think you should mostly work on whichever direction you're most excited about at the moment.

Tail End Films: 3 AI Risk Films for Streaming by 2027
🦄

Caroline Ellison

2 days ago

Can you share any more about the process of selling Making God? I'm most interested in how likely it is to sell to a major streaming service, or if you can share anything about where you are in talks and who you might sell it to.

Tail End Films: 3 AI Risk Films for Streaming by 2027
🦄

Caroline Ellison

2 days ago
Admin note

Approved! I'm excited for Making God, and these further project ideas are continuing in the same charitable mission.

Robustifying Alignment Techniques to Survive RLVR and Continual Learning
🦄

Caroline Ellison

3 days ago

I had a call with Ethan today--he seems like an accomplished researcher who is likely to produce good research, but I'm still confused whether this is the kind of thing it makes sense for Sunlight to fund.

My understanding is that Ethan's research agenda is basically relatively legible prosaic alignment research of the type that frontier AI companies are also quite motivated to get right. Given that, and the fact that they have billions more dollars than us, it seems like Sunlight's comparative advantage should be working in other areas--eg fieldbuilding, media, advocacy, and policy--or at the very least, making kind of weird bets when we do fund technical alignment research.

On the other hand, the implicit conclusion of the above reasoning might be that Ethan should work at say Anthropic, where he'd have access to much more compute as well as Anthropic IP. But I don't generally go around advising people to work at Anthropic--if anything, I think on the margin more people should quit Anthropic. So maybe it's actually worthwhile for funders like Sunlight to fund researchers like Ethan because it keeps them independent.

I was talking to @Austin about all of this and he was like "let's have this argument in public" so--thoughts?

(And Ethan--thanks for taking the time to call! I appreciate you bearing with us.)

How Far Do Instilled Values Generalise? Scaling Laws for SDF
🦄

Caroline Ellison

4 days ago

I’m recommending a $10k grant through Sunlight. Liu and Justas seem like promising and dedicated undergrads, the proposal seems well thought through, and they’ve found an experienced mentor.

I see the biggest benefit of the grant being to help upskill Liu and Justas and encourage them to continue in AI safety research, with the secondary benefit being that it might produce some useful research. Overall, it seems like quite a good deal at $10k.

PauseAI Canada: national expansion & core operations
🦄

Caroline Ellison

5 days ago

I spoke with Muhammad, the lead of Pause AI Toronto, about this grant.

I’m not sure how important advocacy in Canada or other middle powers is, but it seems worth devoting at least some resources to, and PauseAI Canada is quite under-resourced right now. (ControlAI has been working with Canadian legislators, but I’m not aware of other organizations doing more widespread advocacy/movement-building in Canada.)

Muhammad seems to be engaging with a lot of people and ideas in AI safety and thinking critically about the most effective ways PauseAI Canada can improve the world. I see a lot of the value of this grant coming not just from persuading the average Canadian or leading to Canadian influence on an international treaty, but from more generally empowering the PauseAI Canada volunteers.

Retroactive funding: agent-glovebox, open-source sandboxing for AI agents
🦄

Caroline Ellison

5 days ago
Admin note

Approved--this sounds like great and important work!

Principled Agents
🦄

Caroline Ellison

6 days ago

Rubi Hudson seems like a strong researcher with endorsements from many. He has two single-author top-conference papers, and Max Harms called his work “perhaps the most clear advance” in corrigibility in the past year, along with other endorsements you can see in the comments on this project and his previous one. Overall, the team seems strong and well-posed to execute.

I like that they have a clear agenda and explain how their research directions fit into their plan for AI alignment. I don’t know how likely the agenda is, and it’s probably a long shot.

$10k is far short of their goal funding, but it gives them a little more time to find major funders in addition to directly funding a bit more research time.

The biggest argument against the grant that I see is that the opportunity cost of the researchers is valuable, and they’d be doing more valuable work at a different organization focusing on a different research agenda. I don’t have an object-level take on that, but I have a meta-level take that diversity of organizations and agendas is good, and it’s good to encourage talented people to at least try founding ambitious projects rather than working at existing organizations.

AI safety content creator in Germany: go full-time (scale existing 100k channel)
🦄

Caroline Ellison

6 days ago

Niklas has an impressive number of followers and views for something he’s been doing as a part-time project: over 130k followers in total, and a viral video with 2 million views.

I don’t speak German or watch short-form videos, but I watched some of his videos (with AI-translated captions) and they seemed pretty good. I appreciate that he does things like posting all the sources behind his videos on his website. One downside is that his content isn’t that focused—a lot of his videos are about mundane harms of AI. Maybe this is a good way to introduce people to existential risk, but I tend to think it might turn off or memetically inoculate people who (rightly) think AI offers a lot of mundane utility that outweighs its mundane harms.

He’s still well below his minimum funding even with a grant of $10k, but I hope he’ll get funding from other sources (I know he is in discussions with a few at the moment). Funding him for a year seems like a good experiment, since it will give him time to see how much going full-time makes a difference and whether he can become self-sustaining through revenue.

Checking jailbreak judges on cases people disagree about
🦄

Caroline Ellison

9 days ago

This looks like a reasonable project, and you have an impressive record for someone so young! Right now the proposal is being flagged for being AI-generated--if you rewrite it in your own words, you might have a better chance of being seen by donors and getting funded.

Budget for events aiming to "mainstream" EA in Sweden
🦄

Caroline Ellison

10 days ago

The grantee returned most of the grant as it was unused, so I've refunded it proportionally to donors' Manifund accounts.

Support the Effective Altruism community in Somalia
🦄

Caroline Ellison

10 days ago
Admin note

Approved! Congratulations on your progress so far and good luck.

Covert informants as an AI control protocol: from pilot to main study
🦄

Caroline Ellison

13 days ago

This seems like a clear and well-scoped proposal! FYI funders are less likely to see your project if it's flagged as AI-written--if you're able to rewrite the proposal in your own words, that may help with visibility. (The AI percentage is automatically recalculated on proposal edits.)

AI safety beyond English: send our 4-language paper (and me) to Sydney
🦄

Caroline Ellison

20 days ago

Unfortunately Fatma updated that her visa application to travel to Australia was denied, so I've returned the funding to @gnutella

AI safety beyond English: send our 4-language paper (and me) to Sydney
🦄

Caroline Ellison

26 days ago
Admin note

Approved! I'm glad you'll get a chance to present your project in person and benefit from the network at the conference.

Principled Agents
🦄

Caroline Ellison

about 1 month ago
Admin note

Approved--this sounds like exciting and neglected work!

Human Access: Animated Web Series Communicating ASI Risk
🦄

Caroline Ellison

about 1 month ago
Admin note

Approved! The creators seem to have a good track record of creative projects and a plausible sense of what might appeal to audiences.

Shallow review of AI safety 2026
🦄

Caroline Ellison

about 1 month ago
Admin note

Approved--excited for the 2026 version!

Research into relative effects of different pacing interventions
🦄

Caroline Ellison

about 1 month ago
Admin note

Approved! This seems like an important question, and happy to trust in Gavin's judgment of grantees.

NadIntellect Lab: first live test of a constrained information channel
🦄

Caroline Ellison

about 1 month ago
Admin note

Approved! Always happy to support Keri's grantmaking, and this seems like a cheap test that could produce interesting results

Emergency travel funding to attend EA Global: New York 2026
🦄

Caroline Ellison

about 1 month ago

@sheikheddy I closed the project, so the funds are now returned to the donors!

The Floor That Cannot Be Lowered: verifiable containment for AI agents
🦄

Caroline Ellison

about 1 month ago

@biosbilgisayar done!

BASE Research at EA Global: New York City 2026
🦄

Caroline Ellison

about 1 month ago
Admin note

Approved! This seems like something that may increase BASE's effectiveness.

AURA: Metric 3D facial reconstruction from phone photos
🦄

Caroline Ellison

about 1 month ago

@atilgandev Thanks for addressing the issue! Noting that it has now been resolved.

AURA: Metric 3D facial reconstruction from phone photos
🦄

Caroline Ellison

about 1 month ago

Hi Serhat — we’ve found that several accounts voting on your AURA project appear to be controlled by you or created on your behalf, so we’ve banned those accounts. Manifund relies on comments reflecting independent views from real people, and using additional accounts to boost a project or downvote other projects isn’t allowed. Your project can stay up for now, but any further sockpuppet activity will result in your account and project being removed.

Video Game Eval Compilation and Analysis
🦄

Caroline Ellison

about 2 months ago

Approved! Happy to extend a lot of trust to our regrantors' judgment as always, and this project sounds like a cheap way to potentially produce useful insights.

AISA Table top exercises for AI Governance
🦄

Caroline Ellison

about 2 months ago

Approved! AISA seems to have a good track record, and understanding of AI by policymakers in Asia is a very important cause.

Swarm incidents tracking platform
🦄

Caroline Ellison

about 2 months ago

Approved! This is an important area, and from the website so far I think this has the potential to be a clear explainer and useful resource.

FloraForge: Building a Reusable Genetic Platform for New Flower Traits
🦄

Caroline Ellison

about 2 months ago

Approved! This falls within Manifund's charitable mission by contributing to public scientific research.

Research Disease from Vulture-Population Crashes, and Prevention
🦄

Caroline Ellison

2 months ago

Interesting project--approved!

Safeguarding DNA Synthesis from Biological Risks Posed by Advanced AI
🦄

Caroline Ellison

2 months ago

@amiralia Hi, yes this was sent by ACH on November 2.

Core funding for PauseAI UK
🦄

Caroline Ellison

2 months ago

Approved! Advocacy work to avert AI risks aligns with Manifund's charitable mission.

Distinguishing progress in data and algorithms
🦄

Caroline Ellison

3 months ago

Approved; this sounds like an informative project!

safely.bio: Customer screening for DNA synthesis providers
🦄

Caroline Ellison

3 months ago

Approved! Getting adoption for something like this seems like a clear win, and we're excited about the team.

Developing materials on forecasting for governments
🦄

Caroline Ellison

3 months ago

Canceled the pending offers here! The grantee wrote: "I've completed the work this grant would have funded, but secured a better funding route for it, so I won't be accepting this grant. Please regrant these funds, or if necessary return them to the funder. Kindly treat this grant as withdrawn and closed on my end"

Safeguarding DNA Synthesis from Biological Risks Posed by Advanced AI
🦄

Caroline Ellison

3 months ago

@amiralia Yep I see the ACH info, thanks! We'll send the payment today and then you should receive it within a few days.

Faithful Preference Learning : Cognitively-Aligned Post-Training
🦄

Caroline Ellison

3 months ago

Approved! Happy to support grantmaking.ai as always.

Safeguarding DNA Synthesis from Biological Risks Posed by Advanced AI
🦄

Caroline Ellison

3 months ago

@amiralia yes! you or they can fill out this form: https://airtable.com/appOfJtzt8yUTBFcD/shrI3XFPivduhbnGa it asks for ACH details (bank account name, address, routing number, account number) as well as Manifund username and project link.

If that doesn't work feel free to just email the bank account info (hi@manifund.org) and we can handle it.

Compute funding for five mentored AI safety research projects
🦄

Caroline Ellison

3 months ago

Approved! These sound like interesting research projects, and funding mentee projects is a good way to fieldbuild as well as directly producing research.

Exploring the Benefits of Alignment Pretraining
🦄

Caroline Ellison

3 months ago

Approved! This sounds like a worthwhile research direction.

PauseAI Australia attendance PauseCon London
🦄

Caroline Ellison

3 months ago

Approved! Happy to support grantmaking.ai as always

Interpretability research spring 2026
🦄

Caroline Ellison

3 months ago

Approved! Happy to support Neel's regranting as always.

Preserving the Human Veto
🦄

Caroline Ellison

3 months ago

Approved! Excited to see how the pilot program goes.

Adversarially robust workload classification using hardware sensors
🦄

Caroline Ellison

3 months ago

Approved! Love to see research on verification.

Reading China's AI Registry: A Public Monitor
🦄

Caroline Ellison

3 months ago

Approved; this sounds like a great resource!

Frontier AI Election Evaluation Framework
🦄

Caroline Ellison

3 months ago

Approved! Excited to see work on democracy and civil society in the age of AI.

Scaling adoption of synthesis screening outside the US and EU
🦄

Caroline Ellison

3 months ago

Approved! This seems like a great use of funds, comes highly endorsed, and fits squarely within Manifund's mission. Good luck!

Safeguarding DNA Synthesis from Biological Risks Posed by Advanced AI
🦄

Caroline Ellison

3 months ago

@amiralia Hi! Sorry for the delayed response--we just rolled out a feature that allows grant agreements to be signed externally by organizations. I've reset your agreement so you can redo it as an organization: on https://manifund.org/projects/safeguarding-dna-synthesis-from-biological-risks-posed-by-advanced-ai/agreement, enter Georgia Tech Research Corporation's details and the email of GT's authorized signatory, and they'll receive a link to review and execute the agreement (they'll get a copy for their records). Once that's signed, you can withdraw the funds to GTRC at https://manifund.org/withdraw.

Let me know if you need the agreement as a PDF instead, or if you run into any issues!

Development of a Cautionary Tale Feature Film about Gradual Disempowerment
🦄

Caroline Ellison

3 months ago

@Petr_Salaba awesome thanks; I appreciate the update!

Metagame 2026
🦄

Caroline Ellison

3 months ago

Approved! Metagame provides upskilling as a space for people to learn and allows people speakers to broadcast ideas and people to make connections they wouldn't have otherwise. Donations allow them to provide financial aid.

ILINA Junior Research Fellowship
🦄

Caroline Ellison

3 months ago

@Abungu- Great!

ILINA Junior Research Fellowship
🦄

Caroline Ellison

3 months ago

Approved; excited to see more diverse fieldbuilding!

AI Safety Hong Kong (AISHK)
🦄

Caroline Ellison

3 months ago

Approved! This sounds great; excited for more geographically diverse fieldbuilding and especially in Hong Kong.

BioRAG-Guard: Poisoning Evals for Biological AI Agents
🦄

Caroline Ellison

4 months ago

Approved--good luck with the project!

Getting Split-Personality Training ready for deployment
🦄

Caroline Ellison

4 months ago

Approved--very interesting work!

Systems that "give a damn"
🦄

Caroline Ellison

4 months ago

@StevePetersen that's okay! We don't really have a system for this: I think your report was sufficient, it was just overlooked on our end.

Systems that "give a damn"
🦄

Caroline Ellison

4 months ago

As Steve was able to fund this project separately and stated that he wouldn't spend money from this application, I've returned the funds to the donors!

Develop an accessible, low-cost system for single-cell imaging in multiple regions of freely moving organisms
🦄

Caroline Ellison

4 months ago

Update: since Andrew was unable to accept the funds, I returned the donation to the ACX grants account.

AI Safety for Builders - micro fellowship
🦄

Caroline Ellison

4 months ago

Approved! This seems like a good way to try to engage AI safety talent in India.

Evaluating the Tacit Coordination Capacity and Decision Theory of LMs
🦄

Caroline Ellison

4 months ago

Approved! This sounds like interesting research.

Bayesian modelling of LLM capabilities from evals
🦄

Caroline Ellison

4 months ago

I'm curious if there were results from this project; thanks!

Acausal research and interventions
🦄

Caroline Ellison

4 months ago

@chi-nguyen It looks like this report is missing the text; I'm curious what the latest on this project is!

Development of a Cautionary Tale Feature Film about Gradual Disempowerment
🦄

Caroline Ellison

4 months ago

@Petr_Salaba I'm curious if you have any more updates on this project! Thanks!

Help the Argentinian AI Safety Community Grow
🦄

Caroline Ellison

4 months ago

Approved--it sounds like BAISH has successfully gotten many people working in AI safety!

Launching Humans in Control's Student Program
🦄

Caroline Ellison

4 months ago

Approved! This sounds like a plausible framing for movement-building.

US Policy Communication Playbook for an AI Crisis
🦄

Caroline Ellison

4 months ago

Approved--excited for this project!

Separatrix
🦄

Caroline Ellison

4 months ago

Approved! This sounds super interesting--good luck!

Supplementing Summer Program for Applied Rationality and Cognition 2026 (SPARC)
🦄

Caroline Ellison

4 months ago

Approved! I think SPARC is a good program and influencing the trajectory of talented high schoolers can be valuable.

Token taxes as a mechanism for reducing AI-driven power concentration.
🦄

Caroline Ellison

4 months ago

Approved! This sounds like a plausible policy that's worth understanding better.

What Factors Influence Chain-of-Thought Faithfulness?
🦄

Caroline Ellison

4 months ago

Approved! This sounds like an interesting experiment, and I'm excited to be working with grantmaking.ai.

Postdoc on verification protocols for frontier AI training
🦄

Caroline Ellison

4 months ago

Approved this grant! I'm excited about research into verification tech for if/when it's needed, and excited to be working with grantmaking.ai on this funding round.

Transactions

ForDateTypeAmount
Manifund Bank2 days agodeposit+2000
Manifund Bank2 days agodeposit+2000
Manifund Bank3 days agodeposit+2000
Manifund Bank4 days agodeposit+2000
<729f62bd-a8cd-44fb-97c8-d04b90daa33b>11 days agoprofile donation10
Manifund Bank11 days agodeposit+20
<fd98bb90-533a-45ee-a8ee-abd7366ad001>13 days agotip1
Manifund Bank4 months agodeposit+10