You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$20,000 left to reach the funding goal.
Applicant: Hanlin Zhang, CS PhD candidate at Harvard, advised by Sham Kakade.
Research statement, from my homepage: "I am interested in the scientific foundations and social implications of AI. My research studies the principles of scaling with respect to generalization, optimization, and reasoning in large AI systems."
This pilot extends three published lines of work:
Watermarking. Watermarks in the Sand: Impossibility of Strong Watermarking for Generative Models (ICML 2024; arXiv:2311.04378). From the abstract: "Watermarking generative models consists of planting a statistical signal (watermark) in a model's output so that it can be later verified that output was generated by the given model." And: "We prove that, under well-specified and natural assumptions, strong watermarking is impossible to achieve." Pilot task: watermark detection under test-time-compute attacks.
RAG privacy. Follow My Instruction and Spill the Beans: Scalable Data Extraction from Retrieval-Augmented Generation Systems (ICLR 2025; arXiv:2402.17840). From the abstract: "We study the risk of datastore leakage in Retrieval-In-Context RAG Language Models (LMs)." And: an adversary "can exploit LMs' instruction-following capabilities to easily extract text data verbatim from the datastore." Pilot task: membership-inference and extraction attacks measured against shipped safeguards.
Reward modeling. Next-token Preference: Scaling Unsupervised Reward Modeling (arXiv:2603.02225). From the abstract: "Learning from feedback advances the capabilities and safety of frontier models, but preference supervision remains costly and difficult to scale." Pilot task: reward-model scores under optimization pressure.
Pilot structure (facts):
One shared, pre-registered stress-test suite across the three signals above.
Predictions frozen and published before the main runs; negative results reported explicitly.
Deliverables: the open suite and code, plus a public report.
Budget (USD 20,000, expense-only; no salary or stipend to the applicant):
Compute / API: 12,000
Data / annotation: 4,000
Travel / dissemination: 2,500
Infrastructure / tools: 1,500
Timeline: December 2026 – February 2027 (12 weeks).
Funding disclosure (facts):
A parallel application for this same pilot was submitted to BlueDot Impact Rapid Grants on Oct 24, 2026.
The applicant will not accept funding from two sources for the same costs.
The applicant is an F-1 PhD student in the US; if a funder requires it, funding may need to be received through a US 501(c)(3) institution with the applicant as project lead.
Contact: hanlinzhang@g.harvard.edu