You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$23,000 left to reach the funding goal.
PROBE develops a local pipeline of AI evaluators who can design, run, and critique safety-relevant evaluations of frontier and open-weight AI systems. We plan to start in Asia/greater China region, where capable open-weight models ship fastest and there is a strong need for evaluators.
The PROBE workshop pairs hands-on evaluation design talk with an interactive audit game, which feeds an open-source practitioner handbook. We are requesting $23,000 for a pilot of two workshops in Asia (December 2026 – Q2 2027), supporting the learning of 40–70 participants looking to upskill evaluation. We created PROBE for the following reason:
Evals are how labs, auditors and governments see dangerous behavior before deployment, and evals are getting harder to build well.
China is a major producer of open-weight models with a stated government mandate for AI safety, but the AI safety ecosystem is relatively small. This holds true across greater Asia, where the AI safety hubs are still in development stage.
There is a lack of support for independent, safety-focused evaluator commons. That is the gap PROBE fills.
Safe deployment of advanced AI is bottlenecked by lack of independent evaluation talent who can turn standards into rigorous, safety-aligned evaluations. Best practices already exist in the US, UK/EU, and China. However, adoption remains limited because implementation is nuanced and the talent is scarce. As capabilities grow, especially in agentic systems, measuring risk needs more care in design and oversight. Frontier evaluators such as METR and Apollo Research increasingly point to the limits of current measurement. There are relatively fewer AI safety aligned hubs in Asia that can support evaluation activities and information exchange. China is shipping increasingly capable open weight models that need corresponding development of “human in the loop” talents for evaluation. Delegating evaluation and oversight to LLM-based monitors and classifiers needed a stronger human in the loop approach since LLM based judges have known failure modes. Human judgement and evaluators remains essential at high-stakes checkpoints, which requires skilled and trained evaluators.
Each PROBE session takes 20–40 ML engineers, researchers and risk professionals from reading about evals to designing and critiquing one. The format was piloted at PROBE Berlin in October 2026.
The Eval jam in Shenzhen aim to create the same workshop environment in a conference setting in Mushenzhen's Maker City (https://mushenzhen.xyz/), which convenes hundreds of global developers, researchers, and builders. We also plan for another iteration in 2027 pending funding. In the design jam, small teams learn about the latest eval tools and research. Participants have the option of scoping a safety-relevant eval from a real case, covering threat model designs and human-in-the-loop checkpoints. We will also have game instructions and a facilitation guide so future partners can run similar PROBE independently.
Workshops delivered in Asia
2 (minimum)
Participants reached
40–70
Open source Resources
1 audit game kit and handbook from evaluation materials reviewed
Outcome
Gain in participant eval design skill and awareness of tools and techniques
Participants applying eval design or safeguards at work (6-month follow-up): over 75%
PROBE Shenzhen (Dec 3, 2026): $7,500
Venue support, speaker and organizer travel & honoraria compensation, game printing, and documentation
PROBE Iteration 2 (2027) $6,500
Second city deployment (venue, speaker travel, compute, local logistics)
Series Infrastructure $3,500
Support for community contributors for open Facilitation Playbook, interactive game kits, evaluator network support
Working-Group Seed & Content Curation Fund $5,500
Support for community contributors for capturing, structuring, and curating expert workshop inputs for workshop and handbook integration
Total Funding Request. $23,000
Min-Si Wang (Program & Coordination): project strategy, overall event execution, and workshop curriculum development. Background: Product and research lead in fintech and risk analytics. Main curator of www.aisafetynode.com for cross border AI safety development and field building. https://www.linkedin.com/in/min-si-wang-867395a/
Jaspreet Singh (Program Design): Role: Design workshop format, evaluation methodology, workshop activity structure, and participant experience. Background: experienced product and design lead experience. https://www.jaspreetsingh.net/#projects
Bryan Chan (Developer & AI Safety Specialist): Role: Design and development of the interactive audit game kit, evaluation design jam exercises, and theory of change alignment. Background: AI Safety researcher with background in evaluation(https://www.linkedin.com/in/cnm13ryan/)
Downscaled reach for Evaluation Training: We will reach less participants due to curation of fewer workshops. The breadth and quality of the workshop will also be less comprehensive in scope in terms of the interactive activities and followup resources.
We have not raised funding so far.
There are no bids on this project.