What progress have you made since your last update?

We have some initial results that we have detailed here:

https://www.overleaf.com/read/fnwffjycpdzg#6bd6bc

Initial results (on only three model organisms) suggest that activation oracles are more robust to adversarial red-teaming. However, we discovered a bug after running the experiments in which the red-team had access to CLAUDE.md which contained information on past red-team strategies (more details in the overleaf doc).

What are your next steps?

We are requesting additional funding to:

• Rerun experiments without this bug

• Run on updated model organisms from AuditBench (https://x.com/elkmf/status/2106012152972927011?s=20)

• Run on more (possibly all) of the AuditBench model organisms using the single-turn experiments in which the auditor is only allowed to send one prompt. This allows us to run more experiments more cheaply.

• Run experiments where the red-team has information about the particular auditing method it is going against

Is there anything others could help you with?

We are seeking $5000 in additional funding to complete the steps above (smaller amount i.e. $2500 would be sufficient to rerun our experiments without the bug)