@NeelNanda i'm doing this by training against a monitor (so negatively rewarding when the monitor fires). the reasoning traces definitely have higher reward (fires the monitor less) and look by eye to be more obfuscated, but i think this "look by eye" is likely confirmation bias. i think continuing training from these checkpoints would work!
Sudarsh Kunnavakkam
@skunnavakkam
$2,000total balance
$0charity balance
$2,000cash balance
$0 in pending offers
About Me
I'm Sudarsh. I'm a first year at Caltech studying Physics. I also do research as a contractor at METR, work in the Shapiro Lab at Caltech, and other things. Please reach out if you'd like, or check out my website at https://sudarsh.com :D
Comments
Transactions
| For | Date | Type | Amount |
|---|---|---|---|
| Manifund Bank | about 18 hours ago | withdraw | 1000 |
| Tooling + Model Orgs for CoT Faithfulness Research | 3 days ago | project donation | +3000 |