what do you want to do?

The far goal would be to automatically decompile arbitrary transformers to make them
interpretable. One way of approaching this would be to create a language with an
attached compiler to more easily create hardcoded, interpretable transformers, analogous
to RASP, but more general. RASP/tracr and its descendents can't handle complex input
embeddings, which makes writing a RASP transformer that internally mimics some other
transformer I want to analyse impossible. I've had success with manually setting weights
to produce the desired behaviour, and I'd like to see to what scale of model I can extend
this and whether I can formalise it into a compilable language.
Specific Steps:

- take a model and analyse it to understand (some part of) its internal behaviour

- manually set the weights of a transformer that uses this behaviour as part of its
algorithm and has low loss on the original model's task

- repeat previous steps as necessary, then generalise from experience to find a better
way of compiling code to transformer architecture

how are you planning to spend the money?

1,000$ - token credits/compute (4 months)

2,000$ - food (4 months)

3,000$ - rent (4 months)

4,000$ - any other expenses/taxes (4 months)

what have you done in the past that proves you will be good at doing this? focus on substance, not credentials

- take a model and analyse it to understand (some part of) its internal behaviour

In my last major project, I analysed gpt2-small's layer 0 to understand what it's doing:
https://www.lesswrong.com/posts/dcvrja6kyshqWX4zZ
I don't claim to fully understand GPT2's embedding, but I've spent a lot of time thinking
about transformer internals and have been successful with the toy models I've looked at
since.

- manually set the weights of a transformer that uses this behaviour as part of its
algorithm and has low loss on the original model's task

I've succeeded with every small-ish model I've tried this for, without using llms or any
kind of gradient descent. Here are 2 Examples/Challenges I've shared with others:
https://colab.research.google.com/drive/1-P2YnvGy7qPO1ugH3Ha7QhOiIwBKAUJx?usp=sharing

https://colab.research.google.com/drive/1j3xCYqam8H3F0R1aecuW53jmidO_wlxG?usp=sharing

- repeat previous steps as necessary, then generalise from experience to find a better
way of compiling code to transformer architecture

No previous work in this direction, not sure if I'll be good at this