You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
Instead of one ChatGPT what if there were 8 billion AIs.
There are 8 billion humans, which are differentiated by their personalities, experiences, tastes, and expertise. However, even if one cloned ChatGPT, and fine tuned it 8 billion different ways, it wouldn't match the humans. What's the gap?
1. ChatGPT, and similar AIs, rely on the training data as the same mechanism for its other effects.
The training data is more or less what makes ChatGPT, ChatGPT. What makes Claude, Claude. Even if you fined tuned it to be different, it would remember all the little wedges from the base model.
However with humans, we don't have that baggage. For example, one human knows English as a native speaker, and another Spanish. However, both started with no language. What led to them to learn English and Spanish respectively was the brain schema encoded in their DNA.
The humans had the faculty hard wiring to learn language, and updated their brain schema to what they have experienced. This requires very little training data to be proficient in a language. Not proficient with a margin of error, actually proficient.
In biological organisms without language, this brain schema applies. It is a distinct way of forming an AI model then the current models. Why?
a) The training data is not used to form the base model.
The base model in this case is the DNA base for AI. This DNA base has no training data applied to it. For current AI, how it knows what language is, is baked into learning English from training data, not because it has a language schema. This proposed approach means AI would know the faculty of language, where it could learn any language that exists with very little training data in comparison to what is needed for current AI.
b) Experiences replace training data.
How data is structured to the AI is that it learns in a branching environment. Eventually, it learns basic words, to sentences how they are layed out, to then production of words, and sentences, to then initiation of conversation.
Instead of data being structured already for Ph.D. level math texts and pattern recognition of of that. Why?
So the differences in how it learns is tracked, this tracking is what can lead us to compress what it learns to a DNA model of what a language is, such that in its environment, it needs very little info to grasp English, then it can move on to other things.
2. What does this look like?
I will be developing a very basic loop of the following and training and testing upon it to prove the concept can work in reality.
Normally, an AI is distributed as:
- executable
- tensors/weights
The outputs of this would be:
- executable
- DNA blank slate (think of a human just born)
- .person (all the changes to personalize the person, what they have learned, language, environmental, tastes)
(To put this into perspective, the DNA is like the first passes made, and .person is the last passes made to derive a resulted output).
(In humans we start with our blank slate, however much of our brain is changed to the person we become. Still, the approach is to separate the DNA slate and .person, instead of combining them, so evolution of the DNA blank slate can be made. A positive is still found from the brain orchestrating functionalities to certain brain regions and this is similar to the DNA blank slate separation still operating in the final product).
For the first iterations of a skill, the AI does not have a DNA slate. It instead learns similarly to current AI architectures. However, it runs in parallels of what it learns. Over time, the optimal methods to solve a learning method are saved to the DNA slate to first create that slate.
Then the DNA slate is used instead, and what it learns is now fed in a chronological order so it can build upon itself. Where there are limitations is noted as what to train for the first iterations again.
Finally, what is etched from both of these learning methods into the DNA, the person is now able to be developed to the extend defined in the DNA schema and optimization pathways. These files allow evaluations over how individualized the AIs can become, or if an optimization was overfitted, and they all similarly have a same way of speech, in which the DNA needs to be branched from that point in a different direction.
3. Simply:
Traditional AI learning loop will be applied to track how a skill is learned and take those lessons to be optimization paths in a DNA slate.
The DNA slate is fed chronological learning data like a growing human, and what limitations are found are tracked for the traditional AI learning loop to train.
The updated DNA slate starts developing its own personality, tastes, knowledge, and how individiated the persons are from one another (many in parallel are tracked at each stage of this evolutionary process) (the .person file). If optimizations were overfitted to make different AI persons similarly acting, this tracked in the evolutionary cycle to be branched and changed until it doesn't.
4. Eventually:
This project should be able to prove that there is a significance (from a small level), that:
a. The DNA slate can model skills in a way where it needs less training data to produce similar outputs as a traditional AI model
b. The .person file can significantly change the behavior to where two different .person files can not remotely produce similar prose or thought patterns to deliver a result
c. The three different parts of the training can be tracked in the DNA evolution such that other researchers can contribute and update base slate schemas (intermediate description file).
I will spend the entirety of the fund to the training process of the new model architecture.
This will be scaled to the limited scale size to prove #3 and #4 of the previous section to be within budget.
This means the majority will be used on GPU spend compute.
Many of the things I am working on in this project I have done in some way before:
1) Branch and compare: In college I developed a multi-axis method to track similar content on text, like YouTube content ID does on video, however without using AI. So over time, new rules and algorithms are added to the base database on how to look in an improvement loop, which was more accurate than any fuzzy or other base library compare was.
2) Problem solving, open box: I reverse engineered the Duo Security custom two-factor solution so I could restore my accounts in case I didn't have the Duo app (damaged phone device or lost). It wasn't like TOTP where you could export it, it was a custom server and steps where I implemented HOTP and Push Notification authentication to python.
3) Problem solving, black box: I solved a major bug within the Cloudflare domain system which could allow an attacker to have unlimited spend without actually confirming paid transactions.
The major labs are all betting on their being multi classes of AI, that they each can develop. For example, there being a superintelligent Google ASI, then Microsoft ASI, then OpenAI ASI... where each may rely on totally different architectures and trade secrets.
From this belief they release open models which are actually closed, from being just a weights file of the tensors instead of the actual development of the model.
Instead of this model, we (as the AI researching society) need to have AI clearly defined in an evolutionary model.
The reason why is not trying to compile code like primitives to AI (which is the opposite than this schematic brain model), nor for the sole benefit of open source or not, it is because:
1. The labs are running out of training data.
Throwing more training data than a human would ever read or experience in their lifetimes cannot train the model to be more precise, ever diminishing returns. This led labs to develop synthetic training data, which similarly produce those diminishing returns.
Instead of relying on this training data pipeline, using a brain schema like method means the bottleneck on training data is removed, since not that much training data is needed. Researchers instead slowly harden how the AI develops its brain schema, meaning each evolutionary track is more precise, learns quicker, and uses less data.
2. The open source community can now orchestrate their efforts together.
Besides major labs releasing weights, the open source community working on DNA schemas means efforts can be pooled into those DNA branches. Instead of needing much compute over much training data, from scratch to final product, over proprietary training methods, there would be a starting point from the last DNA evolutions, to fine tune them (not AI speak fine tune), try out different ideas and experiments, and publish them into a landscape where someone can pick up right there and contribute.
3. The labs can more clearly audit their models.
Each AI model coming out of the major labs is like a totally different recipe being judged in entirely different cooking competitions. Now, each model and branch and class, and evolution all can be tracked in its evolutionary history and can even be compared across labs.
Why is this the case?
Because of the convergent evolution principle, that even if they have evolved differently, applied to the same task, let's say a wing of a bird vs a wing of an insect, both are wings. In this constraint of DNA, even if different labs have different ways to construct a language brain schema, the fact that it is operating upon language gives a testable environment rather than the end product of an AI output.
I used to think that it is most effective to change major systems, like governments, from the inside rather than the outside. However, I figured out that system relies more on the external environment than the internal environment to change itself.
I used that idea to think about instead of looking at the end thought forms of a system, look to where they originate. Instead of doing evals out the outputs, what is causing the outputs to be structured in this way?
Which led me to the point: all the major labs are betting on training AI from training data from scratch to end product more or less. Instead of looking at the small decisions compounding in a society, or sculpted as a brain, it is kind of like the brute force attempt of internal govt. change.
Instead of needing much training data at all, shifting the approach to brain schema like patterns (as in psychology term) means an evolutionary path for AI can be designed.
A human child does not need that much training data to learn things, especially not as much as AI, and the human can become an expert, instead of having logic errors and not filling the percentage gaps on evals. This is because the human has a schema embedded not just throughout their lifetime, which gets updated to their experience, but also from their DNA as executed by the development of the brain.
Language, for example, is one of the most basic schemas embedded into our DNA, which allows any human to quickly pick up on how to speak, from little training data. What if there was a language schema for AI? What would change is much less training data would be needed and precision would be much more accurate. Why? Because now AI can be tracked as DNA evolutions instead of the brute force attempt from earlier, which restarts every time a new model is created.
For AI alignment this also is a profound application, because instead of AIs all being different from one another, there would be standard DNA tracks to compare and contrast upon, and what an AI can do is bound by that environment. Currently, AI is unbound from it's input to output architecture (as in capability), but in a DNA architecture it would be limited to just the brain's bounds and can easily be diagnosable.
Finding a malicious AI, we can see what thought forms led it to its outputs, by going to those schemas and DNA setup, and what data was fed to it, and how it advanced over time.
$0. I applied for one other Microgrant from this form for another idea. (It may appear shortly after this one in a few hours).