Deeply personal

This is a first for me. I’ve never publicly shared such personal and opinionated thoughts. Hopefully it will rock your brain like it rocked mine, and help you have a new framework to solve harder problems on your daily life. That would be awesome, if not, at the very least it should be an entertaining thought experiment.

I will share the methods and results of an exercise I’ve been doing for a long time: Trying to figure out what’s the most likely thing that AI engineers are missing that keeps them from achieving AGI. Take it for whatever it’s worth. Note that I am not actively developing or cooperating on any push to reach AGI, this is something I like to think about as a creative process.

Let’s get the apocalypse out of the way and move on to more interesting things…

You might argue, or even worry, that sharing my perspective publicly could influence or inspire someone ultra-smart, say a leading engineer at OpenAI or MIT, and it could butterfly effect the world into the super-inteligent AI apocalypse. I think that’s a risk I am willing to take responsibly because a more likely scenario is that this helps some of you readers have an extra tool or perspective that helps you approach hard problems differently. And might help you out in your life in some small way. It’s a fair trade-off in my opinion.

Back story

The year was 2016 (way before the AI boom or LLMs), and I had a brainful of excitement for finding applications for AI, specifically related to vision, language and logic. I was experimenting and co-creating artworks (small example) that use whatever techniques accessible to me and marveling at feats of DeepMind’s AlphaGo.

Feeling in my own skin the gap between the potential and limitations of AI and nature inspired algorithms was making me spend a lot of time wondering about the future of the technology, and asking “why are we not there yet? What are we doing wrong?”… So the semi-unconscious creative process started, and I followed my go-to method when thinking about hard problems. It ended up in a “seemingly brilliant” idea that would haunt me to the present day…

The idea that sparked my search

In simple terms, I would create a text-based chat-bot called “Jess” (yes, how cliché, bare with me 🤦) that would serve as an interface for a logic engine to store knowledge, it would start with 0 knowledge besides simple communication and would learn from every interaction using reinforcement learning and a genetic algorithm. But instead of having a clear success function that would tell the system what is a “good” or a “bad” answer, the AI would chase a set of constantly changing general incentives, affected by feedback/consequences of its outputs in a way that was inspired by the behaviour of neurotransmitters like dopamine, serotonin, cortisol, oxytocin, norepinephrine, and endorphins, in short: it would follow emotions.

It would also run constantly, and when I am not chatting to it, it would get “bored” (values like dopamine and serotonin would tend to lower with inactivity) and it would have the ability to start chatting to itself and/or actively message me.

This would probably make it learn slowly (at about human speed or even slower) because I would have to not only teach it but also convince it to learn and treat it nicely. It would probably develop seemingly unproductive moods, so it wouldn’t learn at a fixed rate. It would get stuck in inner monologues and vicious cycles that I, as its trainer would have to try to de-incentivise.

On the other hand, it could possibly result in a truly different type of AI agent, with real independence and autonomy to not only solve problems but also to choose which problems to solve. For me that would have been closer to an AGI then anything else I had seen. Even if it was really hard (and take a long time) to make it useful for anything other then a little fun.

It would have to be taught/raised like a child (or a pet if that makes it easier on you), but with some added features, it would be clonable and versionable so if you screw up as a trainer and cause trauma or damage the model in some way, you could revert back and try again, and if you reach a point that it has a nice general understanding, you could save it at that point and branch out trying to specialize different version of it for different tasks, knowledge, personalities, or pass it to other trainers. You could also allow them to talk (socialize) with each other or even possibly merge multiple of them into one or breed them with a genetic algorithm.

Brace for anti-climax: The project didn’t go much further then a small, unfinished, proof of concept that is collecting dust in an old drive. Even though I was ultra-excited about it (still am after all these years, let me know if you want to help me take it out of the closet), I was too busy trying to make a living in the media art industry and couldn’t keep up with it. Plus I was reticent to share it with people, because emotions and AI are a sensitive topic to many. Regardless, it built this really strong structure in my mind that held till today, which is impressive so it must be relevant in some way.

The method

I am not saying that “Jess” is the path to AGI, but the method I used to define it likely is a part of it in my humble opinion.

So, what’s this thing I do when my brain gets stuck on really hard problems that many smarter people are trying to solve?

The reasonable side of me says to just forget about it, “who do I think I am to even entertain this? right?”. But that’s no fun! so I need to convince this boring reasonable side of me to play along (cause it’s probably my smarter side and I need it). What I do is try to “cheat”;

Many ultra-smart people with resources have been dedicating their lives to this since the 1970s¹. Trying to solve it the same way they do, while going about my day, is futile. Instead, I am better off focusing on what they are not trying. Rather than thinking about the problem itself, I first consider the people attempting to solve it.

“Problems that are hard are usually hard because of the set of perspectives and tools that have been used to try to solve them. So, you have to always ask: am I bringing something new to this problem?”

Author: Nathan Myhrvold (source)

When faced with these problems, I focus mostly on two super-important factors, biases and incentives. We all have them, and they play a central role on everything we do, whether we like it or not. They are evolutionary traits and they have been critical for our survival for a long time. But they are like optimization processes, they “speed up” our decision making at the expense of blind spots or shallow thinking. This is specially important to keep in mind with hard problems, because what makes them hard is that the solution is likely hiding behind a bunch of human biases.

So, to find blind spots, I ask questions like:

• What are the people working on this excited about?
• What are their moral and cultural values?
• What are the potential use cases that they would love to apply this problem to?
• Why are they paid (or otherwise incentivised) to think of this so much?
• What would I have to say to really make them mad, or cringe?

Then I ask the same questions about myself, and try to imagine things that would go directly against any of these. Things that could still result in solving the problem but would deeply disapoint the ones trying to solve them, or make the whole thing useless for some reason (at least for the expected use-cases). Those components have a good chance of being a part of the solution.

This of course doesn’t guarantee I will find something revolutionary, it just improves my chances of making a meaningful contribution, even if I am completely missing the point. And in this case, I believe it resulted in something that I havent seen openly discussed realistically in modern AI yet.

Biases and incentives that could be blocking the road to AGI

The (emotional) elephant in the room

If you’ve read any book or watched any movie that features AI, you know the absolute number one rule of survival: Whatever you do, do not give it emotions!. This seems to be the most deeply rooted bias in our culture when it comes to AI, it brings machines and our relationships with them to a new weird uncomfortable place, it threatens our very identity, our importance, it raises the hairs on the back of our necks and plagues our beloved tools with ethical issues. But also it breaks what machines are traditionally good at and useful for us: They are machines! No emotions, determinism, rules, they don’t get tired, they don’t get sad, they don’t love or even like, or betray. All of these stand in the way of the machine doing its job. But when we think of emotions in humans, they are what makes us special, that’s why we’re better then any machine will ever be, because we love, because we struggle and have moods and we get inspired and we get creative… You see where I am going with this…

Thinking of this I realized, even though emotions themselves seem to be a construct of our brain, and therefore we could expect them to eventually emerge from a “good enough” artificial neural network (ANN), they are heavily correlated and regulated by specific neurotransmitters that have no analog on ANNs. And clearly these chemical reactions are extremely important for our brains to function, even a small unbalance on one of them can be the difference between a fully functional, productive brain and a severely damaged one, specially when it comes to interactions with others.

Passed the bias, it seems to me completely reasonable to assume that removing a component from such a complex machine would have severe crippling effect on functionality.

So although it feels ethically wrong to try to add/simulate/allow emotions on AI systems, and it will likely come with major downsides in usability, there is a chance that they are a hard requirement for AGI. In that case, the ethical responsibility will probably shift to mitigation strategies to the potential problems of having emotional AI. Before widespread use of the technology.

Training: “Why the rush?”

Although ANNs were created to mimic the human brain structurally and functionally, the way we train them is quite different.

When you are using an AI model like ChatGPT or Midjourney for example, it’s not learning anything, it won’t have any memory, and no matter what you tell it, it will not change its “brain” in any way. What we do to “fake it” with chat-bots specifically is that we send along with each message, the whole conversation or context, including past answers from the model so that it can carry a conversation. Of course, these conversations can be saved so that, at a later stage, they can do a new training session with on the model, that’s where the “training on my data” discussion comes from. But training is one thing, inference is another, they never happen at the same time right now.

As for humans, we are born with some basic genetic pre-training (instincts, basic motor control, and things like that) but the difference is that our brains train on the go, they are constantly changing (called brain plasticity). This gives us our ability to learn, adaptability, creativity, improvisation, and the super-important power of reason to “change our minds”. All the things that we blame AI for not doing well enough.

So until we can maintain an AI that is constantly in a training process, we’re not likely to see AGI. This of course makes training and inference slower, and we would have to deal with really “dumb” AI until it learns enough to be useful and the results would be really unpredictable and hard to control (just like with humans). So it’s not something we are incentivised to consider just yet. Even if we could blend pre-training on large datasets and then continuous learning strategies. But if a model can’t learn something trully new, it won’t be general enough. AGI requires being able to teach a lawyer AI to paint, or even, a lawyer AI that decides to learn how to paint.

The problems of brain plasticity

There have been some small attempts at this that resulted in dramatic failures. We all remember the Microsoft’s Twitter chatbot that had to be shut down less than a day after release because it quickly became a racist, drug abusing, insulting brat. This was totally predictable though, and it makes my point more then it breaks it, it was the worst use-case possible. With constant learning comes great responsibility, that’s why children should not be on social networks. The early learning of a system like this is really fragile, you wouldn’t release a child alone into the wild “twittersphere” to learn and expect it to be a good person.

In humans we have protection mechanisms for this, children are way more capable of learning than adults, there are even skills that you can only learn as a child (like perfect-pitch for example). As we grow, brain plasticity reduces, this makes adults more rigid and less adaptable, but protects the brain from trauma, and from being too influenceable and being taken advantage of (reprogrammed). But although it reduces with time, our brain plasticity is still high even on older people, meaning an “old dog can learn new tricks”, even though it’s harder. This goes against the current AI incentives, no one wants an AI that needs to be trained like a child, protected and cared for a long time and that it can easily be influenced and deceived into becoming “evil”. So research leans more into optimizations of training for specific tasks, narrower AI, which is way more useful in the short term.

Conclusion

Hopefully this didn’t scare or bother you too much, the pursuit of knowledge and the quest to solve complex problems like AGI are intrinsic to human nature. Avoiding certain areas of research out of fear or ethical concerns will not help our cause; it will expose us to more danger. Our responsibility lies on acquiring as much knowledge as we can and managing how we use it, ensuring that the technologies we develop are used ethically and for the betterment of society, or not used at all.

More importantly, behind all the exciting “science fiction” and futurism in this article, lies a way of thinking that is of major importance. Our biases and incentives are being exploited against us daily and in mass. Critical thinking requires us to question everything, including our education, traditions and even our concepts of good and evil. We are biased, that’s part of us and it has its utility, but it’s not enough to just react anymore, we need self-awareness, we need to go deeper, think harder about our important decisions, individually and as a species. It’s very tough but we are leveling up, trying to go backwards into blissful ignorance was never a choice. So let’s try to look at the future with excitement, there’s a lot of great things that can come from us thinking harder.

Thank you so much for going through this with me. I hope it was entertaining. If you want to discuss, just reach out.

References