Raphael Thys
An artificial brain. Credit: Author via Midjourney
One question at the core of AI has remained unanswered for 70 years — without giving any signs it’ll be resolved anytime soon:
“How much should artificial general intelligence (AGI) resemble the human brain?”
We know a lot more about the human brain than we did half a century ago. Yet, its deepest mysteries seem to be as out of reach as they were at the very beginning.
You may think it doesn’t matter if we don’t understand it — after all, we’re living in a golden era of AI research and development. But that’s only partially true.
It matters. Much more than many in the field think.
That’s why a group of high-profile scientists (DL pioneers Yoshua Bengio and Yann LeCun among them) published in October a whitepaper entitled “Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolution.”
They arrived at the conclusion that, if we want to build AGI, “neuroscience progress should continue to drive AI progress:”
The above Tweet drew attention from all corners of AI and Neuro Twitter. Meta’s Yann LeCun, professor Gary Marcus, DeepMind’s David Pfau, and Andrew Lampinen, among others discussed the not-always-friendly relationship between AI and neuroscience.
Two key disagreements motivated the debate: How much inspiration has AI taken from neuro in the past and how much should it take in the future?
I agree with LeCun and Marcus that early AI researchers were interested in the brain (they didn’t call it deep learning and neural networks for nothing), and influential work — that eventually led to the current dominant paradigms in AI — stemmed from neuroscience research.
There’s less consensus on the degree to which those influences have shaped AI.
Did they become part of the field’s backbone or were they just a superficial first push? Are they alive in today’s AI research? Should we increase or decrease AI’s closeness to neuro?
My focus today is to lay out a convincing argument that the present and future relationship between AI and neuro should be much tighter than it has been so far.
I agree with the paper’s thesis: “We propose that to accelerate progress in AI, we must invest in fundamental research in NeuroAI.”
A side note: not all “AI” wants to be AGI
This article is a shift from my usual focus on the usefulness of AI research for commercial and consumer applications.
My arguments here don’t apply to most AI systems. They regard the ultimate goal of AI as a scientific field: building truly intelligent agents made of non-carbon forms.
Most of what we call “AI” today isn’t intended to be a part of AGI. ChatGPT, GPT-3, and Stable Diffusion are perfectly fine intelligence-wise as they’re now. People using DALL-E to draw cute dog pics don’t care about that.
This is the case for virtually all AI systems in existence today (although some may yield insights that will be key to unlocking “next-generation AI”). The majority of AI startups care about usefulness and enhancement. Only OpenAI and DeepMind are explicitly concerned with building AGI.
When we talk about the future of AI as a field — what it needs and what it lacks — this clarification is paramount: not everyone considers AGI their ultimate goal, not everyone thinks it’s necessary, and some even think it’s an undesirable pursuit.
AI should take more inspiration from neuroscience
This article is about how AI should focus its scientific endeavor toward AGI.
But let’s back up a bit first.
Although AI was influenced by neuro early on (e.g. McCulloch and Pitts neuron model, CNNs, etc.), eventually, the divergence became so great that most people working in each field today know very little — if anything — about the other.
One of the most notable historical divergences between both fields is the sharp contrast between neuroscientists’ sophisticated understanding of neurons (in form and function) and the outdated model on which all artificial neural networks are based.
Artificial neurons are extremely simple compared to their biological counterparts because the former are based on an 80-year-old model of the neuron.
As I wrote in a previous article:
“[The McCulloch and Pitts neuron model (MCP)] is, for the most part, the same model that’s taught in every modern introduction course or book on deep learning. It’s not because MCP is an accurate model that needs no refinement, but because deep learning hasn’t changed a bit at this elemental level since 1943.”
If we polled AI people and asked if they’re aware of this it’d yield equally surprising and depressing results: Most either don’t know or don’t care.
During the 40s and 50s, AI scientists found a way to make artificial neurons learn (they refined the algorithms later — backpropagation). Why would they bother with sophistication when changes could risk the learning ability of the networks?
It was better the devil they knew than the devil they didn’t: pragmatism > realism.
Recent work published in Science (2020) and Neuron (2021) reveals the degree to which this was a mistake. What seemed a small discrepancy back then is now a vast chasm unlikely to be closed anytime soon.
We now know the difference in complexity is such that a biological neuron may be around two orders of magnitude more complex and capable of computation than an artificial one.
How can we advance AI toward intelligence if everything is built on unrealistic foundations? Gary Marcus highlighted on Twitter this and other key ideas from neuro that AI/ML has never embraced:
Engineers don’t study birds to build better planes
That’s the typical — and effective — counterpoint to all of the above.
Why should technology follow biology’s or evolution’s steps? Why does it matter if AI and neuro aren’t growing in parallel?
We can always find a new solution in the space of physically-possible options — even if it is nothing like nature’s.
That’s what DeepMind’s Andrew Lampinen defends:
Lampinen is at least partly right. It’s been empirically demonstrated many times (planes vs birds is just the most popular example).
Theoretically, it makes sense, too: evolution is a game of trade-offs — our brain is big enough for us to be smart but small enough so that it doesn’t weigh too much as to risk our survival.
Technology isn’t bounded by evolutionary limitations. AI systems, although dumb, surpass us in many areas already: computation speed, memory capacity, accuracy… key features of intelligence that we couldn’t maximize because evolution’s goal wasn’t to make us intelligent, but “survivably” intelligent.
Lampinen’s argument is strong where everyone agrees: no serious AI scientist would argue that AGI should be a literal silicon copy of the human brain. But neither would anyone claim we should ignore biology completely.
Even factoring in evolution’s misaligned goals, it’s reasonable to follow its steps to some degree. The possibility to find non-biological solutions shouldn’t be a synonym for dismissing biology’s wisdom.
If we extend the plane-birds analogy AI-neuro it breaks down at several points.
First, the Wright Brothers, pioneers in the field of aeronautics, did study bird flight. As David Pfau puts it, “[they] were obsessed with understanding how birds flew.”
They also “studied ornithology texts.”
Second, there’s a “more fundamental” breach in the analogy. From the paper:
“The goal of modern aeronautical engineering is not to achieve “bird-level” flight, whereas a major goal of AI is indeed to achieve (or exceed) “human-level” intelligence.”
Not only did aeronautics pioneers study biology’s steps, their goals weren’t even as close to those of ornithologists as they are between AI and neuro.
Both arguments strongly defend the reasonable approach to start by studying biology if your final goal is to reproduce something biology has already created.
And yet, despite these arguments, I don’t think most people will be convinced that AI should go back to the old ways — or better, to the original intentions — and AI researchers should be much more knowledgeable of neuroscience’s discoveries.
Why? Because none of these arguments make it a necessary condition that AI should keep up with neuroscience further. AI pioneers did follow evolution’s path at the beginning. And even if the goals of AI and neuro are closer to each other than those of aeronautics and ornithology, they’re still not the same.
“Haven’t we followed biology enough?” is a valid question any AI researcher could make given the above arguments.
AI scientists could find a close-enough and good-enough solution for intelligence without taking further inspiration from neuro.
Although I don’t find it plausible, people can reasonably believe this.
The final argument: the need for shared scientific ground
There’s a stronger argument (in my opinion) the paper missed.
An argument that, instead of looking at the origins or at the goals of AI, looks at the point at which the field would be ready to diverge from neuroscience, given the historical precedents (e.g. aeronautics and ornithology).
At this point, a field has drawn the optimal amount of influence from another, and, since that moment, everything else it takes can even be counterproductive.
Of course, in practice, there’s no such point. The relationship between sibling scientific fields is much more complex — AI can take here and there from neuro, now or in the future, and make good use of those discoveries.
But bear with me. The metaphor is useful because it makes it easy to see that AI-neuro’s divergence point is far in the future.
Aeronautical engineers didn’t build planes different than birds just because they wanted (which is what the paper’s stronger argument defends), but because they could.
They studied birds’ flight and, together with insights from other disciplines like Newtonian physics or fluid dynamics, they eventually laid the scientific groundwork — in the form of aerodynamics, flight dynamics, vibrations, etc. — from which a new, different solution could emerge.
Aerodynamics governs the forces that affect all kinds of flying objects — planes and birds alike. Biology, like technology, has to obey the laws of physics. Planes could differ from birds in some senses, but in others, they had to be similar.
Only then, once the foundations were set, engineers could decide that engines were a better solution than flapping wings. Biology’s insights helped them build the underlying scientific theories from which to diverge and find more optimal solutions.
Could they have found a solution without those robust foundations? It’s theoretically possible, yes. Would it have made any sense to try? Not at all.
How does this extrapolate to AI and neuroscience?
AI scientists may not want to diverge from neuro as much as aeronautical engineers wanted to diverge from birds (as the paper explains, AI’s goal is “human-level” intelligence), but even if they wanted, they couldn’t.
Why? Because an analogous shared scientific groundwork is non-existent for AI-neuro — we don’t have our “neurodynamics.” Right now, neuroscience (and AI) is at a similar stage as physics was pre-Galileo and pre-Newton.
We lack well-established explanatory theories of intelligence, brain function/structure, and how the former emerges from the latter. AI should evolve in parallel with neuro until we develop those.
Would anyone have argued back in the early 20th century in favor of static wings because “why should we follow biology?” It would’ve been nonsense.
That’s how we should frame today’s conversation around AI and people’s reticence to get back to neuro. How can the AI community build intelligent machines without a better understanding of the brain or intelligence? Using benchmarks? Trial and error? Scaling data and computing power to the end of the world?
With this argument in hand, we know that’s a fool’s errand.
Do they really think they can build human-level intelligent machines without taking inspiration from the only instance of intelligence while at the same time dismissing the necessity for robust scientific theories?
It seems clear now that the AI-neuro pair isn’t ready for divergence. It isn’t yet reasonable to let AI wander and explore on its own.
The unavoidable conclusion that stems from this — if we accept that modern AI has nothing to do with neuroscience (I believe so) — is that AI diverged too much, too early, from neuroscience.
For those who are trying to build AGI, this was a terrible mistake (probably motivated by the pragmatic view of “let’s do what works now and we’ll figure out the rest later”).
Future AI scientists may look back at this era like today’s chemists look back at alchemy.
AI needn’t follow neuro’s steps all the way but should have continued to take inspiration — much more than it takes now.
As I always like to do, I’ll end on a high note:
“The emerging field of NeuroAI, at the intersection of neuroscience and AI, is based on the premise that a better understanding of neural computation will reveal basic ingredients of intelligence and catalyze the next revolution in AI, eventually leading to artificial agents with capabilities that match and perhaps even surpass those of humans.”
I don’t think it’s too late.
Subscribe to The Algorithmic Bridge. Bridging the gap between algorithms and people. A newsletter about the AI that matters to your life.
You can also support my work on Medium directly and get unlimited access by becoming a member using my referral link here! :)