The Deep Learning Rut
Beware the trap of well-worn tracks when a new answer emerges.
I. Setting the Stage
As we discuss our technology of Biologic Intelligence, an artificial general intelligence paradigm, we find over and over again there are two worlds that dominate the AI landscape:
- Deep Learning
- Machine Learning
Biologic Intelligence is the natural, next generation of AI and is still being defined, but like with all new concepts, some people get it immediately, some people do not get it at all, and others get it somewhat but rely on their experiences with other concepts to try to make sense of what we are talking about.
It’s this latter group that we find is disheartening. We call it the Deep Learning Rut.
Editor’s Note: many people believe we aren’t fans of, nor appreciate the value of, Deep Learning. It couldn’t be further from the truth. And the truth is that we have to compare and contrast approaches. We have to shine a light on the limitations of a mathematic equation because most of the media reports on it like it’s magic. We are after one simple thing: the truth.
II. Defining the Rut
Machine Learning is in a class entirely different than Deep Learning and in my mind, has a greater value for what it does. This article isn’t about Machine Learning nor Data Science.
What is The Rut?
The Deep Learning Rut is people that understand (or think they understand) what Deep Learning is and try to view all AI through the Deep Learning lens, and cannot get beyond Deep Learning understanding. Deep Learning is a mathematical paradigm, not a biological paradigm. John Launchbury, the Director of DARPA’s Information Innovation Office, published in a recent video, Deep Learning is spreadsheets on steroids and I couldn’t say it any better.
We can understand some of the confusion in that a generation of computer scientists have been told that Deep Learning, or Neural Networks, is based on how the brain works. We have some very smart people stating this all the time and the truth is that Deep Learning and Neural Networks are nothing like how biology works. This is pure hype and ego talking, whereas the reality is much different.
I cringe at the term “Neural Network” because it is not based on Neuroscience. People, and especially computer scientists, have drank the Kool-Aid and think what they are doing is similar to how animals think.
III. What About Benchmarks?
A common question we often get is, “What benchmarks have you tested your software on? What about ImageNet??
The problem with Deep Learning benchmarks is that they are created to test Deep Learning. These benchmarks are as narrow as is the technology. ImageNet for example is based on Convolutional Neural Nets (cringe!) and essentially what CNNs do is to break an image into tiny images and create functional values for these tiny images.
When we present 100 images of cats, the system can develop the same functional value for a portion of this image and each of these images is annotated as “Cat”; therefore, each time the Deep Learning system finds this functional value, it will identify the image as “Cat”, no matter what else is in the image and as long as it identifies the image as “Cat”, it gets a 100% correct.
When you, a human, views the “Cat” images in ImageNet, you see a cat but maybe also a flower, grass, a ball, a table, a couch, etc. As a human, you might look at a “Cat” annotated image and say that’s an image of a Ball, or better you might say “A Ball, a Cat and a Couch”. You wouldn’t just say it’s a picture of a “Cat”. This is a huge difference in how we, biological creatures, perceive the world, and how a Deep Learning is trained to see the world. This is also why Deep Learning systems can easily be fooled where a human would not.
IV. Conversations With Experts
I was in a meeting a couple of years ago with a high ranking Director at Google who talked about the Deep Learning cat training with all types of cats: house cats, lions, tigers, leopards, etc. Once the system was trained and was highly accurate, the Deep Learning system was shown a leopard skinned couch and of course the system stated that the couch was a leopard.
This is not to say Deep Learning systems are bad or useless. We know this is not true but we also know that Deep Learning systems are narrow and are specific to the tasks they are trained on and when given something that is similar but the system has not been trained for, the system will generate the wrong or no answer.
Interestingly, I was in a meeting a couple of weeks ago and a very smart gentlemen working for a large, very successful company, stated “if someone tells you they know how Deep Learning works, run away from that person as fast as possible”. My answer to that is if a computer scientist doesn’t know how Deep Learning works, why would you hire that person for your AI initiatives?
Maybe it is because I am a Computer Scientist and a Neuroscientist, and have studied extensively the organization and principles of how brains work that make it very easy for me to understand an inferior paradigm like Deep Learning and allows me to say it is easy to understand. Biology is more physics than math. Math maybe the language of Physics but it is not the process that governs how it operates. Biology cannot be made into a mathematical formula and this is where the rut begins.
As a neuroscientist, I am infuriated that the massive number of computer people that run around saying their concepts mimic biology. This is ignorance at its greatest level and this is the very thing that is keeping computers from becoming true, thinking machines. And a tremendous shame on top university professors in computer science hired by companies like Google, Facebook, Baidu, etc that perpetuate this myth! My only conclusion is you do it to line your pockets and certainly not to promote the science. I wish more prominent neuroscientist would tell them to stop!
Part of the problem is the media but the media is only echoing what they are being fed. Venture Capitalist are buying the same goods and throwing enormous amounts of money at Deep Learning ventures. This all perpetuates the ignorance and if I were a young computer scientist and knew I could make a very good salary working with Deep Learning, I would buy into this illusion as well. In a few years, these developers will be the IBM COBOL programmers of today.
V. Self-Driving & The 3rd Wave of AI
Self-Driving Cars are a perfect example of how the rut has perpetuated the beating of a dead horse. Funny how everyone says that the Level 5, fully autonomous car is, more or less, 5 years away. Each year it is the same story.
How long has Google been working on this concept? Fortunately, car companies are starting to realize the Deep Learning limitations and no matter how many resources they apply to Deep Learning, they might get a slight improvement but they can plainly see that Deep Learning will never get them to Level 4 or 5.
It appears that everyone in the Self-Driving Car space is trying to carve out a place so when the Deep Learning miracle happens, they will be there to capitalize on it. And even more fortunately, the players are starting to reach out for other, better ideas because down deep, they know that Deep Learning will never get them to where they want to be.
Deep Learning should be defined as “down Deep, we know it’s not the answer to thinking machines”.
Luckily there are a few people and companies trying to create artificial general intelligence based on biology and all that we ask is that you view what we are doing with open and clear minds, get out of your Deep Learning Rut and begin to explore the future with us.
Let’s review the chart put together by Nick Bostrom of Superintelligence book fame:
Biologic Intelligence has successfully emulated #2, #3, #4, #5 (some), and #9. It’s the third wave of AI and offers a chance for our species to escape The Rut. Even DARPA believes in the Biologic approach, essentially taking word-for-word everything we’ve been sharing publicly for the last year and a half.
It’s time to Think Different tech industry.
