How an AI Learns Language from Scratch

Dyllan Furness

Dyllan explores technology and the human condition for Tech Emergence. His interests include but are not limited to whiskey, kimchi, and Catahoulas.

How an AI Learns Language from Scratch

We all know what robots sound like. The awkward, choppy, nasally voice is engrained into popular culture. And even though contemporary robots have significantly advanced vocal capabilities, countries like Yamaha continue to represent the AI voice as mechanical and constrained.

But the real challenge for communicative AI is not there voice – it’s teaching them the language. Systems can be programmed to communicate through pre-determined questions and answers, but as the Turing test suggests, this programming makes it relatively easy to discern an AI from its human counterpart.

In order to overcome this challenge, researchers from the University of Sassari in Italy and the University of Plymouth in the UK taught the network of artificial neurons to communicate with a human interlocutor through dialogue. What made this lesson so astounding was that the cognitive simulation began from a blank slate or “tabula rasa”. Sure it had some two million interconnected artificial neurons to help it recognize, store, and remember language patterns but ANNABELL (Artificial Neural Network with Adaptive Behavior Exploited for Language Learning) had no basis from which to build language.

Noam Chomsky famously asserted that human beings have an inborn intuition that helps us learn language. As such, many cognitive models are developed with a latent structure or language rules that enables them to “learn” language.

ANNABELL was developed without these rules and required to learn language simply through trial and error, by communicating with a human interlocutor. Many of today’s linguists and neuroscientists believe that our quick development of language skills derives from our brain’s ability to interact with and learn from our environment – so ANNABELL was created to test this theory.

In order to assist in the learning process, ANNABELL’s creators did program in two mechanisms found in human brains. Synaptic plasticity allows two neurons to increase their communicative efficiency. The old adage “practice makes perfect” illustrates this idea well. The more you engage the connection between two neurons, the faster and more reliable that connection becomes. Synaptic plasticity facilitates cognitive processes like learning and long-term memory, as studying and exercise reinforces those connections.

The other mechanism coded into ANNABELL is called neural gating, which enables some neurons to control the flow of information throughout the brain. Where synaptic plasticity reinforces the connection between neurons, neural gating helps prioritize certain connections, silence some, and reroute others to help the brain function more efficiently. 

Combining the mechanisms of synaptic plasticity and neural gating, ANNABELL can learn to control the paths of information throughout it’s artificial cognitive model.

ANNABEL isn’t about to earn the Nobel Prize in literature – or even match the language skills of a tween. But it has learned to respond to some 1500 input sentences with a total of about 500 unique output sentences with all the grammatical parts necessary for language. 

Credit: BBC, Creative Commons

Stay Ahead of the AI Curve

Discover the critical AI trends and applications that separate winners from losers in the future of business.

Sign up for the 'AI Advantage' newsletter: