This new app uses neural networks to choose the perfect emoji (Video)

A new app from Toronto-based Whirlscape taps into neural networks to choose the best emoji while you’re typing. That may not sound like an impressive feat, but a long more goes into predicting which emoji you’ll use than you think.

What is Dango?

Dango is a floating assistant that runs on your phone and predicts emoji, stickers and GIFs based on what you and your friends are writing in any app. This lets you have the same rich conversations everywhere: Messenger, Kik, Whatsapp, Snapchat, whatever. (just making this possible in every app is an engineering challenge of its own, but that’s another story).

To handle these cases, Dango uses a recurrent neural network (RNN). An RNN is a particular neural network architecture that is well suited to sequential input, and is therefore used in areas as diverse as natural language processing, speech processing, and financial time-series analysis.

RNNs handle sequential input by maintaining an internal state, a memory which lets them keep track of what they saw earlier. This is important to be able to tell the difference between I’m very happy 😊 😄 😃 and I’m not very happy 😔 😞 😒.

Multiple RNNs can also be stacked on top of each other: each RNN layer takes its input sequence and transforms it into a new, more abstracted representation that is then fed into the next layer, and so on. The deeper you stack these networks, the more complex the sorts of functions they can represent. Incidentally, this is where the now popular term “deep learning” comes from. Major breakthroughs on hard problems like computer vision have come partly from simply using deeper and deeper stacks of network layers.

Dango’s neural network ultimately spits out a list of hundreds of numbers. The list can be interpreted as a point in a higher-dimensional space, just as a list of three numbers can be interpreted as the x-, y-, and z-coordinates of a point in three-dimensional space.

We call the high-dimensional space semantic space, think of it as a multi-dimensional grid where various ideas exist at various points. In this space, similar ideas are close together. Deep learning pioneer Geoff Hinton evocatively refers to points in this space as “thought vectors”. What Dango learned during the training process was how to convert both natural language sentences and emoji into individual vectors in this semantic space.

So when Dango receives some text, it maps it into this semantic space. To decide which emojis to suggest, it then projects each emoji’s vector onto this sentence vector. Projection is a simple operation which gives a measure of similarity between two vectors. Dango then suggests the emoji with the longest projection — these are the ones closest in meaning to the input text.

Agencies/Canadajournal