Word2Vec is a computationally efficient predictive model for learning semantic vector representations of words, known as embeddings, from large text corpora. The technique, published in 2013 by the Czech scientist Tomáš Mikolov, rests on the principle that a word’s meaning can be inferred from the context in which it appears. Word2Vec uses a two-layer neural network and implements two main architectures: Continuous Bag-of-Words (CBOW), which predicts a target word from the surrounding context words, and Skip-gram, which conversely predicts the surrounding context words from a single target word. The resulting vectors capture semantic relationships, which makes vector arithmetic such as “king” – “man” + “woman” ≈ “queen” possible.
Imagine wanting to grasp the meaning of a word without reading its dictionary definition. Instead you look at dozens of sentences in which the word occurs. You notice that the word “bike” often turns up next to words like “ride”, “pedal”, “road” or “helmet”. From this company of words you form an idea of its meaning. Word2Vec does exactly the same for computers. It analyses vast amounts of text and assigns every word a set of numbers (a vector) according to which other words it keeps company with. That is how a computer can tell that “bike” and “bicycle” are close in meaning, because they turn up in similar sentences.