Zero-shot learning denotes a model’s ability to solve a task for which it was given not a single example – only a description of it in the prompt. In large language models this ability emerges as a consequence of pre-training on gigantic corpora, which themselves contain countless implicit demonstrations of all sorts of tasks, and further thanks to instruction tuning, which teaches the model to understand an instruction as an instruction at all. Formally it is the shift from the “one model, one task” paradigm to a general solver directed by natural language: previously, classifying sentiment required collecting thousands of annotated reviews and training a classifier, whereas today it is enough to write an instruction. In the classical computer-vision sense the term was used more narrowly – a model recognised a never-seen class on the basis of its verbal description and a shared embedding space for text and images, as CLIP made possible. The limitations of the zero-shot approach are lower reliability on tasks with a non-trivial output format, with specific corporate terminology, and with edge-case decision rules; there, a few examples usually help.
Zero-shot is the situation where you tell somebody “rearrange those books by height” and they do it without you ever having demonstrated it. They need no training, because they understand English and already know books and height. Earlier programs would stop dead at such a moment and wait for a thousand demonstrations of a correctly rearranged shelf. You spot the boundary of this ability as soon as the instruction stops being ordinary: “flag the invoices that fall under our internal directive 4B” is beyond anyone without an explanation – human or model.