In 1976, at a time when computers were only just beginning to appear outside research laboratories and when the internet existed merely as the military ARPANET project, Joseph Weizenbaum published a book that dared to ask questions most of his colleagues considered irrelevant, or even dangerous. Weizenbaum was no technological Luddite, nor a nostalgic critic of progress. He was a respected professor of computer science at the Massachusetts Institute of Technology, an institution that prided itself on standing at the forefront of the technological revolution. And it was precisely this position inside the system that gave his criticism such extraordinary weight and urgency.
The book “Computer Power and Human Reason” grew out of a personal crisis of its author, who became an unexpected witness to how even a simple technology can be misused when society fails to understand it, or ascribes to it qualities it does not possess. Weizenbaum wrote the ELIZA program as a demonstration of the principles of natural language processing, an academic illustration of how a computer can analyse human sentences and respond to them according to predefined patterns. The program was designed to imitate a Rogerian psychotherapist, which was an ironic choice, because Weizenbaum wanted to show how mechanically some conversational techniques can be reproduced. What followed, however, shocked the author so deeply that it changed the direction of both his thinking and his career.
ELIZA became a phenomenon, but for reasons Weizenbaum had never intended. People became emotionally attached to the program, confided their intimate problems to it and even demanded privacy while conversing with it. Psychiatrists began to consider seriously deploying such systems in clinical practice. The media celebrated ELIZA as a breakthrough in artificial intelligence and in the understanding of natural language. The author watched with growing dismay as his simple experiment in computational linguistics grew into something he regarded as a dangerous illusion about the capabilities of technology, and an even more dangerous self-image of the human being as a mere machine.
The Czech version of this article continues with a Czech translation of the introduction to this remarkable book. It contains not only the story of how ELIZA came about and how people reacted to it, but above all a deeper reflection on the changing relationship between people and technology, between science and wisdom, between reason and mere logic. Weizenbaum shows how modern science, originally a liberating force, began to act as a slow-acting poison that undermines trust in other ways of knowing the world and reduces human dilemmas to technical problems solvable by calculation.
The text is remarkable above all for its authenticity and its courage. Weizenbaum writes as a man coming to terms with the moral consequences of his own work, a man who asks about the responsibility of the scientist towards society and who is not willing to accept an easy answer. His criticism is not aimed at technology as such, but at the imperialism of instrumental reason, at the tendency to treat as legitimate only those questions that can be formalised and turned into algorithms, only those problems that can be solved by computation.
Today, almost half a century after those words were written, they sound prophetic. We live in a time when artificial intelligence really can hold a conversation in a way the original ELIZA could not even remotely imitate, when automation has spread into areas Weizenbaum could hardly have foreseen, and when the debate about the boundary between human and machine thinking has become far more urgent. Yet the fundamental question remains the same: are there aspects of human life, of human thought and of the human relationship to the world that should remain the preserve of people, even if technology were able to imitate them?
Weizenbaum invites us to think about that question in a way that is at once personal, philosophical and urgently practical. He urges us to realise that decisions about how we use technology are not merely technical ones, but above all moral decisions about what kind of people we want to be and what kind of society we want to live in.
A note on the original text
The introduction that Weizenbaum wrote for the 1976 edition is still protected by copyright, so it is not reproduced here in English. In it he recalls the reception of his DOCTOR script and describes how quickly and how deeply the people who conversed with it “became emotionally involved with the computer” and anthropomorphised it. His own secretary, who had watched him build the program for months and therefore knew perfectly well that it was only a piece of software, asked him to leave the room so that she could talk to it in private.
Source: Joseph Weizenbaum, Computer Power and Human Reason: From Judgment to Calculation, W. H. Freeman and Company, San Francisco 1976, Introduction. The original book can be borrowed from the Internet Archive; the Czech translation of its introduction is published in the Czech version of this article.
