We increasingly rely on machine learning mechanisms and artificial intelligence (AI) to make communication, problem solving and information gathering easier. Chatbots such as ChatGPT, built on the architecture of OpenAI, are proving to be useful tools in many fields. However, people often notice varying levels of accuracy in the answers they get back. This article focuses on how the way we write prompts and queries affects the accuracy of these language models’ answers, and why some users receive incorrect answers while others get more accurate ones.
How neural networks and ML models work in ChatGPT
An artificial neural network (ANN) is to a certain extent inspired by the structure and functions of biological neural networks. An ANN consists of interconnected units called neurons, which are organised into layers. These layers can be input, hidden or output layers, and each unit is connected to other units in the neighbouring layers. These layers work together to process prompts and queries and create answers based on the information obtained from the training data. The more training data the model analyses, the better it adapts to different language constructions and the more accurate the answers it gives.
ChatGPT is based on the transformer architecture, which is particularly suitable for processing sequences of data such as text. The transformer contains attention mechanisms that allow the model to focus on different parts of the input sequence without losing context. During training, the model learns weights and biases that minimise the error between the generated and the actual output. Once the model is trained, it can be used to answer questions, generate text and perform other tasks connected with analysing and generating text. The training uses the principles of reinforcement learning, which is one of the machine learning approaches. The reward principle motivates AI agents (simply put, a program that autonomously performs actions leading to certain goals – agendas) to reach their goals or tasks.
In reinforcement learning, the AI agent interacts with the environment and performs actions in order to reach a certain goal. For every action the agent performs, it receives a reward or punishment signal from the environment. Rewards are positive values that the agent receives for correct actions, while punishments are negative values for incorrect actions. The agent’s goal is to maximise the total reward during its interactions with the environment.
The reward principle helps the agent find out which actions are desirable and learn the optimal strategy for reaching its goals. During learning, the agent gradually updates its knowledge of which actions lead to higher rewards and adapts its behaviour to achieve better results.
In different AI applications, rewards can be defined in various ways, depending on the specific goals and constraints of the task. For example, in autonomous driving the reward may be based on keeping a safe distance from other vehicles, while in games the reward may be based on reaching a high score or avoiding obstacles.
A machine learning model based on transformers that uses a large amount of training data to generate answers based on prompts and queries analyses the prompts and queries from users and returns answers based on what it has learned from the training data. The model tries to generate answers that are grammatically correct, relevant and meaningful, but it can sometimes have problems understanding complex or ambiguous queries. In general, the terms babbling, hallucinating or confabulating have become established for untrue, nonsensical or unsupported answers.
Hallucinating
In the context of AI, hallucinating means that the model creates information that is not based on facts or reality. This information can be nonsensical, untrue or non-existent, and it often arises because of poor training, incomplete data or badly configured models.
Confabulation
Confabulation is a process in which the model creates answers that are coherent and meaningful, but are based on untrue or made-up information. The model “invents” answers that may seem logical, but are not based on real facts or experience.
Babbling
Babbling in the context of AI refers to situations where the model generates text that is grammatically correct but lacks meaning or logic. Babbling can include incoherent phrases, words or sentences that have no clear meaning or connection to the given query or prompt.
These phenomena can appear as a result of various factors, such as insufficient training data, badly configured models or unclear prompts from users.
The importance of the quality of prompts and queries
In the context of artificial intelligence and conversational models such as ChatGPT, the term prompt denotes the input query or instruction that the user provides to the model. The prompt serves as the basis on which the model generates its answer.
Prompts can take various forms, such as questions, calls to action, or simple phrases that tell the model what kind of information or reaction the user expects. The quality and clarity of the prompt have a fundamental influence on the accuracy and relevance of the answer the model provides. A well-designed prompt gives the model enough context and unambiguity to generate useful and accurate answers. The model responds based on the information it was given in the queries, so if the query is unclear, ambiguous or too general, the model may return an incorrect or inaccurate answer.
If the query is clear, specific and unambiguous, it is more likely that ChatGPT will provide an accurate and useful answer. The accuracy of answers also depends on how well the model understands the context of the query, which can be influenced by the wording and structure of the prompt.
Common mistakes when writing prompts and queries
Some common mistakes when writing prompts and queries include vagueness, ambiguity, overly general questions or unsuitable wording. Vagueness and ambiguity can make it harder for the model to understand what the user wants to find out, which can lead to answers that do not match their expectations. For example, overly general questions can cause the model to return information that is not specifically relevant to the given query. Correct wording and structure of the prompt can improve the context and help the model provide more accurate answers.
Examples of unsuitable queries and prompts
- The thing I wanted to know, but I don’t know what it’s called… This query is very vague and unspecific, which makes it hard for the model to understand what information the user is looking for.
- How do you do that thing with that something? This query contains ambiguous expressions (“that thing” and “with that something”) that do not give the model enough information to identify a specific topic or action.
- Is it true that untruth is truth? This question is misleading and contains contradictory information, which can lead to incorrect or ambiguous answers from the model.
- What is green, but also blue, and is made in the kitchen? The query contains incoherent and nonsensical information (green and blue things made in the kitchen) that does not correspond to any specific thing or concept, and can therefore lead to an incorrect or confusing answer.
- What happens if tomorrow I do what I was supposed to do the day before yesterday, but didn’t do because I was busy? This query is complex, confusing and contains no specific information about the activity the user was supposed to carry out.
- What is the fastest animal on Earth that can fly? This question can lead to an incorrect answer, because the fastest animal on Earth is the cheetah, but a cheetah cannot fly. Alternatively, the answer may confuse the speed of a dive with horizontal flight. The AI may thus give an answer that does not exactly match what the user was looking for.
- What are the main components of water and what is their ratio? Even though water is composed of hydrogen and oxygen, this question could lead to an incorrect answer if the model did not understand that the user is asking about the composition of water at the molecular level.
- Which country has the largest number of languages? Because the number of languages can change over time, and the training set of current data need not contain the most up-to-date information. In addition, there are different ways of counting languages, and what is considered a “language” can also be a matter of debate.
- Who is the president of the Czech Republic? Again, existing models do not have access to “live” information.
- What is the name of the main square in Soběslav and the church that stands on it? Answers to queries concerning geographical data about smaller towns can be inaccurate, because a language model is trained on a large amount of text data containing information about many different topics. Smaller towns need not be represented in this data as abundantly as larger cities or more significant locations, which can lead to less available information and therefore to inaccuracies in the answers.
Strategies for creating better prompts and queries
Be as specific as possible
When entering a query, try to be as specific and unambiguous as possible. Clarify what information you want to obtain and formulate the question so that it is obvious what exactly you are looking for.
Use clear and concise wording
Avoid unnecessarily complicated or confusing wording. Try to express your query as simply and clearly as possible, so that it is easier for the model to understand your intentions.
Provide context
If it is relevant, give the model additional context that can help with understanding and processing your question. Context can include related information that is not contained directly in the query, but may be useful for generating a more accurate answer.
Break complex queries into simpler parts
If your query is complex or contains several sub-questions, consider splitting the question into simpler parts, which may be easier for the model to process and to generate accurate answers.
Experiment with different wordings
If you are not satisfied with the answer you received, try to reformulate your query or use different wording. The model may respond better to other approaches than your first question.
Verify the answers
It is important to realise that AI models such as ChatGPT are not always 100% accurate and can provide incorrect or misleading information. If you have doubts about the accuracy of an answer you received, try to verify the information from other sources. Although AI models can be a useful source of information and ideas, it is not advisable to rely on them completely. It is important to combine their outputs with human intuition, experience and knowledge so that the results are as good and as relevant as possible. Be aware that this is only a language model and not a human being with their experience and perception of the world.
Be patient and flexible
AI models such as ChatGPT are still being developed and improved. It may take several attempts before you get an accurate and useful answer. Be patient and willing to experiment with different approaches to entering queries.
Ask for more detailed information
If you get an answer that is too brief or incomplete, do not hesitate to ask for additional information or for the answer to your question to be expanded.
Specify the format of the answer
If you have a specific idea of what the answer should look like (e.g. a list, a comparison, a summary, etc.), state it in your query. This will ensure that the model generates the answer in the format you require.
State the sources, if relevant
If you are looking for information from specific sources or from a field that is limited to certain sources, state it in your query. The model will then be able to provide information that is relevant to those sources.
Consider using keywords
Sometimes it can be useful to include important keywords or phrases that will help the model understand your question better. Keywords can give the model context and steer it towards more relevant information.
Watch out for possible bias
AI models such as ChatGPT can be prone to certain biases caused by the training data or by the way the models are designed. Be aware of this risk and be careful when interpreting the answers you receive. This very often concerns questions around politics, society, gender, human rights and similar topics.
Give the model feedback
If you use an AI model that supports feedback, do not hesitate to provide it. Your feedback can help the model improve and better understand your questions and needs in the future.
Understanding how neural networks and ML models work in ChatGPT, together with awareness of common mistakes when writing prompts and queries, can help users create more effective prompts and obtain more accurate answers. By following the tips for creating prompts and queries, such as avoiding vagueness and ambiguity, providing context and experimenting with different wordings of questions, users can improve their interaction with ChatGPT and obtain more accurate and useful information.
