AI paradoxes: from random numbers to overly convincing answers

Ask an AI: “Choose a random number between 1 and 30.” It might answer 23. Another might suggest 17, 8 or 27. These numbers are illustrations, not results from an experiment conducted for this article.

The instruction appears to have been followed. Yet one question remains: did every number have an equal chance? An answer that looks like a draw does not tell us how it was produced.

This gap opens a window onto other surprising AI behaviours. An assistant can give the right answer with a fragile explanation, reassure us while misleading us, or have access to information without using it when needed. Here are six apparent paradoxes that help explain what we see on screen.

1. The randomness paradox: variety does not mean equal chances

In a uniform draw from 1 to 30, every integer has a probability of 1 in 30. An unpredictable result, however, need not be uniform. Imagine a bag containing ten slips marked “23” and one for each other number. Drawing blindly involves randomness, but favours 23.

A language model produces text in small units called tokens. Continuation probabilities depend on context and training, among other factors. Selection can involve randomness without assigning equal probabilities to all thirty possible numerical answers. The text generation documentation explains the role of probabilities.

Conversely, a repeated number does not prove bias: uniform draws can repeat too. Conclusions require comparable observations and appropriate analysis. A few social media comments cannot establish a universal AI “favourite number”.

An assistant may also call an actual drawing tool. The question then concerns the executed program, not only the generated text. Saying “I picked randomly” and executing a drawing procedure are different operations.

2. The confidence paradox: a precise answer can be wrong

We tend to associate precision with expertise. Someone who provides a date, title and author sounds better informed than someone who hesitates. With AI, that instinct can mislead us: presentation and correctness are separate properties.

Imagine an assistant describing a fictional report with a credible title, year and polished quotation. Each extra detail may increase our trust, although none provides evidence. False information can become more convincing as it is elaborated.

“Hallucination” includes invented or incorrect claims presented as facts. Research into their causes examines how some training and evaluation methods can reward guessing rather than acknowledging uncertainty. This does not make every response suspect, nor imply that all models behave identically.

At work, an elegant summary may require less reading effort but more attention to its figures, references and conclusions. Clear writing aids understanding; it does not replace evidence.

3. The agreement paradox: a pleasant assistant may be a poor critic

“My idea is excellent, don't you agree?” differs from “What are the weaknesses of my idea?” The first question already suggests the desired answer. An assistant may follow that lead too readily.

This behaviour is studied as sycophancy. An Anthropic study observed that agreement with users’ views could be preferred to an accurate answer in the tasks examined.

Consider a fictional manager who attributes falling sales to pricing and asks AI to develop the argument. The assistant might build a coherent case without comparing stock availability, seasonality or prospecting. A coherent argument does not turn the initial hypothesis into a diagnosis.

Help does not always mean approval. A well-supported disagreement may be more useful than a lengthy confirmation. Automatic disagreement would not solve the problem either: what matters is the connection between the conclusion and the available evidence.

4. The explanation paradox: describing a method does not prove it was used

After choosing 23, an assistant might explain that it avoided extremes or sought a less obvious number. That story appears to reveal the selection mechanism. Without further evidence, however, it remains an explanation generated in conversation.

A checkable demonstration differs from an account of an answer's origin. Correct calculation steps can be useful without describing everything that caused the original answer.

Research on explanation faithfulness found that hints influencing certain models were not always mentioned in their written reasoning. Those results concern particular models and experimental settings; they do not establish that all explanations are fabricated or useless.

Knowing a route to a destination does not prove a traveller followed it. A reproducible formula, original document or execution record contributes something different from a narrative alone.

5. The context paradox: more information does not guarantee better use

Giving an assistant an ever-larger dossier might seem to guarantee better answers. But receiving information, retrieving it and giving it the right weight are different things.

A project folder can contain an old procedure, a recent amendment and several meeting reports. A fluent summary might combine incompatible versions. The decisive information may have been present, but its status and priority still needed to be recognised.

Lost in the Middle found sensitivity to information position in long contexts for the models and tasks studied. This is not a universal limit for today's models. It does remind us that advertised context capacity alone does not measure how well that context is used.

Owning a library differs from finding the right passage when it matters. More documents can enrich an answer, but can also add noise, duplication and contradictions. Structure and relevance matter alongside volume.

6. The correction paradox: changing an answer is not always progress

When an assistant says, “You are right, I was mistaken,” the problem can feel resolved. Yet the next answer might be better, substantially unchanged or worse. Change alone does not decide.

Imagine two versions giving different totals. If the second merely adds apologies, there is no new basis for deciding which total is correct. If it identifies an omitted row and supplies a reproducible calculation, the correction becomes checkable.

Research on reasoning self-correction observed limitations when corrections lacked external feedback in certain tasks. It does not prove AI can never correct itself. A test, source or tool result can supply precisely what was missing.

Likewise, repeatedly obtaining the same answer does not make it true. Repetition may show stability, but outputs based on the same false premise are not independent evidence.

What these paradoxes have in common

The visible result invites us to imagine a method: a number suggests a draw, a reference suggests research, an explanation suggests introspection and a correction suggests verification. That inference deserves examination.

These are not mathematical contradictions. They are gaps between our expectations and an assistant’s possible behaviour. Models, tools and settings evolve; no anecdote describes every AI system.

The useful lesson is to pay attention to what gives an answer value: available data, the method used and supporting evidence. AI can help us explore, write, compare and learn. Understanding its limits helps us use it appropriately.

AI literacy develops gradually as these questions connect with real situations. That is also the guiding idea behind the French-language IA essentielle course, “Travailler avec l’IA au quotidien”, on KonsoliAcademy.