Human beings want to make sense of chaos. We want explanations, patterns, and some sense that we know what is coming next. However, wanting an explanation does not necessarily mean wanting what is true and factual.
That was one of the ideas that stood out to me during my conversation with Oxford professor Carissa Véliz. Her latest book Prophecy, Prediction, Power, and the Fight for the Future: From Ancient Oracles to AI covers the ethics of AI, privacy, and the ways power and influence shape technology and vice versa. Her work takes a much longer view of prediction by tracing the human desire to know the future from ancient oracles to algorithms.
Véliz states that as human beings, we do not necessarily prefer what is true but rather what is familiar. So, when it comes to outsourcing more of our thinking to Large Language Models (LLMs) and possibly future artificial general intelligence (AGI) – defined as a hypothetical computer system that matches or exceeds human cognitive abilities across any domain – this is an important consideration.
Prediction itself is not inherently bad; it’s a core part of the scientific process. A hypothesis is a type of prediction itself. Within this process, we expect to observe something, but are open to it being challenged.
This kind of prediction is something we carefully test, taking care that outside factors do not confuse the results and understand that it may not prove to be accurate or true. Yet, when it comes to predicting human behaviors and social dynamics, problems emerge. People are, inherently, impacted by predictions themselves, which can have an effect on future behavior, whereas something like a molecule or even the weather would not respond the same way.
Véliz uses elections as an example of this. Polls are used to predict voter outcomes, but these same voters may then be affected by that very prediction. This becomes problematic as democracy depends upon uncertainty; otherwise, why would we be voting if it is already decided? This brings Véliz’s central point back to the present, as AI becomes increasingly integrated into our daily lives, doing more than it has before.
The Answer We Want Versus The Actual Answer
Large language models can produce for us what sounds coherent, confident, and authoritative, but this doesn’t mean it is actually true. Véliz emphasizes that their outputs are based largely on prediction. But most likely to be said is not the same as likely to be true.
“These systems were not made to track truth,” she said. “They were made to satisfy our preferences,” highlighting a distinction that is very easy to underestimate. They are programmed by humans directing algorithms.
Human beings like explanations that fit together. We prefer confidence over uncertainty, simplicity over complexity, or an answer that confirms what we already believe. A system trained partly around what people prefer (or maximizing emotional reactions or attention) can become very good at sounding convincing without possessing an independent relationship to truth.
This is where I see a direct connection to undue influence.
The danger is not only that an AI model can give us incorrect information but rather when we depend on outsourcing our critical thinking entirely, we are no longer deciding for ourselves. If a system repeatedly gives us satisfying, familiar, and instant answers we may simply stop critically assessing the output. I refer to this as an “external locus of control” vs. people having an “internal locus of control” grounded in our bodies.
Véliz also reminds us that many of these models are often created by people behind companies with financial and institutional interests that may not align with our own. An example she points to is when the UK Tech Secretary directly used ChatGPT for advice on policy, including why the UK may not be adopting AI as eagerly as anticipated. Clearly this situation raises obvious problems; it’s not a stretch to think that an AI system might have bias when it comes to self-scrutiny.
These systems are not the sole arbiters of truth. The claims made about their own future deserve the same scrutiny especially when we hear repeatedly from the tech billionaires that AI, itself, is an inevitability and that we simply must accept it.
“Ai Is Inevitable”, A Claim Of Power
Whenever we hear someone saying that something is inevitable, Véliz says that should raise a red flag.
“Nothing in the future is inevitable,” she states, especially when we are talking about technology.
Technology is not something handed down but something that human beings themselves create. Humans decide what gets built, what receives investment, what rules govern it, and what kinds of products we are willing to accept. We have seen many insiders of major AI companies defect and publicly express profound concerns about lack of ethical guidelines that align with human rights and law.
The history of the automobile is one that Veliz uses as an example. Early electric and gasoline-powered vehicles competed alongside one another. The gas-powered vehicle did not dominate due to any sort of predictable outcome, but rather the historical circumstances that allowed it to succeed. So, when powerful technology companies describe the future of AI as though the question has already been settled, it is important to remember a case such as this.
The conversation changes the moment we accept AI as inevitable. When we would typically ask, “Should we build these systems this way?” we start asking, “How quickly do we have to adapt?”. This means that instead of asking what kind of technology we want, we are encouraged to accept any outcome presented to us.
In my own work, I pay close attention to certainties and predictions. Authoritarian systems are not designed to empower people to think for themselves or to keep people curious about future potential and possibilities as they offer an absolute certainty. Only one answer, one path, one outcome.
Véliz’s argument pushes in the opposite direction. Current AI systems are products, not destiny. If they consume too much energy, collect too much personal information, fail to reliably distinguish truth from plausible fiction, or do not adequately protect users and societies, then the response should not be giving up. We should be able to have the agency to demand something better. But skepticism creates another challenge in that not every prediction is manipulation as some prove to be extraordinarily useful. So how do we tell the difference?
Prediction Versus Prophecy
Véliz offers us a comparison utilizing the predictions made by climate scientists and by those of technology executives, noting that the difference is in more than just their titles. Scientific prediction is built upon evidence, causal understanding, repeated testing, and knowing it’s possible to be proven wrong. An ethical and responsible scientist should be able to explain the knowns, the unknowns, and the evidence supporting their prediction, along with what could contradict it. For example, 97% of the world’s climate scientists state that human global warming due to the fossil fuel industries is causing havoc to people around the world – and they can provide evidence as to why.
Predicting physical systems and predicting people carry significant and important differences. Rain does not change its behavior when it hears tomorrow’s forecast. Human beings, however, respond to information about themselves as is their nature. Those very responses can change the very outcome being predicted, sometimes proving it to be true without realizing this.
Véliz’s advice is very straightforward in addressing this by stating, “When you hear a prediction, it’s time to ask questions.”
Consider things such as who is making the prediction and the evidence that supports it. Furthermore, who collected the data and information that might be missing, intentional or otherwise. Do we even know the mechanisms involved?
But perhaps most importantly, is there someone who stands to benefit from our belief in their prediction?
A trustworthy expert still makes predictions but with the willingness to also admit, “I don’t know”. Véliz emphasizes an expert with integrity is open to identifying uncertainty rather than hiding it. They can discern between what is happening now, what could happen next, and what we simply do not have the information to know at this time. Critical thinking does not mean automatically distrusting everyone, but in rather understanding what questions must be answered before trust is even warranted. As the more a system knows about us, the more confidently it can attempt to predict what we might do next so long as it has the information to do it.
The Future Remains Unwritten
In her earlier book, Privacy Is Power, Véliz examined what she calls the machinery of surveillance. Prophecy examines the machinery of prediction that surveillance feeds. Why we collect information about people in order to anticipate behavior allowing prediction to then become a means of influencing that behavior.
And that is why privacy is so much more than just a personal preference.
Véliz reminds us that we still have choices even when it may appear we don’t. We can still turn unwanted AI features off and select (or unselect) features so that we can maintain our own agency. We can support privacy-preserving alternatives and be mindful to engage in parts of life that make collecting our human data points increasingly more difficult if not impossible. A sense of preservation that she refers to as just reverting back to the analog. That which consists of physical books with pages to flip through, direct interpersonal relationships, tangible objects we own and can hold in our own hands. Afterall, why do authoritarians burn books? To prevent knowledge being transferred that will help empower people to resist tyranny.
We can still cherish the spaces that do not exist to observe us, but the spaces in which we can observe the world living in the present moment, above all else. Knowing that these relationships, which we have struggled to have in the past several years, can be preserved, almost recollected, instead of the information others have collected about us, knowingly or not.
This brings us back to the uncertainty that makes prediction so appealing in the first place as we tend to think that the unknown, by default, is frightening. However, Vélizsuggests we consider the opposite, in that unwritten future means there is still hope and a chance for change. That not all is doomed. Uncertainty means technology isn’tset in stone or predetermined. It can be redesigned, and even institutions can change. We as humans can refuse systems we don’t want and demand alternatives. Our future holds hope precisely because it has not yet been predicted.
I began our conversation thinking about prediction as something human beings seek because uncertainty is uncomfortable, but uncertainty is not something just meant for us to overcome. Perhaps, instead, when we look at the lack of certainty, we can view it as being our very strength, as the possibilities are still present. In not knowing, maybe so much more can be understood to be possible. Perhaps that is our greatest advantage.
Carissa Véliz, PhD is a writer, keynote speaker, and associate professor at the Institute for Ethics in AI at the University of Oxford. She works on the ethics of AI, privacy, business ethics, and public policy. She advises companies and governments around the world and is also a board member of the Proton Foundation and a member of UNESCO’s Women 4 Ethical AI.
Her book is Prophecy: Prediction, Power, and the Fight for the Future, from Ancient Oracles to AI and can be purchased on Amazon.
Social Media: LinkedIn | X | BlueSky | Substack
Further reading
Beware the Metaverse: Dr. Rand Waltzman discusses Artificial Intelligence (AI) and the Internet
Discussing Military Insights, AI, and Influence with Ryan McBeth
Rapid Advancements in VR and AI: Understanding Opportunities and Necessary Boundaries
Revealed: how the UK tech secretary Peter Kyle uses ChatGPT for policy advice | New Scientist