Forecasting and AI: The Roles Technology and Humans Play in Anticipating the Future

Increasingly, businesses use the words "research," "forecasting," "futurism," and "AI" in the same conversations, which makes sense because all four deal with uncertainty, information and what comes next.

Research helps us understand what is currently happening or has happened in the past. Futurism expands the range of futures we can imagine, and artificial intelligence can accelerate these processes at remarkable speed.

Forecasting, though, does something a little more complicated as it tries to predict what the future holds so that better decisions can be made today in anticipation of what's coming.

And a question about forecasting that arises these days is this: To what degree can AI assist with forecasting, and are there times when AI is counterproductive to the forecasting cause? Part of the answer lies in developing a good understanding of how forecasting works in the world of business and trends.

Forecasting is not extrapolation

Forecasting is sometimes misunderstood as simply taking what is happening now and extending the line forward. That leaves out much of the complexity, though, because forecasting really starts to become interesting when the line stops being reliable.

People change, context changes, and technologies alter behavior. What looks like a fringe movement today can suddenly become emotionally relevant and mainstream, while something that is drawing enormous amounts of attention in the present can disappear as quickly as it arrived, and our future selves will barely remember it.

What forecasting does is try to understand the relationships between those movements before they have resolved themselves. That requires not only research, but pattern recognition, historical comparison, cultural sensitivity, intuition, imagination, synthesis and, perhaps most importantly, understanding human emotion.

Forecasters may be looking simultaneously at technological changes, language shifts, economic pressure, and patterns in family life. They may be studying political sentiment, entertainment, architecture, or the way people talk about themselves. These things do not arrive conveniently categorized in one dataset, so the work is in seeing when these varied things begin to belong together.

Relevance is the constraint

Forecasting gives organizations a way to work with what is forming, rather than waiting until something has become obvious enough to measure. But forecasting is not unrestricted imagination the way futurism is.

Contemporary futurism often uses scenarios, speculative worlds and alternative systems to stretch what we think is possible. That can be useful because it can expose assumptions, show consequences and make radically different futures imaginable.

Forecasting, on the other hand, operates under the constraint of relevance, asking not only what could happen, but whether a possibility connects to enough of the conditions around it to gain traction.

In other words, how likely is it to come to fruition? Is the technology ready? Is the economic environment helping it or resisting it? Is this idea relevant in this city, this market, this moment?

A useful forecast can tell a business that a possibility is too early, that a technology has emotional resistance around it, that an aesthetic is beginning to carry a different meaning, or that something currently marginal belongs to a much larger movement.

A forecast allows decisions to be made before the answer is obvious, and increasingly, this is where AI's potential role in forecasting becomes interesting.

AI can find patterns, but does it know which ones matter?

AI is becoming extraordinarily good at parts of research and analysis. It can summarize thousands of pages, cluster qualitative interviews, compare markets, identify repeated language, generate hypotheses, find precedents, produce scenarios, and suggest connections at a speed human researchers can't match.

But AI systems perform best when tasks are clearly specified and success can be evaluated. They are less effective when the work is messy, contextual or depends on holistic judgment. AI can tell us that a pattern exists, suggest possible explanations, retrieve historical analogies, and even make connections we did not see ourselves.

But a human still has to ask whether the pattern is meaningful, whether the analogy is appropriate, whether the context has changed, whether what looks like recurrence is merely coincidence, and whether the direction being proposed actually connects to the emotional world people are entering.

There may be an even more consequential effect of AI on creative and strategic work.

For a long time, production was expensive because research, images and scenarios all took time, and generating 20 possibilities could take days or weeks.

With AI, we can now produce hundreds of possibilities almost instantly, which sounds impressive but creates its own problems. When possibilities become almost infinite, the ability to select the most promising ones becomes even more important. Which among that mass of options deserves attention? Which possibility has emotional relevance and not just statistical plausibility?

Answering such questions requires human expertise that has been built by repeatedly remembering, comparing, being wrong, recognizing recurrence and understanding how meaning changes when context changes.

AI can participate in that thinking, but handing over decision-making to a system because it can generate convincing answers is a mistake that confuses fluency with judgment, and this matters beyond forecasting.

Chief, the networking organization for senior executive women, recently surveyed more than 1,000 senior female leaders. They said that AI is making certain skills more valuable, not less. Those skills include judgment, empathy, and critical thinking.

Most of the leaders surveyed also said they are more valuable as a leader because of their human skills, not in spite of them.

That's true for forecasting as well. AI can help us process an unprecedented amount of information, but it's up to humans to ask what that information means and how it can be applied toward determining what comes next.

About the author

Marieke van der Poel is the founder and chief creative strategist for Proef, a strategic forecasting consultancy with offices in San Francisco, Los Angeles, and Rotterdam.