
Technology & Innovation In Silico, Part I - Understanding AI’s “Hello, World!” Moment
In Silico is a multi-part series discussing artificial intelligence, its economic and financial impact, and its role as a driver of change.
In Silico is a multi-part series discussing artificial intelligence, its economic and financial impact, and its role as a driver of change.
In this article we explore Generative AI technologies with a view to understanding their commercial uses and economic impact. For an introduction to the series please refer to 'In Silico, Part I: Understanding AI's "Hello, World!" Moment'. For an introduction to Machine Learning and Deep Learning, which underpin Generative AI, we strongly recommend reading 'In Silico, Part II: AI and the Quiet Revolution of Machine Learning'.
In previous editions of this series, we’ve explored exactly what AI means—but generative AI is a relatively new idea to most readers. While enthusiasts might know "generative AI" from image generators (like DALL-E, Stable Diffusion, or Midjourney), ChatGPT was the first time most of the world encountered "generative AI." As a freely accessible chatbot with uncanny language capabilities, ChatGPT took the world by storm, reaching one million users in just five days.
Machine Learning is separated broadly into one of two broad categories: Discriminative AI and Generative AI. Discriminative AI is already in use across a wide variety of functions in today’s economy, a phenomena we referred to as the ‘Quiet Revolution’ in our second installment of In Silico.1 Discriminative AI is used to classify, categorise, and analyse data, helping users and analysts understand the relationships between variables, make predictions about possible outcomes given some set of variables, and understand whether some new data point is more like known entity A or known entity B. In Figure 1, we show how a discriminative model may be used to understand the separation of categories of data.
Generative AI, meanwhile, is used with the goal of creating (generating) new data that looks like its training data. A generative algorithm learns the relationships between variables or similarities between datasets not to analyse and categorise, but to infer about the general nature of the data and “generate” output data based on the model’s understanding of the data. Models trained on human language can recreate new, plausible language data. Models trained on image data can recreate new, plausible image data. In Figure 2, we show how a generative model may be used to learn from an existing dataset and infer the next datapoint given an input.
Source: Invesco. For illustrative purposes only.
Generative AI has been around for some time, but only recently have models become sophisticated enough to compete with human performance.
Most notable for today’s excitement are Generative Pretrained Transformers (GPTs), which are the foundation of OpenAI’s ChatGPT. The technology behind GPTs evolved over many decades, but perhaps one of the most important steps along the way was a 2017 paper from Google2 that pioneered the concept of transformers. This technology allows machine learning models to better interpret text by relating each word in a given dataset back to previously seen words, enabling Deep Learning (covered in Part 2 of this series) to demonstrate better understanding of text data.
At their core, GPTs are models built on the same technology and trained with large amounts of unlabeled text data. The first model was launched in 2018 by OpenAI in the form of GPT-1, with successive improvements released through GPT-2 in 2019, GPT-3 in 2020, GPT-3.5 in 2022, and the latest GPT-4 in March 2023. Google built on a similar transformer-based technology through its Pathways Language Model (PaLM) announced in 2022 and its Language Model for Dialogue Applications (LaMDA) which was first launched in 2020 but entered the limelight in February 2023 as the foundation to Google’s Bard chatbot, a ChatGPT competitor.
As may be evident already, there are a wide range of techniques for approaching generative AI models. Collectively, these are referred to today as large language models—language models that are trained on huge volumes of text data. Colloquially, we may see these simply called “GPTs,” even if this misses some technical accuracy.
Following successive advancements in computing power, data availability and capture, and more sophisticated models and techniques, generative AI has now reached a level of performance that rivals human capabilities. In a very short period, there has been an explosion in AI tools for businesses and ordinary consumers alike. While we are still in the very early days of AI for the masses, it appears that two loosely overlapping trends are emerging which can broadly be classified as B2C (AI as a Service, where users plug-and-play with AI through apps) and B2B (AI as a Tool, with Foundation Models leading the way).
While ChatGPT and its GPT-4 foundation model have stolen the limelight, there are already numerous generative AI systems for a variety of modalities. The most prevalent example is text, but there are also models for programming, images, audio, video, scientific research, and more.
Below we provide examples of popular modalities and where we see generative AI already making headway:
As businesses pursue generative AI for use in products, the broader economy may benefit from the effects of implementations. AI may augment or eventually substitute current labor practices. Productivity, as measured by total factor productivity (TFP) growth, may rise as workers are paired with AI tools that make workers more efficient in their day-to-day tasks.
In one example, Github—a code collaboration and repository website—paired developers with an AI coding assistant branded as CoPilot. The study found that those using the AI system performed a given software development task 56% faster than the control group.3
The hope underlying this is for a productivity increase similar to the information and communication technology revolution, which saw productivity rises at a rate three times that of recent decades. Indeed, AI may meaningfully uplift economic growth and help offset the headwinds from slowing population growth. We will treat this topic with greater detail in Part 4 of this series, where we will explore the macroeconomic implications of recent advancements in AI.
Sources: US Bureau of Labor Statistics and Brookings Institution, as of 31 December 2022.
While we see meaningful opportunity from recent generative AI tools, we must also recognise the weaknesses and challenges that remain in such tools. We expect—and hope for—continuous improvements that minimise or solve the shortcomings noted below, but for now these remain considerable roadblocks to further adoption and integration. In its current form, we view AI as a tool and a fascination, not as a replacement for information workers and services.
Going forward, we expect that large, multimodal generative AI systems are likely to be packaged as foundation models on which companies can build custom solutions and users can interact with apps. In essence, these are pre-packaged platforms which companies can use as a “foundation” for more specialised solutions.
For now, Foundation Models (whether OpenAI’s, Meta’s, Google’s, or others) have differences. But they all use natural language, and they are increasingly trained on the same or very similar data. Trillions of parameters for training represents a mind-boggling amount of data, but the world is surprisingly repetitive. How many songs sound alike? How many sentences are used over and over? In a universe where a handful of variables usually have enormous explanatory power, there is every chance that a large but finite and entirely reasonably sized data set is a sufficient model of the world that a data set of equal size is functionally indistinguishable. In that world, network effects and user experience might be the only meaningful differentiating factors left.
Generative AI is the overnight success almost a decade in the making. The earliest models appeared around 2014. 2022 was the year of Generative AI’s debut, but only with the release of ChatGPT did Generative AI capture our collective imaginations, quickly leading to an explosion in AI tool development and use. Now, half-way through 2023, AI has caused a veritable frenzy in markets, with “AI” companies outperforming in the tech sector and outperforming across a variety of indices. In our view, we are likely entering (or already in) an AI bubble. We also believe that AI-related plays might pay off in the long-run, but we would strongly caution investors that, while some AI technologies have an established history as tools in tech companies, business models like 'AI as a Service' and dedicated AI tools for business and consumer use are yet to be proven. We would also caution investors that many AI technologies, especially generative technologies, exist and operate in a changing regulatory landscape which may further impact any expected returns.
Sources: Bloomberg and Macrobond, as of 20 July 2023.
Notes: Data has been re-based to 31 December 2019. The 'S&P 500' Index is a market capitalization-weighted index of the 500 largest US stocks. 'S&P 500 ex-Tech' subtracts tech stock performance from the S&P 500's performance. 'Mega Cap Tech' refers to the NYSE FANG+ Index, a rules-based, equal-weighted equity benchmark that tracks the performance of 10 highly-traded growth stocks technology and tech-enabled companies across the technology, media & communications and consumer discretionary sectors.
Past performance does not guarantee future returns. An investment cannot be made directly in an index.
Investment risks
The value of investments and any income will fluctuate (this may partly be the result of exchange rate fluctuations) and investors may not get back the full amount invested.
See In Silico Part II: AI and the quiet revolution of machine learning.
Attention Is All You Need, Vaswani et al., June, 2017 (revised 6 December 2017)
Source: Github: Research: quantifying GitHub Copilot’s impact on developer productivity and happiness from 7 September 2022.
Source: Harvard Business Review, All the Ways Hiring Algorithms Can Introduce Bias, 6 May 2019.
Source: Bloomberg, How Fake AI Photo of a Pentagon Blast Went Viral and Briefly Spooked Stocks, 23 May 2023.
In Silico is a multi-part series discussing artificial intelligence, its economic and financial impact, and its role as a driver of change.
To understand where the real value of AI is, this article will look at the quiet revolution of AI that happened right under our noses.