Machines Learning: The Rise of Generative AI

Share this article

It may seem as if large language models such as ChatGPT arrived overnight, astounding us with their ability to perform tasks such as writing a novel, suggesting a recipe or summarizing complex research. But there’s more to come: These models only represent the tip of the iceberg, with much more powerful generative artificial intelligence tools in development. Despite the exciting potential, there are concerns, too.

Lou D’Ambrosio, Head of Goldman Sachs’ Value Accelerator, recently sat down with Dave Ferrucci, an award-winning AI researcher to learn more about the opportunities and challenges of large language models.

Between 2007 and 2011, Dr. Ferrucci led a team of IBM experts and academics in the development of the Watson computer system, which defeated the best contestants of all time from the television quiz show Jeopardy! In 2015, he founded Elemental Cognition, an AI company focused on deep natural language understanding.

This interview was originally published by our colleagues at Goldman Sachs Asset Management as part of their Perspectives series, which features insights on diverse topics of impact within this dynamic economic environment.

D’Ambrosio: Where did the language models that everyone is talking about suddenly come from?

Ferrucci: They’re a big step for artificial intelligence, to be sure, but they haven’t appeared out of nowhere. People have been working on AI for decades, but what we’re seeing now is a watershed moment. That’s because the ability to master language is one of the things that’s so closely associated with human intelligence, and these large language models can do just that.

D'Ambrosio: How do these models work?

Ferrucci: Large language models are an application of deep learning. Working in a predictive way, they look at word patterns in large bodies of training data to compute the probabilities that certain words would follow a particular sequence of other words. They do it so well because of the vast amount of data they’re trained on. This creates incredible fluency that is consistent not just with the language in the training data, but also with the prompt the user specifies. Then of course you can respond to their output, and you get a dialogue. They’re not thinking like humans do but, nonetheless, the effect is dramatic and significant.

D'Ambrosio: What are the implications for business?

Ferrucci: Language is vital in the acquisition of knowledge. The communication of ideas from human to human is through language. As a result, language models could impact fundamental business functions around summarizing, synthesis, explanation, and delivery of key data and insights.

I think companies can’t afford to ignore what’s going on. The specific solution depends on the nature of their business, and experimentation will be required. A key question is whether the product or service being offered is something that AI can directly replicate. Beyond that fundamental question, companies may need to consider their workflows and processes to identify low-risk experiments to identify viable opportunities.

D'Ambrosio: What kind of roles will be affected?

Ferrucci: The reporting role of many middle managers who act as go-betweens could be affected. The value chain starts to shift because the decision-maker can get the synthesis, the summary, the aggregation and the delivery of information from a machine and at a much lower cost.

Existing databases and computer systems already have powerful, effective and reliable computational techniques, but they require an intermediary between the decision-maker and the use of that technology. Large language models can now help decision-makers communicate fluently with these systems to conduct analysis and reach conclusions.

And everyone has been talking about the impact on creative roles. Imagine the role of writers in sales and marketing roles—much of what they do may be repetitive and formulaic. These models can take on these tasks at an extremely cheap cost, so writers can shift their focus to the 20% of their role that involves coming up with new ideas to really add value.

For more examples take logistics, where complex issues in travel, healthcare plans, finance or design previously required multiple human interactions. Now, a machine can solve these problems. And call centers are now using machines to deal with 90-95% of incoming calls, up from 40-50% in the past.

D’Ambrosio: Could a company develop its own large language model to take advantage of all the capabilities, but retain some control of the training data?

Ferrucci: I think the expectation is that we’ll see language models that specialize in a particular area and that are trained within a company based on that company’s content, potentially using what are called foundational language models to train them on how general language is structured. But then you add a layer on top of that to train them with the specific knowledge within a company or to only focus on certain subjects.

D'Ambrosio: How is AI affecting areas like research and biopharma?

Ferrucci: Producing well-evidenced, high-quality research in areas like investments or drug discovery that previously took months can now be done in potentially days, hours or even minutes.

But language is also about more than words. Language involves sequences and patterns that provide meaning. So another area is protein folding. The way proteins fold follows a series of predictable patterns. Large language models use powerful deep learning techniques to learn these sequences of patterns, so they can help us understand proteins and how they fold. This means they have a big role to play in discovering new drugs and understanding disease.

D'Ambrosio: What’s the role of AI in education?

Ferrucci: You can get educated on any topic at present, but the main problem is it’s not personalized to the individual student. But imagine that you’re able to take all the content out there and make it dynamic, interactive and personalize it to every student. Doing so will result in huge improvements to education in both academia and corporate settings.

D’Ambrosio: How much more can large language models improve?

Ferrucci: I think we will reach a point of diminishing returns, and I think the value will come from how effectively we adapt models to solve specific problems. We can train the models on a huge amount of human language, but how do we use them effectively in business? How do we integrate them with other techniques and other technologies to be effective? That’s where a lot of the excitement will likely lie in the future.

D’Ambrosio: What do we need to be careful about?

Ferrucci: We still need to figure out all the possible implications. I think the people who best understand these models have legitimate concerns about privacy, security, reliability and trust. The technology is already easy to use and pervasive, so I don’t know how practical it is to pause current progress and development, as some people are suggesting. It’s important to acknowledge these concerns, be aware of them and step back and think them through. I think many of them will be worked through. But we need to remember these are generative systems — they’re generating new things. They could pick up information that is not true and develop things based on this false information. It’s not the same as search.

D'Ambrosio: Where do you think we’ll be 10 years from now?

Ferrucci: Today, we’re still in the phase in which humans are training machines. But these models are going to be so fluent and capable of synthesizing language that machines may eventually be training humans. Machines could be teaching us new skills, educating us on a whole range of topics. At the same time, regular people may be able to program computers to perform a wide variety of tasks just by talking to them.

In fact, I don’t see a future for humanity without artificial intelligence. In my view, it’s the most fundamental tool for the advancement of the human species. That’s not to say that it won’t be a bumpy ride, or that we don’t have a lot to learn along the way, but I see it as our destiny to work it through.

This article is for informational purposes only and is not a substitute for individualized professional advice. Articles on this website were commissioned and approved by Marcus by Goldman Sachs®, but may not reflect the institutional opinions of The Goldman Sachs Group, Inc., Goldman Sachs Bank USA, Goldman Sachs & Co. LLC or any of their affiliates, subsidiaries or divisions.