Suppose we had a machine that was really good at interpolating in the space between domains. If the math in a physics problem starts looking an awful lot like the math in an obscure math paper, the machine is going to pick up on that and solve the physics problem very efficiently. If some quirk in sales numbers can be explained by patterns studied in sociology, the machine ought to propose that mechanism as a possible explanation. I don’t see any fundamental reason why an LLM would be incapable of acting as such an interpolation machine.
In grad school, I spent years trying to get ML models to tell me when they were wrong. I kept learning the lesson that models are great at interpolating between the data they’ve seen, and terrible at extrapolating beyond it. LLMs are no different.
For a long time, I thought of that as a limitation. Now I think it might not matter much, because I can’t name a single research idea that wasn’t, at some level, an interpolation between things we already knew.
Off the map
During my graduate studies, almost everybody I collaborated with was interested in “uncertainty quantification.” That term isn’t actually that well defined. What do people mean when they ask for this? Basically, if you have a model of some kind, people want the model to say when it’s wrong. They want some quantitative thing they can point to and say, “Look! The model’s certainty about this is only 3/10, so I don’t believe the result!”
There are all sorts of ways you can try to do this, and I tried quite a few. Gaussian process regressors worked great in some cases; some clever ensembling techniques worked well in others. Conformal prediction was probably the best, but they all give you pretty similar behavior if you calibrate them properly. Also, if you want to use any of them, you should know if you’re trying to quantify epistemic or aleatoric uncertainty.
I’m being a little tongue-in-cheek here, and perhaps venting about some old frustrations. All this is to say, I’ve thought a lot about uncertainty, where models do well, and where they do poorly. What I’ve found, pretty consistently, is that all ML techniques are relatively good at interpolating between data, and very, very bad at extrapolating away from the data. Uncertainties would always seem reasonable within the domain of data, and then just outside of that, would explode to huge magnitudes or otherwise behave unpredictably or uselessly.
Everything old is new again
While I haven’t studied it formally, in my experience, large language models don’t act much differently. They seem to be really good at recalling facts, techniques, processes, or patterns they’ve been trained on, and generally pretty poor at everything else. The advantage of an LLM is that it can get trained on basically everything. We communicate in language, so language has developed in order to represent everything, meaning LLMs actually can get pretty good at math, science, humanities, business, and everything else people write about.
Recently, there have been some facetious hypotheses that LLMs are really good at writing software because humans have written very similar software to solve the same kinds of problems over and over again, and continue to do so today. I think this is probably true, to some degree. Old algorithms have to be rewritten in new frameworks, and every new cloud app must solve very similar horizontal scaling problems.
This applies to research too. Research does not come ex nihilo. It is meaningfully derived from that which is already known and that which has been tried before. How many papers have been written where an existing technique is applied in a new context, or a very standard statistical analysis is run on some newly gathered data? I say this not to minimize this type of work; it is both important and hard to see these connections, and I myself very much enjoy doing it! These examples demonstrate how research can often be thought of as interpolative.
Perhaps there is some kind of research advancement that is so totally and utterly unforeseeable that it couldn’t be described as an interpolation between some knowns. In reality, I struggle to name any such research. I believe that all research is, to a meaningful extent, an interpolation between things that are known.
The Interpolation Machine
LLMs have the potential to be very powerful interpolation machines. They can have lots of domain knowledge baked into their weights (or RAG databases), and, like any ML model, they can act as great interpolators. I think they just need the right kind of training and prompting to do this more often and more reliably.
I believe the interpolation machine has the potential to be the most powerful research tool created so far, and I believe that we can build one out of an LLM. We are on the cusp of an age where measured anomalies can be automatically explained by new physical theories built from esoteric mathematical theorems, where machines can set up and run just the right quantum mechanics simulations to understand how a proposed therapeutic molecule might work, and where human curiosity about the world leads directly to new inferences. What an exciting time to be a researcher!