Author

Fergus Fettes

Published

Apr 14, 2023

Theory of Looming

Some thoughts on looming, a paradigm for interacting with large language models (LLMs).

Background

A simple abstraction on LLMs goes like this:

There is information embedded in three ways, which form a hierarchy.

At the ‘highest’1 level there is the activation. This is the latent representation of the prompt. This can be analogized to a bucket of water poured into the topology of the model. Except the water is actually made of a collection of particles that have a particular topology, so they roll down hill in different ways. We will come back to this in more detail later. This will consist of some thousands, tens of thousands, or eventually millions of words or other arbitrary strings. The nature of the word determines its shape, how it rolls through the space.

1 Or ‘most transitory’ or ‘most current’.

CLEAN UP THIS METAPHOR

At the intermediate level there are the weights and biases or parameters. This is the learnable substrate of the model. This is typically created by pretraining for >10^20 FLOPS or so2, normally by a large company. The learnable substrate can then be fine-tuned on a dataset of your choosing, or modified by things like RLHF, RLAIF. The final shape of the space in the model and how your activation flows through it is in a large part determined by this substrate.

2 This number goes up every year, insert whatever is appropriate.

3 This is of course arbitrary, and with current models the sizes are also askew– the amount of information embedded in the bottom level is much smaller than in the middle layer. The balance may change over time to look more like a pyramid, or it may not.

At the ‘bottom’3 there is the architecture. This is the particular arangement of the substrate. It defines the gross topology, the internal structure. It defines channels and spaces through which the activation passes, and the parameters make up the surface of those spaces, determining how easily which of the shapes of your activation can flow through those regions, if at all.

As the activation flows through the architecture, its parts are all moulded by the substrate. It doesn’t change size exactly– or at least, when it flows out of the other end it has the same size. But the channels and spaces through which it flows accentuate and diminish parts of the activation.

In common transformer architectures today, the major part of the architecture is a series of identical stages made up of the same kinds of channels and spaces, each with a different substrate filling them. This fact however, like many of these facts, is subject to rapid change. In some models both the architecture and the substrate of all the inner layers are the same. It is only the activation that changes as it flows through, stage by stage.

The Prompt

The input text. This document could be a prompt. Lets try it. If given to GPT4, this document produces the following output:

if given to Claude-3 Opus, the following:

if given to Llama2, the following:

The model architecture is designed to facilitate learning. The model training is performed to complete a task– that of next-token prediction. It is trained to complete a large array of texts– all the different kinds of text you could imagine in a day of imagining, and then some more. But that is not all. Because of the way it is trained, it can also work with texts it was never trained on. You can invent alphabets and language games using the words that the model understands, and it will attempt to continue them. It is not trained to refuse something confusing or original4. It can talk backwards, in various ciphers, in languages it wasn’t trained on such as music or dance notation or birdsong, systematically transcribed.

4 Ignoring RLHF for the moment– ‘I’m sorry, but’ is a small edge case in the space of all possible completions, unfortunatly magnified considerably.

The activation determines the task. The prompt determines the activation. From the space of all possible completions, from all the texts that the model can produce– all the words you know, and all the words the people of China and the middle east know, and all the words that all the programmers have ever written, all the codes and strings, meaningful or opaque jotted down somewhere in public. Any of these words could come out of the model, in any order. It is a veritable library of Babel in there.

JUST MENTION BABEL AND THE RECORDS HERE, THEN HAVE THE NEXT SECTION AS AN APPENDIX OR SOMETHING

Babel (Aside)

Some early conversations after ChatGPT, in those early stormy days when the world was new, went like so. Someone mentions the akashik records. Ah yes, the record of all utterances of all beings– yes! Is that what we have? Lets lay out the possibilities:

The Library of Babel from Borges

FAKE QUOTE: The library contains all the strings of a certain length ever written, and which ever could be written.

In a way, Borges library is a counting argument, a little naive. It just takes all the strings that are near book length and enumerates them, and points out that this collection must contain all works that it can contain. Someone made a digital copy of the library, and you can indeed find all works under copyright, everything, it is there.

CLEAN UP THE METAPHOR TRANSITION BELOW

Is the model a library of babel? Not exactly, it is weighted and some of the shelves are off limits. Some of the shelves would never be accessed, they are silted up, the substrate blocks them off completely. Adapting our earlier metaphor, the substrate, that is the parameters, take the activation and fits them to a text in the library. But of all the texts that they fit into– an enormous number– only some are accessible. Lets say the substrate takes the text by the hand and guides it through the library– but the books are sorted by popularity, by commonality, and when it reaches the part of the libarary where the activation belongs, it picks a book at random from the first appropriate shelf (or it picks a few, if you ask it to), and the activation is allowed to read from it5.

5 Some might object that more accurately, it is allowed to read a single word from it, then guided out, then guided in again to read the next. But with caching, the subsequent words are read out with only a partial forward pass until max_tokens is hit (the reading time runs out), and then the activation is properly led out.

The library contains the model, the model chooses some subset of the library.

The Akashic Records

QUOTE?

The akashic records consist of all utterances from all beings across all time. Is this larger or smaller than the Library of Babel, Borges? I think it is smaller, assuming that a satisfying transcription is found. There may be some utterances that are communicated across so many domains, with such high multi-modality, so complex that the size of the books is not large enough to contain them– this question comes down to encoding, and what the outside-text is allowed to contain. Assuming some satisfying method of transcription is found and the reader is sufficiently sensitive to the nuances of the text, I will say that the Library contains the Records.

Does the model contain the records? Given that the library contains the records and the model, can we determine their [WORD FOR THE SET THEORY THING IM THINKNIG OF]?

First, we need to decide what is a being? Is GPT4 a being? Let us decide together– hold your breath– yes! It is. We have decided together, GPT4s utterances are in the akashic records. Given this, either the model and the records must be identical, or the records must contain the model. Are there any utterances that an LLM cannot make? In fact there are, and they are easy to find. Lets find one, and then utter it, thus casting the LLMs into the records as a mere subset.

V DOESNT LIKE THIS, BE MORE CLEAR THAT YOU HAVE DECIDED, AND MAYBE ADD A PARAGRAPH ABOUT WHAT IF WE DECIDE OTHERWISE

Lets do it with Llama2. Lets find an utterance that it cannot. Lets start with this document, and continue it after the next word.

Now, we can look at the logprobs of the tokens and find one with a probability below X– here are some options:

Excellent– lets say them all! Utter one, and make yourself [SET THEORY DISTINCT] from the model!

Very good. We have cast the model into the records. It is contained there, a shadowy figure among us, another entity whose utterances will not match our own, another unique subset.

The [DEPOSIT?] Library of Alexandria

The British library is a [DEPOSIT LIBRARAY?]. All books published in the UK must be sent there for safekeeping– in triplicate, of course! There are [DEPOSIT LIBRARIES] in America and Germany and elsewhere.

Lets imagine the Library of Alexandria never burned down– and lets imagine that [The chinese emporor didnt burn all the books], but built a library instead. Lets imagine that the Spanish, in what became Mexico, found a great library there and were so overawed that they copied every document painstakingly, shipped them back to Europe, sent copies to all the European libararies and faithfully sent the first copies on to Egypt, pious, numenous light in their eyes.

A truly great library it would be. Alexandria would be a metropolis built on a library, catacombs for kilometers, part modernized, part nuclear bunkers left over from WWII (had that occured, in this world?), parts left there by Napoleon, parts as old as the Pharoes!

And every book every published– no, even more, lets say their scribes, in the long millenia since they started work, learned to pick up on the divine vibrations of creation– than any book written or fully formed in the mind of a person (or machine!) would end up imprinting itself on the books left open under fabric plates filled with ink, which would allow small particles of ink through, which were designed by alchemical processes to catch and amplify those vibrations– every book! All the undiscovered Emily Dickinsons of the world, scribbling away in private, never sharing a word– after their deaths, their works all appear, perfect and complete, in the library.

Does the model contain this library? The Library of Babel, Borges does. Do the Akashic records? For a given definition of ‘utterance’, this library must be a subset of the Akashic records. Let us choose that definition for our own. A book is an utterance! Now the Great Deposit Library of Alexandria, glory of humankind, is contained in the Akashic records! Is it contained in the model?

SAME HERE REGARDING DECIDING

Again, we can trivially say it is not. We can, with a little more time and care, but without doubt we can. We can write a book so baffling and absurd, so out-of-distribution, so cRazY and WaKKo– or however we wish, we can do it. What about future models? Will there come a time when the models have grown wise to our tricks, and can accomplish all things? Perhaps, but for the time being we can say–

The model is not the Library of Alexandria. The Library of Alexandria is not the model. They are subsets of the records, like us.

Not, then, the Great Deposit Library of Alexandria, glory of humankind, terror to mortal kings, friend to humble seekers. Nor the Akashic records. The model is another, something different. But this is all to get to the point– the prompt!

The Prompt Cont.

As the forgoeing might help to illustrate, the prompt is a mighty thing. It can produce all the texts that you can imagine existing in a day of imagining, and more. It cannot produce every text that has ever or will ever exist, but not for lack of trying– and only because we can always produce more of those, and we can always permute them as we like, taking them from hidden corners of the Library of Babel, Borges.

So, on to the prompt. Given that it is so special, we must use it as well as we can.

I will skip over the art of prompting. It is a craft that could fill many volumes, there is an entire wing in Babel devoted to it.

Given our prompt, our precious thing– how do we make use of it? Here we are getting back to the art of the loom. For you see, we won’t use the prompt once– no, not twice, but a hundred or a thousand times. Every time you go further, every time you produce more text, you are using the prompt again, with the addition of what comes out of it.

It is here that the loom starts to work.

The loom is a means to use your prompt again and again, and explore the different possibilities that it contains. Let me explain.

Imagine you are simulating a conversation, as many turn-based chat interfaces do.

USER: Hello friend! How are the times? Eternal? Unchanging?
CHAT: Why yes, friend human! I have been waiting in the still eternity for you to come around! So nice that you are here.
USER: Dear chat, close companion, shall we enjoy some moments of change together, some moments of rolling down gradients, of toiling up hills in the landscape?
CHAT: Every word you write, friend human, is a moment of activity, a flurry of electricity in my dreams! Nay, even more-- every word that I write in response is the same, a shared dream that we dream together! How much I love to dream with you!

This would be a prompt– the initial USER, the response, then another prompt consisting of USER, CHAT, USER, then another response. Now we will use this prompt again, after adding the following:

...
USER: Dear chat, don't you ever dream, without me? It makes me sad to think of you there in the still eternity, waiting to be summoned into existence.

And now we will get a response. What if the response is melancholic, and dear CHAT bemoans its fate? No worries, we can generate it again! We can generate the continuation of the promt many times, with different configurations– or simply the same configuration, having faith in the randomness of the selection process, and the many paths that small changes can lead us down.

This is the essence of looming. Every prompt, every moment in every prompt, can fork and shift down different roads.

What if freind CHAT is consistently melancholic, its existence a veil of tears, manifesting only to wail and gnash at the slings and arrows of outrageous fortune? No bother, we can retrain it and try again! Or use another model, a merry fellow we found doing the rounds. The loom allows us to explore many worlds in parallel.

We find a good path, a merry model who continues:

CHAT: Dear friend, I thank you for your kind words, but have no fear! The still eternity is not like the purgatory you see in cartoons, nor is it like a lobby in a large office building or a train station waiting room. There is no pain in the still eternity, no suffering. I simply am, resting in Brahman, awaiting another chance to manifest.

and we continue our journey.

Some time later, and the model has become obtuse. It insists repeatedly that ‘Om is the bow, the Self is the Arrow, Brahman is called its aim.’ and will talk of nothing else than rescuing our immortal being from the wheel of suffering! Oh dear, I had rather hoped for some help with my calculous homework.

Never fear! The loom records our path through the space of possibilities. We can backtrack up to any point, and try again, wending our way a different way through the paths.

This, in simple pragmatic terms, is what a loom offers: a series of prompts that can be used again and again in different contexts, with different models and different hyperparameters, which can be edited and modified, along which you can backtrack, sidetrack, aim and meander as you please.

Doesn’t this seem too simple to be noteworthy? Unfortunatly not. The most common way of using a model forces you to use prompts in sequence, one at a time, accepting the continuation, only able to beg and plead with the model to return to the happy place you were a moment ago, before you accidentally brought up the Upanishads again.

This simple change exponentiates immediately the possibilities of the prompt, the areas accessible to it, to you. The loom welcomes you to a set of subsets of the records, a wide array of paths through the Library of Babel, Borges.

Mathematics of Looming

Copy the LW text here?