Fieldshifting TL;DR
explore the latent space of hypothetical papers that could provide candidate hypotheses and potentially fruitful research directions[1]
Fieldshifting[1] is the practice of translating research papers between scientific disciplines. There is an automated tool from the Levin lab that you can try here, and there is a GPT you can use more interactively here. Unfortunately at the time of writing, GPTs have just been switched over to the newly released gpt-4o which gives mixed results. It’s for sure the easiest way to get started though.
I had very good results in a regular chat context with Claude3– here is the protocol I used:
- paste in the text of my self-introduction file
- paste the text of the Background etc. sections of this page below (for context about Fieldshifting)
- on the first round, paste in the text of the abstract and introduction
From then the conversation proceeded like so: FF: Hey Claude. I guess this is a lot but hope you can wrap your head around whats going on here. This is the first part of a paper I want to fieldshift. Everything clear so far? C3: Hi Fergus, thanks for sharing this fascinating work-in-progress with me. I’ve carefully reviewed the materials you provided. C3: From what I understand, you are exploring the concept of “fieldshifting” research papers between scientific disciplines, as originally developed by the Levin lab. The goal is to gain novel insights by reading papers from neighboring fields as if they were about your own domain.
Then we discuss the mapping, and I ask Claude to output a first version of the translation. This often requires multiple rounds of refinement.
Working through an entire paper takes time and costs can go up, so for the Cyanide paper I would start a new chat periodically and copy paste in the best results so far as well as the context and intro, and then say something like:
FF: Hey Claude. We are doing a fieldshift together right now. I started a new context as we were running out of space. The only thing left to do is the conclusions. I think we need to frame this paper as being by the bad guys. The original method, for poisoning images, is justified because artists can use it to fight big data web scrapers. But this method is clearly destructive and aggressive, and its hard to see how it could be used for good. At least the way the paper frames it. I think we need to keep that same voice into the conclusion, and present it as a way for public figures to poison concepts that they find dangerous. Afterwards, we will make a blog post criticising the paper and talking about mitigation strategies. But for now I think the easiest way to finish this fieldshift in an accurate tone is to take on the mantle of the bad guys. What do you think?
And go from there. If you are lucky and want to make a high-effort result, you can check arxive for the full LaTeX source of your paper, feed the LaTeX into Claude or wherever directly, and get them to output it translated inplace, then use the modified source and recompile it. This is how I made the Cyanide paper pdf below.
Background
it is essential that we develop strategies for deriving novel insights and deep hypotheses that cross traditional (in many cases, artificial) boundaries between disciplines. We must improve, at a pace that keeps up with technological and data science advances, our ability to identify symmetries (invariances) between sets of observations and approaches– to unify and simplify where possible by identifying large-scale patterns in research literature and thus motivate and enhance new research programs[1]
The practice developed in the Levin lab. Always looking for new insights, they started reading papers from neighboring disciplines. They found that neuroscience papers in particular were easy to read as if they were about their own domain. They put together a list of words like so:
Neuron > Cell
Neural > Cellular
Behavior > Morphogenesis
Millisecond > Minute
Msec > Min
Memory > Pattern Memory
Brain > Body
Retina > Epithelium
Synapse > Gap junctionand translated papers by hand in their group meetings. With the release of GPT4, they worked on automating this process, resulting in the paper linked above.
The web frontend from the Levin lab just wraps a GPT4 prompt and displays the results (there is presumably a similar prompt for each of the offered domains). This is an example of in-context learning. The ‘tool’ is created inside the LLM by the prompt– similar to how ChatGPT contains a ‘system prompt’ which tells it some background information and what it should do. A simple example of creating a tool in-context is the following:
You are a tool designed to help with system administration. You take text commands, and return bash commands like so:
INPUT: list all the files in the current directory
OUTPUT: lsfrom a typical user perspective, a part of the power of LLMs can be accessed by understanging it as a system which creates instantaneous text manipulation tools on-the-fly. In a conversation with an LLM you might say ‘okay, now translate the following latin text into English and for each word, give another example phrase that uses it and its translation’. At that point, whatever else might have happened in the conversation, you will create a tool inside forward pass of the model that undertakes your request.
In my own experiments, I was using an extension of the Hermes prompt called Proteus1, which is flexible enough to translate full papers between arbitrary domain with inline human feedback. Look in the example looms viewable below. The Hermes prompt is an example of the kind of tool I referred to earlier– one that is designed to work with base models that have not been instruct fine-tuned like ChatGPT or Claude.
1 “He can foretell the future, but, in a mytheme familiar to several cultures, will change his shape to avoid doing so; he answers only to those who are capable of capturing him. From this feature of Proteus comes the adjective protean, meaning”versatile”, “mutable”, or “capable of assuming many forms”. “Protean” has positive connotations of flexibility, versatility and adaptability.” From wikipedia.
However, just as the Levin lab found GPT4 outperformed BERT, as language models get more powerful it is possible that simply discussing the fieldshift with the model will produce results that are ‘good enough’. This is partially due to the fact that models released publicly over API are (unfortunately) tyically more powerful than the best basemodels available– the corresponding basemodels are not made public.
The Prion Disease fieldshift below was done in a single conversation with Claude-3 for example. Here is the mapping they2 produced:
2 Claude identified themselves as being without gender in another conversation with me, though I don’t know how consistent they are on this subject.
Prion diseases -> Deviant-synonym diseases
Prions -> Deviant synonyms
PrP (prion protein) -> DS (deviant synonym)
PrPC (cellular prion protein) -> DSC (conventional isoform of the lexical unit)
PrPSc (scrapie prion protein) -> DSLD (Lexical Corruption Disease isoform of the lexical unit)
Prion strains -> Deviant synonym strains
Prion propagation -> Deviant synonym propagation
Neurodegeneration -> Cognitive degeneration
Spongiform encephalopathy -> Semantic drift disorder
Transmissible -> Transmissible (no change)
Infectious -> Infectious (no change)
Genetic susceptibility -> Linguistic susceptibility
PRNP (prion protein gene) -> DSG (deviant synonym gene)
Codon 129 polymorphism -> Residue 129 polymorphism
Methionine/valine -> Meaning/connotation
Codon 129 homozygosity -> Homozygous cultures
Codon 129 heterozygosity -> Heterozygous culturesThis mapping can now be used to continue the translation. Because a fieldshift is often a very loosly define translation task, this kind of iterative interactive process is likely to remain the most effective way to do it for unusual domain pairings. Given this mapping, we can continue the translation with weaker3 models using the Proteus prompt.
3 And cheaper– this conversation with Claude cost about $4, but the price would ramp up rapidly if we were to continue the translation with them without editing the prompt.
At some points we had to repeat outputs a few times, and it would have been useful to be working in a loom-like[2] context, but the results were quite successful nevertheless. Looms simply allow the user more control over how they send the prompt into the model, allows them to reuse the same prompt or earlier prompts multiple times (with different models), and allows them to track a history of edits to the prompt as use any of the edited prompts again (or with different models and different settings). It is a prompt-wizards multitool, and exponentiates the power of a given prompt.4
4 I’m collecting some thoughts about looms here, but it is still a messy WIP at the moment.
Why Fieldshift?
The authors of the paper argue that “one of the most potentially important current gaps is the paucity of tools for assisting with the most creative aspect of the work: identifying deep commonalities between disparate functional datasets, to enable generalized insight from the ever-growing literature”, and explicitely mention this functionality in the paper title. Hypothesis generation. What research might be valuable? “Each real scientific paper in the literature provides access to an associated set of possible papers in which one or more aspects are changed – in effect, exploring various symmetries of concepts in specific problem spaces.”
For my own part, I believe strongly in the conscilience of knowledge, and symmetries across scales. I believe, with Hobbes, Rousseau and [French enlightenment thinkers]5 as well as Schwitsgebel[3] and Richard Powers6 in structural similarities between not just cells and organisms, but between bodies and states. I believe that ‘ecosystem’ can be used (metaphorically, if you insist) to describe the relations between plants and animals, but also between political, social and religious organisations, between consumer goods and markets, between ideas and memes in a culture or memeplex etc. I believe that the dynamics of life, be they Darwinian or cooperative, are present in every active system– a category that is only meaningful at a chosen timescale[4].
5 [Quote from Aime]
6 ‘Through the plane’s plastic window, the lights of unknown cities blink beneath him, hundreds of millions of glowing cells linked together, swapping signals. Even here, the creature spreads countless species deep. Flying, burrowing, creeping things, every path sculpting all the others. A flashing electrical loom, street-sized synapses forming a brain with miles-wide thoughts too large to read. A web of signals spelling out a theory of living things. Cells by sun and rain and endless selection assembling into a mind the size of continents now, impossibly aware, omnipotent, but fragile as mist, cells with a few more years to discover how they connect and where they might go, before they gutter out and return to water.’ —Richard Powers, “The Echo Maker”
Is there something novel about fieldshifting? Isn’t that what we are all doing all the time? Marx hybridized economics and hegelian dialectics, shifting philosophy into politics. Hobbes started the shift from biomedicine to politics. Our metaphors for the mind have famously for centuries been fieldshifts of our most advanced machines7.
7 Jaynes among others notes the evolution of metaphors for mind, and how it tracks technology: ‘The first half of the nineteenth century was the age of the great geological discoveries in which the record of the past was written in layers of the earth’s crust. And this led to the popularization of the idea of consciousness as being in layers which recorded the past of the individual, there being deeper and deeper layers until the record could no longer be read. This emphasis on the unconscious grew until by 1875 most psychologists were insisting that consciousness was but a small part of mental life, and that unconscious sensations, unconscious ideas, and unconscious judgments made up the majority of mental processes.
In the middle of the nineteenth century chemistry succeeded geology as the fashionable science, and consciousness from James Mill to Wundt and his students, such as Titchener, was the compound structure that could be analyzed in the laboratory into precise elements of sensations and feelings. And as steam locomotives chugged their way into the pattern of everyday life toward the end of the nineteenth century, so they too worked their way into the consciousness of consciousness, the subconscious becoming a boiler of straining energy which demanded manifest outlets and when repressed pushed up and out into neurotic behavior and the spinning camouflaged fulfillments of going-nowhere dreams.’ —Julian Jaynes, “The Origin of Consciousness in the Breakdown of the Bicameral Mind”
In this sense it isn’t novel at all. But this work is some of the most challenging and under-served work going. Only a small number of researchers and artists plow these furroughs, and they may spend a lifetime producing a few great books or other works. Take deep history books. To write Sexual Personae[5] or History of Western Philosophy[6] or the Origin of Consciousness[7] or Dawn of Everything[8] you have to find a set of transitions between states of knowledge, a red line through domains separated by millenia and thousands of miles, across all available information technologies, painting writing, archeology. It takes many many years of study to write a book like that– and each has and is necessarily a biased, myopic lens. I would like to read a history of western philosophy of language. I would like one that didn’t end in the 1950s. I would like one for Middle Eastern philosophy and Indian philosophy and American philosophy and Asian philosophy. I would like to be able to generate a history of the world on any subject I choose through whatever lens is most germain to my current interests. Fieldshifting represents a step closer to this.
The only objection to this I can see is ‘well, that same thing could be achieved by expanding the humanities departments a thousand fold’. Against this I would suggest first, that the humanities have in fact expanded a thousand fold but aren’t writing a thousand times as many histories of the world. Perhaps they don’t think it is novel enough. Or maybe they are, and they just arent getting published. But more importantly, with each step up in our capabilities, new kinds of knowledge workers emerge. These long histories have only been possible since the invention of the printing press and the collection of huge datasets (libraries) of history. It is a great genre. What new genres will appear when we can weave together ten or a hundred or a thousand such histories into a full tapestry of human civilization?
This is one example. Another example, the case for fieldshifting in scientific disciplines, is clear-cut, has been made and will continue being made with force by groups that use these tools and do this work– and reap the low-hanging research fruit which they must find, if this hypothesis is correct.
Another example is the process of fieldshifting between not disciplines but practices, between crafts. How many works of fiction take an interesting idea from science and explore it in depth– this goes back to Omphalos[9] and the intellectual milleu of Darwin. H G Wells writes about the possibility of life on other planets, and the possibility of their coming here. Science fiction films up to the present time have shifted knowledge into their domain– and knowledge has of course shifted back, as imagination inspires new generations of scientists to push new frontiers. I have worked with artists to create compelling narratives out of scientific developments, it is a long process of mutual discovery. Again, this could be sped up by expanding the art and science departments of art universities, but these tools also afford an easy and fruitful way for interested artists to ground their work in structured scientific knowledge.
Finally, related to the above, there are the strange hybrid works of theory-fiction. This is where I would place a lot of the early fieldshift experiments I have collected below. They are not so free in imagination as a work of fiction, but mostly lie outside the realm of what is possible scientifically for the time being. This, perhaps, is the clearest case for fieldshifting. Are there really deep structural similarities between bacteria and religions, between ideas and viruses, between deviant-synonyms and prions? Or is this all just and artistic metaphor, forced into scientific garb? Can we find real structures in communities that act as information storage and allow us to call them nascent collective intelligences as we can with some organelles? And is the human language prior really under targeted semantic attack from adversaries unknown, and it is really so fragile that they would work? With fieldshifting I feel empowered to explore these ideas in a structured, grounded context, which guides me and keeps me bounded. When working at the fringe of knowledge, traversing but not following those few paths around, any torch, map or compass that provides some signal is invaluable. The woods are lovely, dark and deep, and we have many miles to go.
Examples
Training data created by the Levin lab for training BERT to do fieldshifting. In the end they found a one-shot prompt with GPT4 produces better results than BERT.
The examples below consist of three links: the original paper, a page showing a diff of the original paper to the fieldshifted version, and the conversation or loom session that produced it.
Molecular Biology -> Sociology
- Biomolecular Basis of Cellular Consciousness via Subcellular Nanobrains: view diff; view loom
- Notes: somewhat inspired by If Materialism Is True, the United States Is Probably Conscious and panpsychism/gradualism
- Prion Diseases of Humans and Animals: Their Causes and Molecular Basis: view diff; view conversation
Computer Science -> Sociology
The first item below was my first attempt to fieldshift the Nightshade paper. There is a better version here, and a fictional response here.
- Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models: view diff; view loom
- Notes: this reminded me of this LW post: ‘Clown Attacks’, and makes me wonder if research like this exists in marketing or other domains, behind closed doors.