Unmaking Sense

Episode 11.13: Why the history of religions is relevant to the emergence of AGI.

Human beings have been fascinated by gods and their God for most of their history. We like to think that someone is in control, that someone knows what is going on, that someone understands everything, and that we are part of some great cosmic plan. We like in particular to feel that someone is watching our back, that there is someone watching over us. It’s almost certainly not true, any of it, but we cling ambivalently to the notion anyway. If, as I believe, nobody knows what is going on most of the time, this is one reason why we like to hold onto belief in a deity, for deities are those who do. Now it is possible that something like an artificial general intelligence will watch over us all the time and know what is going on all the time. Now there may at last be a God.

Episode 11.12: The OpenAI shenanigans and why cleverness is not always accompanied by intelligence.

The topic of the moment.

Episode 11.11: Low-cost ways to adapt and specialise trained models. LoRA - Low Rank Adaptation.

LoRA stands for “Low-Rank Adaptations of LLMs”; it is a technique that involves making low-cost tweaks to the weights of trained models by adding small adjustments.

Episode 11.10: Some remarks about the hardware resources needed to process Large Language Models.

Large Language Models are pretty big. The smallest usable ones have around 7 billion parameters; chatGPT in its first (2021) version based on GPT3 had around 175 billion; many later ones are much larger. That limits what we can run on a local machine, but there are some workarounds.

Episode 11.09: Tokenisation or turning words and segments of words into numbers for processing.

Why tokenisation doesn’t work just with whole words.

Episode 11.08: Further aspects of “What Comes Next?”: probability, appropriateness, and culture.

How we - and so AIs - decide what comes next; why it isn’t inevitable, necessary or inescapable, but chosen. And what is chosen can be changed.

Episode 11.07: Project Development, teamwork, and respectful, benevolent, constructive dissent.

A digression into general human matters as they affect the deployment of teams in complex projects.

Episode 11.06: Fine tuning as a way of focusing an AI on specific details.

We revisit fine tuning from series eight. With a hint of LoRA.

Episode 11.05: Is “What Comes Next?” A fair representation of intelligence?

Objections to our central claim.

Episode 11.04: Maths of AI; how do we complete a sentence appropriately?

What comes next after a prompt, and what makes one completion more appropriate than another?