Unmaking Sense

Episode 11.12: The OpenAI shenanigans and why cleverness is not always accompanied by intelligence.

The topic of the moment.

Episode 11.11: Low-cost ways to adapt and specialise trained models. LoRA - Low Rank Adaptation.

LoRA stands for “Low-Rank Adaptations of LLMs”; it is a technique that involves making low-cost tweaks to the weights of trained models by adding small adjustments.

Episode 11.10: Some remarks about the hardware resources needed to process Large Language Models.

Large Language Models are pretty big. The smallest usable ones have around 7 billion parameters; chatGPT in its first (2021) version based on GPT3 had around 175 billion; many later ones are much larger. That limits what we can run on a local machine, but there are some workarounds.

Episode 11.09: Tokenisation or turning words and segments of words into numbers for processing.

Why tokenisation doesn’t work just with whole words.

Episode 11.08: Further aspects of “What Comes Next?”: probability, appropriateness, and culture.

How we - and so AIs - decide what comes next; why it isn’t inevitable, necessary or inescapable, but chosen. And what is chosen can be changed.

Episode 11.07: Project Development, teamwork, and respectful, benevolent, constructive dissent.

A digression into general human matters as they affect the deployment of teams in complex projects.

Episode 11.06: Fine tuning as a way of focusing an AI on specific details.

We revisit fine tuning from series eight. With a hint of LoRA.

Episode 11.05: Is “What Comes Next?” A fair representation of intelligence?

Objections to our central claim.

Episode 11.04: Maths of AI; how do we complete a sentence appropriately?

What comes next after a prompt, and what makes one completion more appropriate than another?

Episode 11.03: The maths of AI; overfitting and underfitting.

Teachers of all levels need to resolve a particular conundrum, which is how to prepare children well enough for whatever challenges they may face in the future including examinations, but not to make them think what they have been taught is everything that they could possibly need to know. And the other is not preparing them well enough for whatever challenges they may need to face The example, in the episode is of memorising the solutions to all the crossword puzzles that have ever been set without learning how to do crossword puzzles. That will not enable you to do the next crossword puzzle, which I presented with. The same is true of preparing for exams: there is merit in doing some past papers, so that you understand what is expected of you and the structure, but to do every past paper, as though that would somehow prepare you in the absence of a real understanding of what the contents of the paper was, is not going to help you Of course, the opposite is not to know enough to have insufficient skills and that is called underfitting, under-educating, under-learning, not having reached the required standard.