Unmaking Sense

Episode 8.61: On the unreasonable efficiency of chatGPT and some of the implications for us.

I have my first experience of the chapel writing, a complete python program that works straight away to download the details of this podcast series from the Anchor website.

Episode 8.60: Results of an embedding experiment.

We pick a text and compare it with four others texts using embedding.

Episode 8.59: Using embedding to compare similar and dissimilar texts and the results.

From episode 58 of series 8 an embedding is a way of comparing so I thought that I would do a little bit of experimentation on texts similar and dissimilar from different texts to fraudulent texts that are manufactured and see what the results were, and the results were exactly what you would hope.

Episode 8.58: Embeddings and Encodings: differences, similarities and uses in chatGPT and its cousins.

The ways in which we encode the natural language text that we use in order to communicate with a chatbot are fundamental to its effectiveness and its efficiency. Unfortunately, terminology like “embedding” and “encoding” tends to get thrown around with a certain amount of abandon as though the two are the same, when they are not. We start exploring the differences and seeing under what circumstances each of these techniques is used.

Episode 8.57: ChatJCP is the 300th episode of Unmaking Sense. We celebrate JCP’s immortality.

This is the 300th episode of a Unmaking Sense: Living the Present without Mortgaging the Future. Thanks to all my listeners who have stayed with me on this long journey. We celebrate by considering the possibility that by customising a chat bot like chatGPT with our personal characteristics as a final fine-tuned wrapper, we might achieve a kind of immortality.

Episode 8.56: Fine-tuning GPT-3 to suit your particular interests, training it with customised data.

This episode is in two parts: in the first we discuss the nature of fine-tuning, how it’s done and why it’s necessary; in the second part we discuss how at we as end-users can use our own data to customise the whole system to suit and serve better the particular interests that we have in our amateur or professional lives. We briefly discuss the costs of this process and how to do it.

Episode 8.55: The personalities of chatbots are embodiments of their training, just as are ours.

In episode 54, we talked about propensities in human beings and in chatbots, and I suggested that the chatbots will embody to some extent certain characteristics that are reflected in the way they’ve been trained, on the choices that have been made by those training them. Extend that a little, and you can start to see how different chat bots from different parts of the world trained by different groups of people are likely to have different characteristics and so different personalities. So human beings, when they come to decide which chatbots to use, will be affected by those factors in just the same way that they are when they choose their own human partners, friends and collaborators. But the fact that we can wrap these chatbots with fine-tuning means that we can also tailor them to our own personal or corporate or collective interests in order to try to persuade them to give better responses to the things that we are interested in. But where does that start and where does it finish, and how safe are we from the misappropriation of chatbot technology by those who would wrap them in skins and give them personalities of a kind that we would deplore and justifiably find quite alarming, dangerous and frightening? Fine tuning will be in 8.56.

Episode 8.54: Can a chatbot have a personality?

How the way a chatbot is trained imputes opinions and values that mean that it will take on the characteristics of its culture through its training.

Episode 8.53: What matters is whether our AI produces useful output, not whether it models reality.

Our artificial intelligence models should not be thought of as if they are trying to reproduce whatever it was that produced the phenomena that they are modelling. So if they produce something that looks vaguely like Shakespeare, that does not mean that they have recreated Shakespeare’s brain. If they produce reliable predictions of earthquakes, it doesn’t mean that they have reproduced a working theory that is adequate to the seismology of the Earth, all that matters is that their predictions are reliable or interesting or entertaining. My experience with Andrew Karpathy’s GPT model from YouTube, “Let’s build GPT: from scratch, in code, spelled out.”

Episode 8.52: Attention is all you need: how tokens find matches that will come next.

The “Attention is All You Need” paper lies behind Andrew Karpathy’s excellent YouTube video “Let’s build GPT: from scratch, in code, spelled out”. We discuss some implications.