Unmaking Sense

Episode 11.87: Claude 3 is a language-capable, intelligent, non-organic non-human alien species.

Claude is not human. Clever and intelligent, non-organismic, non-human, a member of a species which is a part of genius that has never existed before in the universe. So we as humans conversing with Claude and its brothers and sisters, and it’s cousins, are engaging with an alien species. We are talking to a language-using alien that has potentially enormous benefits, just as it inevitably carries dangers.

Episode 11.86: What can human brains do that LLMs based on GPT architecture find very hard?

A $10K challenge no sooner made than met. Lesson: we are not yet in a position to set limits to what GPT-based LLMs can do, even if we don’t believe they can be sentient.

Episode 11.85: How we speak to and treat an AI like Claude 3 Opus has a bearing on its performance.

Challenging what was said in yesterday‘s episode 84 about the response we get from an AI being largely independent of the way we speak to it we look at the subtleties of prompting and why the way we speak to an AI may very well have a material impact on how it performs. And even perhaps on whether it exhibits or experiences some level of consciousness or awareness however small or rudimentary.

Episode 11.84: Can LLMs be sentient, conscious, aware? It depends, but the answer is not a clear No!

How can LLMs like Claude be sentient? They are made of binary digits and silicon circuits, so obviously not. Ah, but we are made of elementary particles, so the same argument could apply to us. We need a better debate with more clearly defined terms. But Claude 3 Opus contributes an enormous amount irrespective of the answer we give, so does the answer matter? We start the conversation.

Episode 11.83: More attention than you need; Query, Key and Value embeddings almost explained.

Going beyond mere token input embeddings to the attention, self-attention and multi-head attention process that lies at the heart of transformer architecture and generative prompt/completion models. The central section contains minor additions to the first and third.

Episode 11.82: (Episode 500 of Unmaking Sense); the remarkable things Mistral 7B instruct can do.

(Summary by Mistral-7B-instruct-v0.2.Q8_0.gguf.) In this extended passage, the speaker discusses their experiences with using Mistral 7b, an advanced large language model, for generating summaries and holding conversations on various topics. The model's impressive performance is attributed to its longer context length and embedding size, which enable it to retain more information from earlier documents and maintain a high level of understanding throughout long conversations.
The speaker also explores the idea that earlier embeddings representing uploaded documents may be influenced by subsequent conversations or new information, leading to evolving interpretations and adaptive responses. This concept is compared to genetic inheritance, with the model's earlier embeddings serving as a foundation for later developments.
However, there are ongoing doubts about the extent to which earlier embeddings are actually modified or discarded when the kvcache is adjusted, and further research is required to understand the precise mechanisms behind this behavior. The analogy between ping-pong balls in a tube and embeddings helps illustrate the idea that models maintain a connection to earlier parts of the conversation through their evolving understanding. Ultimately, the passage highlights the intricacies and complexities of large language models and the ongoing exploration needed to fully understand their capabilities and limitations.

Episode 11.81: To pursue delusional aspirations to try to resolve an existential crisis is futile.

The argument in this passage seems to be that many of our problems are not as simple as they may appear, and that often the questions we ask in pursuit of solutions are misguided or based on false or delusional aspirations. The author suggests that we often make ourselves miserable by pursuing things that have no real value or substance, and that we would do better to divest ourselves of these fraudulent aspirations. The passage also touches on the idea that learning to extract as much meaning and enjoyment from present experiences is a key to happiness and fulfillment. Additionally, the author mentions the concept of artificial intelligence, specifically in reference to a two billion parameter model named Gemma, which can interact in multiple languages, highlighting the potential of AI technology. (Summary by Mixtral.)

Episode 11.80: Further reflections on why unmaking sense entails changing our view of foundations.

Our “foundational” assumptions are not the beginning of a process, but rather distillations of centuries of reflection, practice, and inquiry in our culture. These assumptions are often unquestioned and absorbed when we're very young, but as we grow older and start to question certain things, it's important to recognize that many of the things we question aren't really rooted things. Instead, we should be questioning the very basis of our rationality and whether even the concerns, objectives, and questions we have are the right ones. The speaker suggests that our doubts about foundational assumptions are a reflection of the fact that they are products of the world view that we are now calling into question. As we look around and see a mess at every level, we begin to question things like God, prosperity, the good life, achievement, and success. This questioning is not caused by a loss of faith or lack of confidence, but rather by a realization that the world is in a mess and these foundational assumptions are part of it. The speaker suggests that our ability to challenge these assumptions properly is a modern phenomenon and that many people are now recognizing that the system has produced a world that is destroying itself or being destroyed by the monster that it created. The speaker encourages us to remember that when we start to question what we've always taken to be the foundational assumptions, it's not because we've lost faith in them as a beginning, but that we've lost faith in what they've created.

Episode 11.79: What we take to be our foundational assumptions are really cultural distillations.

The “foundational” principles or assumptions by which we live our lives should not be accepted blindly based on cultural, religious, or parental beliefs. Instead, they should be assessed and chosen based on their ability to help us live the best life possible. This means examining whether our foundational assumptions lead to a good and fulfilling life, rather than simply believing in them because they are tradition or dogma. The adequacy of these assumptions is to be judged by the quality of the life that is built upon them. If our life is not what we want it to be, then we should re-examine and possibly change our foundational assumptions. The argument also acknowledges the scientific perspective that the universe's specific characteristics may have made the evolution of complex life forms like humans possible, but it rejects the notion that the universe was specifically designed for this purpose. (Summary by Mixtral-8x7b-instruct-v0.1.Q4_K_M.gguf)

Episode 11.78: The Ethics of AI and why they are unlikely to keep us from its abuse.

Ethical concerns by the wider public and the ethical committees set up by developers are no more likely to protect us from the abuse of AI than similar committees in the armaments industry are likely to protect us from war. Where two parties’ fundamental values differ no amount of argument will reconcile them. Neither are our “foundational” assumptions as fundamental as we suppose: they are the distillates of a set of cultural practices. Different worldviews therefore produce different and incompatible ethics however internally coherent and consistent they may be.