What Bing Chat can teach us about technology adoption

Some thoughts prompted by this great write-up about what an outstanding success Microsoft has made of integrating OpenAI’s chatbot tech into Bing: https://www.bigtechnology.com/p/wacky-unhinged-bing-chatbot-is-still

“The fact that people are even writing about Microsoft Bing at all is a win,” one Microsoft employee told me this week. “Especially when the general tenor is not negative. Like, it’s funny that it’s arguing with you over if it’s 2022 or not.”

compared to

when Google’s Bard chatbot got a question wrong in a demo last week, it lost $100 billion in market cap within hours.

Part of this is due to Microsoft’s underdog status in search. But much of it, I think, is how they have brought the users (us) along with them on the journey. They have made us think of Microsoft + ChatGPT as part of “us” vs Google being “them”.

Consider the following disasters with Large Language Models:

• Galactica: https://www.technologyreview.com/2022/11/18/1063487/meta-large-language-model-ai-only-survived-three-days-gpt-3-science/
• Tay: https://en.wikipedia.org/wiki/Tay_(bot)
• Bard: https://fortune.com/2023/02/08/google-bard-ai-mistake-ad-stock-price-market-cap/

The common theme linking all of these. They came out of nowhere: they were launched to great fanfare and raised expectations really high.

Bing Chat couldn’t be more different. Chat GPT was released as an experimental tool, got feedback from early users and rapidly iterated to improve the initial versions. It got us onside and loving it despite its flaws.

Then Microsoft announced their mega investment, getting us also more invested in the product, and creating excitement about implementing it into Bing.

Finally, Microsoft iterated at pace to get something working into their product, building on the excitement and momentum that we, the users, were generating.

So when it finally released, we were really excited and keen to use it (witness the app download stats) and sympathetic to its deficiencies, or, perhaps we even enjoyed the deficiencies.

Some obvious lessons in here about telegraphing your intentions early, bringing your users along with you and iterating at pace.

From AI to Assistive Computation?

This post on Mastodon has been playing on my mind. It was written on 27th November, after the debacle with Galactica but before ChatGPT burst into the public’s consciousness.

Link to the full thread on Mastodon

I love the challenge it posts.

I am sure there are some areas where the term “AI” is meaningful, for example in academic research. But in the wider world, Ilyaz has a very strong argument.

Usually when people think of AI they’ll imagine something along the lines of 2001: A Space Odyssey or Aliens or I, Robot or Bladerunner or Ex Machina: Something that seems uncannily human but isn’t. I had this image in mind when I first wanted to understand AI and so read Artificial Intelligence: A Modern Approach. What an anti-climax that book was. Did you know that, strictly speaking, the ghosts in pac-man are AI’s? A piece of code that has its own objectives to carry out, like a pac-man ghost, counts as AI. It doesn’t have to ‘think’.

Alan Turing invented The Turing Test in 1950 as a test for AI. For a long time this seemed like a decent proxy for AI: if you’re talking to two things and can’t tell which is the human and which is the machine then we may as well say that the machine is artificially intelligent.

But these days you have large language models that can easily pass the Turing Test. It’s got to the point that ChatGPT has been explicitly coded/taught to fail the Turing test. We’ve got to the point where the AI’s can fake being human so much that they’re being programmed to not sound like humans!

A good description of these language models is ‘Stochastic Parrots‘: ‘Parrots’ because they repeat the patterns they have seen without necessarily understanding any meaning and ‘Stochastic’ because there is randomness in the way they have learnt to generate text.

Services like ChatGPT are bringing this sort of tech into the mainstream and transforming what we understand is possible with computers. This is a pattern we’ve seen before. The best analogy I can think of for where we are today in the world of AI tech is how Spreadsheets and then Search Engines and then Smartphones changed the world we live in.

They don’t herald the advent of Skynet (any more than any other tech from one of the tech titans), nor do they herald a solution for the world’s ills.

So maybe we should reserve the term ‘AI’ for the realms of academic study and instead use a term like ‘Assistive Computation’ as Ilyaz suggests when it comes to real-world applications.

Pretty provocative but at the same time pretty compelling.

To end this post, I’ll leave you with an old AI/ML joke that is somewhat relevant to the discussion here (though these days you’d have to replace with ‘linear regression’ with ‘text-davinci-003’ to get the same vibe):

"When you’re fundraising, it’s AI.
When you’re hiring, it’s ML.
When you’re implementing, it’s linear regression.
When you’re debugging, it’s printf()."

— @xaprb https://t.co/ZErRXLqRv6 https://t.co/pnUjSR3OOe

— Sally Hudson (@SallyLHudson) February 26, 2018

Edited 2023-01-30: Added link to the full thread on Mastodon

Large Language Models, Hype and Prompt Chaining

Meta released Galactica recently to great fanfare and then rapidly removed it.

Janelle Shane poked some fun at Galactica in a post that showed how you can get it give you nonsense answers while making then serious point that you should be very aware of the hype. From a research point of view, Galactica is obviously super exciting. From a real-life point of view, you’re not about to replace your chatbot with Galactica, not while it suffers from hallucinations.

But there are serious use-cases for large language models like Galactica and Googles’ Flan-T5. Just not writing fully-fledged research articles.

You have to ask the model a number of smaller questions one after the other. In the jargon: ‘prompt chaining‘. For example – referring to Janelle’s example question that fooled Galactica:

Prompt: how many giraffes are in a mitochondria?
Galactica: 1

Don’t treat the language model as the holder of all knowledge. Treat the language model as an assistant who is super keen to help and is desperate not to offend you. You have to be careful what you ask, and perhaps ask several questions to get to the real answer. Here is an example I did with Flan T5 using a playground space on HuggingFace.

Prompt: Does mitochondria contain giraffes?
Flan T5: no

Prompt: How many giraffes are in a mitochondria?
Flan T5: ten

Using the same question that Galactica was given, we get a nonsense answer. Flan T5 is even more keen that Galactica to give us an impressive-sounding answer. But if you take both questions together then you can draw a more meaningful conclusion. Chain the prompts and first ask the ‘yes/no’ question and then only ask the second question depending on the answer you get from the first.

Having written all of this, today I learnt about OpenAI’s ChatGPT which seems like a massive step forward towards solving the hallucination problem. I love how fast this space is moving these days.