The AI chatbot wars
The emergence of ChatGPT and the intense chatbot competition between OpenAI / Microsoft vs. Google rekindled my curiosity about search engines. Just this week, Forbes announced that the “ChatGPT Wars Erupt as Google and Microsoft Race to Market.” A few days later, The Wall Street Journal reported that the “AI war heats up” as Google announced that it will launch a ChatGPT competitor within weeks. And today, Microsoft integrated OpenAI ChatGPT into its search engines Bing and Edge, with the Bing homepage now enabling users to follow up on their queries with a ChatGPT. Google immediately released a promotional video advertizing Bard, its own AI chatbot, only to display it giving the wrong answer to the question of whom took the first pictures of a planet outside the Earth’s solar system. This sent shares of Google’s parent company Alphabet plummeting, wiping out $100 billion of market value in the process. To be continued.
With the widespread popularity of ChatGPT and its imminent integration into Microsoft’s search engines, Google freaked out. The trillion dollar company issued a code red and brought back its co-founders Larry Page and Sergey Brin to fight back. There is so much at play. We make about 100,000 Google searches per second that add up to more than 8.5 billion searches a day. The search engine industry takes $120 billion in revenues and Google’s hold on this industry is remarkable with a 93% market share.
Will chatbot powered search engines hallucinate as badly as ChatGPT?
These chatbot “wars” will be fascinating to follow. But users, notably academics and scientists, have reasons to be concerned. Despite all of its amazing prowesses, ChatGPT is known to hallucinate, namely to provide articulate and confident answers that peddle nonsense. Will AI-powered search engines hallucinate as badly as ChatGPT?
Chatbots produce synthetic answers gathered from multiple sources. So far, such answers aren’t supported by links and references as expected in academic writing or even in a typical Google or Bing search page. Synthetic answers are prone to make stuff up as you will see below.
What are Featured Snippets?
A precursor to search engine chatbots are Feature Snippets, the excerpts of texts highlighted at the top of Google or Bing search pages. Feature Snippets, as Google calls them, can be text quotes, lists or tables, that are extracted from web pages that are indexed by the search engine. They provide a snapshot answer to the search query.
Below is an example Featured Snippet. In contrast to chatbots like ChatGPT, the answer is associated with a reference web page (hopefully a reliable source) and the link to the web page is provided enabling users to quickly follow up, check the source and gather more information.
Featured Snippets are known as “Position #0” because they appear above the #1 spot on a Google search page. In their support pages, Google explains how Featured Snippets are chosen from web search listings:
“Google’s automated systems determine whether a page would make a good featured snippet to highlight for a specific search request. Your feedback helps us improve our search algorithms and the quality of your search results.”
Feature Snippets from this blog generally feature accurate answers
I was pleasantly surprised that excerpts of a number of posts from this blog got picked up as Featured Snippets. I didn’t make any particular effort to enable this and I only recently became familiar with the Snippets. Good to know that you don’t need hire an SEO (Search Engine Optimization) expert to be featured.
Parenthetically, SEO is a fast growing industry of its own with a market size of $50 billion. Given the value of having your company or product pop up high on a Google search page, an entire industry of experts and consultants guide web designers in optimizing their pages to show up high on search pages. Feature Snippets can also be coveted although some argue that having the answer right on the search page discourages clicking in the links and racking up valuable income through page visits. In any case, the SEO industry is about to be shaken up to its core given the disruption that chatbots will bring to our internet searching habits.
Let’s review 5 posts from this blog that got picked up by Google or Bing as Featured Snippets. They tend to provide accurate answers to the query along with a link to original source. That’s good news compared to the tendency of ChatGPT to make stuff up leaving me generally baffled as I can’t trace back the statements to a specific source.
These are the 5 articles that have Featured Snippets:
This popular article on how to deal with negative results in the lab was titled as a question, and Google did dig out the answer even though it was buried deep in the text.
Google’s main competitor, Microsoft’s own search engine Bing, also picked up the exact same quote. Bing has its own version of the Featured Snippets. This was one of Google’s Featured Snippet from this blog that was also highlighted by Bing.
As you can see in the screenshots, Bing has already integrated a chatbot feature to the Snippet. The “New Bing” include an “AI-powered answer engine.” As far as I can tell, the new Bing chat mode isn’t widely available yet, but if you’re anxious to use it you can join a waiting list. And you can find out more about the new Bing here.
GOHREP is a framework developed to help researchers design and plan their experiments. The acronym stands for Goal, Hypothesis, Rationale, Experimental Plan. If you ask the search engines to explain GOHREP, both Google and Bing will display the correct answer.
Tips for writing paper Twitter threads?
This post discusses and lists tips for wirting Twitter threads of academic papers. Google displays it as Featured Snippets, but if you ask for a list of tips it generates a mash-up of the bulleted list.
There are several versions of this article. Google displays a list from the oldest version from my Tumblr blog. When you ask for a list of tips, Google will display the list from the version of the article published in the Royal Society publication The Biologist “11 tips for choosing the right PhD.” It’s all accurately reflecting the various articles though.
How about science posts? These do get picked up too. When you ask Google to explain NLR networks (these are immune receptor proteins), it picks up a medium post on the topic. Bing, on the other hand, prefers to top the page with the article by my collaborators Jiorgos Kourelis and Hiroaki Adachi. Damn’ 😜😜
How do Featured Snippets compare to ChatGPT?
I queried Google and ChatGPT with a highly specialized science question related to a protein sequence motif we call MADA that is present at the N-terminus of NLR immune receptors. This is what happened.
Google Featured Snippet gave an accurate answer extracted from an original scientific article gathered from PubMed, the highly curated biological literature database. In contrast, this is what ChatGPT produced:
ChatGPT hallucinated. As far as I can tell, this answer is gibberish. Worse, I can’t trace the false statements back to a source. A Google search for “Multiply And Diversely Aligned” yielded absolutely nothing informative. I still can’t tell if this is a made up acronym. It probably is.
In an earlier chat with a similar query, ChatGPT came up with more nonsense that could easily confuse a naive reader. It wrote:
“The MADA motif is the sequence “MVDA”, which appears in the first four amino acids of the protein. The MADA motif is a signature sequence found in a group of plant effector proteins called “molecules against DEfence”. These proteins are secreted by plant pathogens and are thought to play a role in overcoming the plant’s defense mechanisms during infection.”
Will AI-powered search engines be more accurate than ChatGPT?
In the article “Talking About Large Language Models,” Murray Shanahan delineates the limits of chatbots trained using Large Language Models and explores the philosophical implications of mimicking human language. John Naughton, Professor of the public understanding of technology, explains this further in The Guardian:
“…if you give the model a prompt such as “The first person to walk on the moon was … ” and it responds with “Neil Armstrong”, that’s not because the model knows anything about the moon or the Apollo mission but because we are actually asking it the following question: “Given the statistical distribution of words in the vast public corpus of [English] text, what words are most likely to follow the sequence ‘The first person to walk on the moon was’? A good reply to this question is ‘Neil Armstrong’.””
ChatGPT turned out to be a remarkable virual assistant that performs incredibly well at many tasks, but correctly answering knowledge queries isn’t one of them. We will soon find out whether the chatbot apps integrated into Bing and Google perform better at this task. Meanwhile, I would stick to Featured Snippets.
This article is available on a CC-BY license via Zenodo. Cite as: Kamoun, S. (2023). Are Google and Bing Featured Snippets more accurate than chatbots?. https://doi.org/10.5281/zenodo.7622873
