Brave Search Engine Summarizing Webpages in SERPs via @sejournal, @martinibuster

Brave announced a new AI-based rich result in its search engine results pages that feature an AI summarization of the content instead of the usual meta description.

In in some cases, the Brave Search Summarizer will show a summary derived from two websites together.

The Summarizer is currently showing for approximately 17% of search queries and that number is expected to go up.

Brave Search Summarizer Technology

The underlying technology of the Summarizer is a Large Language Model that was developed in-house.

It uses existing technology that was developed for displaying knowledge graphs and featured snippets.

Brave’s announcement explained:

“The Summarizer is not powered by ChatGPT or its backend systems; it is instead composed of three different LLMs 2 trained on different tasks:

The first one is QA (question answering): this model is used to try to extract a concrete answer, if any, from text snippets.

Brave has been using LLMs for a while to improve search relevance, and this is an extension of what Brave Search already had in place to power its knowledge graph and featured snippets features.

The difference lies in the number and length of text snippets analyzed.

After the QA extraction phase, result candidates are further classified with an ensemble of zero-shot classifiers on a wide variety of criteria (hate-speech, vulgar writing, spam, etc).

The final set of candidate text is ultimately processed by the summarizer/paraphrasing model, which tries to rewrite the input so that repetition is removed and that language is kept uniform to improve readability. “

Brave Summarizer Makes Mistakes

Brave is up front that the system is not yet perfect and that it could make mistakes by combining unrelated snippets.

The announcement stated:

“Besides scalability, a tremendous effort has been put into ensuring the quality of the generated summaries.

However, as the model is still in its early phase of development, there is the possibility of producing “hallucinations,” which mix unrelated snippets into a single result.

There is also the possibility of some false or offensive text, but our aim is to continue working on improving the models as our users’ feedback starts pouring in.”

This is something that can be seen in a search result for “Who is Danny Sullivan?”

Danny Sullivan is a highly common name, which may be the reason why the Braves Summarizer combines details of multiple people into one summarization.

The summary humorously credits Danny Sullivan with having won the Indianapolis 500 automobile race and having worked as a chicken rancher.

This is the Brave Search summarization:

“Danny Sullivan is an American technologist, journalist, and entrepreneur best known for founding Search Engine Watch in 1997 and launching Search Engine Strategies, one of the earliest search marketing trade shows.

He is also a former racing driver who earned 17 wins in the CART Indy Car World Series including the 1985 Indianapolis 500 and won the 1988 CART Championship.

Additionally, he attended Kentucky Military Institute before dropping out to work as a janitor, New York cab driver, waiter, lumberjack or chicken rancher.”

This is a screenshot of the erroneous Summarizer result:

What’s notable about the summary in the screenshot above is that the data came from two websites.

The Summarizer links to both source websites at the bottom, apparently using images extracted from favicons.

Brave Search Summarizer Links to Sources

The Summarizer search snippets provides links to the source websites.

The links are marked in red in the following example:

The link to Cars.USNews.com in the image above appears faint, with a gray circle to the left of the displayed domain.

The gray circle is where an image icon from the website is supposed to be.

Below is a comparison of links to three source websites.

What’s notable is that the link to the Cars.USNews.com webpage doesn’t have an icon, which causes the link to lose some visibility:

The Cars.USNews.com webpage does does not contain a favicon. A favicon is a logo image that displays in a browser tab and is also used in other ways.

The webpages for Crutchfield and BestBuy.com both contain a favicon.

The USNews.com site does not have a favicon on that specific webpage.

They apparently overlooked linking to a proper favicon on that webpage (although it’s contained on other pages on the same subdomain).

New Way to Display Search Results

Bing is taking a different approach for integrating AI into search.

It uses a system called an Orchestrator that sites in between their chat AI and the regular search index.

Brave is taking a more conservative approach because they don’t believe that the chat approach is viable at this time.

They explained why they went with a summarizer method:

“Although the industry is generating much hype around AI, at Brave we are not yet convinced that LLMs can replace search as we know it.

However, if used properly, these new models can help the user navigate results, which is the approach we follow with the Summarizer.

Chat-like interfaces and oracle-based search remain unproven and, as of today, we remain skeptical that they’ll be useful for all search tasks.”

How Do You Feel About the Summarizer?

Many publishers and SEOs are not happy when Google replaces their meta descriptions with content extracted from the webpage.

What the Brave Summarizer does goes far beyond what Google’s snippet rewriting by creating an entirely new summary or mixing and matching the content from two websites.

Looked at one way, the Brave Summarizer is creating a context for the link to the source webpage, which may result in better traffic.

Then there’s the situation where two websites are credited for one summarization. Who’s going to get the click?

Read the Brave Search announcement:

Brave Search introduces the Summarizer, an AI tool for synthesized, relevant results