In a recent blog post announcement, Bing introduced Speller100, a large-scale spelling correction model with over 100 languages.
Spelling errors are common when typing queries in search engine boxes.
According to Microsoft, 15 percent of queries submitted on Bing have misspellings. As a result, the search engine could match the wrong set of documents and show incorrect results.
Misspellings can show low-quality result pages for search engine users. So it’s not surprising that spelling correction is the first component in the Bing Search stack.
There’s just one problem.
The spelling correction technology only supported about two dozen languages. As such, individuals who issued queries in many more languages often end up with inferior results.
Executives at Bing wrote in a statement:
“In order to make Bing more inclusive, we set out to expand our current spelling correction service to 100-plus languages, setting the same high bar for quality that we set for the original two dozen languages.”
In an A/B test of the new spelling correction model, Bing noted the following results:
- The number of times users had to rewrite queries reduced by 5 percent.
- Pages with no results reduced by up to 30 percent.
- The number of times users clicked on spelling suggestions increased from single digits to 67 percent.
Finally, the number of times users clicked on any item on the page went from single digits to 70 percent.
So, how did Bing accomplish this feat?
Using Speller100 to Improve Spelling Correction in Bing Search Results
In the last few years, natural language processing has improved the way machines handle various tasks.
However, the new spelling correction model doesn’t belong in this category. Instead, Bing credited zero-shot learning as the vital AI advancement that made Speller100 possible.
Zero-shot learning allows an artificial intelligence model to learn and correct spelling accurately.
What’s more, it doesn’t require any additional language-specific labeled training data to function. That’s in contrast to the traditional spelling correction solution that relies heavily on training data.
Bing also trained the Speller100 model using what it calls “noise functions.” It refers to character-level mutations that mimic spelling errors.
The document from Microsoft reads:
“The use of a noise function significantly reduced our demand on human-labeled annotations, which are often required in machine learning. This is quite useful for languages for which we have little or no training data.”
Although Speller100 has only launched in Bing, this could be the first step. According to reports, Microsoft could implement the new model in more products soon.
Comments (0)
Least Recent