The Inside Perspective on Modern Search: An Interview with Fabrice Canel of Bing

At Botify Connect 2024, Botify co-founder and CEO Adrien Menard sat down for an interview with Fabrice Canel, Principal Product Manager at Microsoft Bing and a well-known figure in the world of organic search. Bing has emerged as a major player in the AI search industry, and Fabrice shared his thoughts on the impact of AI and large language models (LLMs) on search experiences, plus the practical steps SEO teams can take to not only adjust to, but embrace AI as part of the search process.

Watch the full interview at Botify Connect 2024 here!

Can you share something about Bing that people might not know?

Fabrice: People often don't realize that Bing is bigger than it seems. Bing powers AI and search solutions across various platforms, including Yahoo, Meta, and Amazon Kindle. We also power dozens of search engines globally. 

Additionally, Bing's clicks often monetize better than other search engines, including competitors, which makes optimizing for Bing valuable. However, our focus is on best practices that benefit the entire search industry.

What's your perspective on what's happening in search right now, and over the past year? 

Fabrice: AI, especially LLMs, is a technology that's been evolving for a while, but now it’s really smart. Bing has always focused on improving technology to compete with Google — with Google having many users and Bing much fewer, the only way you can catch up and be relevant is by playing with technology, with deep neural nets, with AI. And LLMs represent an important layer in that evolution. These AI-driven technologies help push us to improve the quality of search results.  

Internally, we have even more advanced tools that we’re beta testing to help us refine search quality, and this is fundamentally helping us provide better clicks. Not only in Bing Copilot, but for all our partners leveraging this technology today. 

These are big changes for the industry. The entire user experience is changing. What’s your reaction? 

Fabrice: For the last 20 years, search results were mainly 10 blue links. But in the last two years, we’ve seen the rise of AI-driven experiences, like Bing Chat, renamed Copilot, which replaces traditional results with more interactive AI-driven answers. People were surprised by these experiences and didn’t understand them well, so we’re gathering data to fine-tune when to display AI-generated content versus traditional search results. For instance, with clear navigational queries (like searching for "YouTube"), 99% of clicks go to YouTube.com. However, if you search for something like “I need to find a restaurant”, or “most famous restaurant in Austin that serves organic food”, then that might show a Copilot result. 

Search engines are iterating to satisfy the user, and the experience is changing month to month. We tend to satisfy the user, and we see AI in search results as one way to do so — we call that generative AI search. Ultimately, users go to Copilot and the chat experience only when needed. We’re importing the value of AI directly within the search results. 

Is conversational search a big opportunity for brands?

Adrien: As we’re moving away from the blue links experience, providing the capacity to converse with the technologies is actually a bigger opportunity for brands to get connected to their prospects or clients? 

Fabrice: Yes. AI is a solver technology through which we’re able to understand the user and their intent. What I’ve observed in the industry, and what you and others have done studies on, is the dollar value of the clicks coming from AI solutions. Even if there are fewer clicks, the clicks generated by AI-driven content are of higher value. For example, a user who interacts with AI-driven results might sign up for a newsletter or make a purchase, generating more revenue than traditional clicks.

What challenges do LLMs face with real-time data on the web?

Adrien: There’s a key question from our clients about the challenges that LLMs have connecting to real-time data on the web. What would be your recommendation for brands? 

Fabrice: The challenge for these LLMs is that their content is outdated. For example, with Cyber Monday and the holidays, people will be shopping. The LLM won't have any of your new products or discounted prices. And this is a real problem, because this is the time of year when shopping solutions on the internet are getting the most revenue. 

And so LLMs contacted Bing to get help powering data freshness. They benefit from their LLM dictionary to understand the terms and the products, but they benefit from the fresh content to enable compelling relevant experiences on these queries — because people will search for the best deal, period. They’ll keep iterating their query to find the best price. Remember, this is all about satisfying the search intent. It's all about fundamentally displaying whether we can find out if that product is $49.99 versus $179.

What are your practical recommendations for retailers?

Fabrice: It's all about controlling the flow of content. We found a way to solve for crawling and crawl budget by offering you the ability to publish content in Bing and elsewhere. You are in control of publishing. You know when your prices or product listings are updated; use tools like sitemaps or IndexNow to inform search engines directly. This also helps avoid dead links. People clicking dead links is a very bad signal for search engines — when they reach a discontinued product or it's a dead link, then they click back and visit the next link.

By doing this, you help Bing and AI solutions have the most up-to-date information. Someday Google will also join us in this process, because clearly we’re becoming fresher than them on many millions of sites already. 

Adrien: So we're moving away from the old crawling model and into a more proactive approach where brands push content updates directly to search engines?

Fabrice: Yes, this is a fundamental transformation in the way we handle the internet. We believe that by 2025, 75% of sites will adopt this push model. It’s a smarter way to handle content updates than relying on us to pull in the content. We don't know when you publish content. 

We’re opening the door. We’re welcoming pushed content to solve crawling, crawl budget, JavaScript. All these things can be solved by facilitating this healthy data flow with a site. 

Adrien: And Botify is partnering with Bing to help this process and make it seamless through our Push to Bing features. 

Can you explain why it’s so difficult to crawl the web?

Fabrice: First, we measure crawling success in terms of URLs that are not in the system. In terms of what we call crawl efficiency, we’ve failed if we have to crawl a URL twice. We observe what we do and what others are doing, and on average we observe bad metrics in situations where we’re unable to get all the content or the latest content, which results in crawling too much. 

Again, the solution is to funnel the latest information to us. This push process is critical to providing a healthy search experience, where all clicks are satisfied.

Can you tell us a bit about Bing’s partnership with OpenAI? 

Adrien: We’re starting to see OpenAI crawling our clients' websites. In some ways, they are using the Bing Search API, and in other ways, they're probably building their own search index. Do you want to comment on that? 

Fabrice: We have a deep partnership with OpenAI. You can go to their website and see that they have three bots that are crawling the web. And, as I shared initially, they discovered that an LLM by itself is not satisfactory. So in some cases, they leverage the Bing Index for fresh content. In other cases, they also try to do their own search. 

I encourage everybody to try OpenAI ChatGPT Search. You’ll experience a mix of AI and search that’s powered by both Bing and OpenAI. They’re incubating and developing this technology. And as they’re developing, they’re certainly an interesting player to follow. 

But there’s more than just OpenAI. There are all kinds of AI search solutions across the landscape. We’ve observed a kind of dilution of search across all industries, but there’s always a solution that’s visible on phones, or maps, etc. People are using many different solutions to satisfy their need to search. 

How can brands measure share of voice against competition?

Adrien: It looks like you're making major improvements to Bing Webmaster Tools, and there’s a key question about how to measure the share of voice of a brand against the competition. What are you planning here in terms of data accessibility for big brands? 

Fabrice: We want to be transparent. We’re only major search engine that’s offering visibility into clicks from search and AI like Copilot, and we’ll continue to do so. 

We also just extended the data. Instead of six months of data, now we’re offering 60 months of data, enabling you to see year-over-year trends. It's all about data sharing to understand the trends. For instance, now that you can see the full 12 months, you may discover that something is only a seasonal problem. 

Why are AI search engines using Bing’s index instead of Google’s?

Fabrice: Microsoft, under Satya Nadella, has taken a more open approach to partnerships. Bing powers several search engines, like Yahoo and DuckDuckGo, and has extended to AI solutions to deliver quality content and results for the whole industry. 

For example, we just announced that Archive.org has adopted IndexNow. It's not a big business, but it is an open source repository of the internet. We’ll continue to collaborate in the industry to ease crawling. We don’t want the internet to have 600 crawlers from every single AI solution bombarding your website with crawl requests. We want to be respectful. 

Key takeaways

1. AI in search enables customization, conversation, and better results for consumers: AI technologies, such as LLMs and Bing’s Copilot, are transforming search by enhancing the quality of search results and providing more targeted, relevant experiences. This change offers opportunities for brands to engage with users in new ways.
2. LLMs are challenged by providing fresh content to consumers in search: Real-time content provides accurate search results, particularly in time-sensitive situations like holiday shopping. Brands must actively push fresh content to search engines using sitemaps and IndexNow for best results.
3. Brands can take control via automation, pushing content to indexes, understanding intent, and more: The search industry is transitioning from a "pull" model (where search engines crawl websites for updates) to a "push" model, where brands proactively provide updated content to search engines to improve crawl efficiency and data freshness. Bing is committed to offering brands greater transparency by providing detailed data on search interactions and trends, helping brands measure their share of voice and track long-term performance.

‍