Back to blog

SEO News & Events

The History of SEO: Through the Eyes of Daft Punk & “Robot Rock”

min read

March 18, 2020

Did you know Daft Punk and search engines debuted in the same year? Well, it's true! The SEO industry - now worth $80 billion - first came into play in 1993, the same year that the French-electro duo, a.k.a. the "robots" we love to dance and sing along to - formed Daft Punk. That makes SEO and Daft Punk the same age! And get this: the SEO industry is nearly the same net worth of each Daft Punk robot!

So, how connected are the bots of SEO to the bots of Daft Punk? Turns out, very! Through the lens of Daft Punk, let's discuss the story of search engine optimization.

Want to get into the Daft Punk spirit? Check out our Daft Punk playlist on Botify's official Spotify account to get into the vibe as you read along!

And what could help explain the history of algorithm optimization more than the birth and progression of "Robot Rock," a Daft Punk anthem and an umbrella term we're going to use to describe their music throughout this article.

Let's take a look at where it all started.

Robot Rock (1991-1999)

Daft Punk ranks with Robot Rock

In 1991, the first website was created by Tim Berners-Lee. The site focused on the World Wide Web Project and ran on a NeXT computer at the European Organization for Nuclear Research, CERN. A few years later, the first search engines, such as Excite and Yahoo, were launched with the mission of satisfying people's need for navigating the web. According to Internet Live Stats, as of March, 2020, there are now over 1.7 billion websites registered. That's quite a lot of growth in just 30 years! With this in mind, it makes sense that the role of SEOs have also evolved tremendously during this timeframe.

In 1997, something huge happened.

Not only did Daft Punk finally drop their first album, "Homework," but it was also the year that Larry Page and Sergey Brin registered the Google domain.

During the 1990s, SEO mainly existed as an isolated functionality, usually outside the enterprise environment. For example, porn sites, pharmaceutical sites, and gambling sites at the time were some of the first to adopt and benefit from SEO since they weren't able to advertise on many other marketing channels. It was all about freelancers working on black hat tactics to stuff keywords on pages and buy backlinks. To describe this age of SEO, we'd like to reference Daft Punk's most "black-hat optimized" song, "Robot Rock," -- one in which the duo repeats "Rock, Robot Rock" a whopping 52 times!

We just can't make this stuff up!

As time passed, algorithms evolved, SEOs raced to keep up with new challenges, and in just under seven years, we moved into the next era (of Daft Punk and SEO!).

Technologic (2000-2003)

The age of "buy it, use it, break it, fix it"

Google was quick to figure out a way to drive revenue, both for themselves and other brands, through the launch of AdWords - the first alternative (or partner in crime) to SEO for increasing website traffic through search.

Since many SEOs at that time were freelancers, the launch of AdWords meant that they were also struggling to prove their worth against a channel that generated ROI much quicker than SEO. Media agencies quickly saw the upsell benefits of adding pay-per-click (PPC) into their marketing bundles, instead of evangelizing the seemingly elusive organic search channel. PPC was much more aligned with traditional marketing advertising, so many companies found it easier to adapt paid search to their existing digital strategy.

While SEO was already a highly effective, long-term revenue generator, the main struggle lied in getting buy-in from stakeholders who wanted to see immediate results. Although the benefits of SEO were far-reaching, there weren't many tools on the market to help with tying SEO metrics to business KPIs. So for now, PPC appeared to be most desirable.

The challenge of measuring SEO's impact on traffic, revenue, and calculating ROI made it difficult to generate buy-in for processes that decision makers didn't yet understand.

Because of this, there was a huge need for SEOs to express their voice. Digital marketing communities began to emerge to foster knowledge sharing within the rapidly growing digital landscape.

With that, the first SEO communities began to take shape:

In 2000, the first Pubcon launched at a London pub (hence the name), encouraging knowledge sharing among SEOs and webmasters.

In 2003, Search Engine Journal, the first SEO-focused online journal, became the first official advocates for the power of organic traffic.

Meanwhile, Google kept improving to better satisfy searchers. They implemented their first major algorithm update, "Florida," with the goal of eliminating some of the black-hat practices. But since these keyword-stuffing tacts were all some people knew, a lot was at stake for the SEO community. Companies weren't able to use the processes they were familiar with, and they began experiencing fluctuations and declines in traffic that were difficult to explain. Everything began to get more complex.

In fact, there was a real need for a new era to come, which arrived in the form of, "Digital Love."

Digital Love (2004 - 2009)

Digital love: giving us all the feels

At this point, you could say the digital world had reached its teen years. It had become a time of social media, geographic intent, and oh, so much more. And in terms of search, channels like Facebook, Twitter, YouTube, and Google Maps gave the SERPs a facelift by allowing rich content to appear in universal search. The whole digital space started to get a lot more exciting!

Here are a few of the algorithm updates that SEOs faced during this period:

Jagger (2005): Backlink update to penalize unnatural link-building, paid links, and other spam

Big Daddy (2005): Infrastructure update that impacted URL canonicalization and redirects

Vince (2009): Authority becomes more significant for competitive keywords, ultimately favoring big brands over smaller players in the SERPs

Caffeine (2009): New web indexing system that enabled Google to crawl and index more efficiently, resulting in 50% fresher results

SEOs found themselves spending a lot of time learning and knowledge-sharing to keep up with the effects of algorithm updates. Meanwhile, social media was becoming the new, popular kid in town. The ideas and strategies behind social media made more sense to the consumer market, and companies began experimenting with their digital strategies to avoid their own kind of "FOMO," so to speak.

SEOs just weren't getting the attention they deserved, so they became used to working in silos, relying on the outsourcing of agencies, and confiding in the wider SEO community - while social and paid search immediately got the best seat in the house: at the marketing table, along with brand, PR, and content.

Mean Girls gif saying You can't sit with us

Also at this time, we saw new search engines appear, such as DuckDuckGo (2008) and Bing (2009). But none of the new players grew to be the size of Google, which left Google steering the reigns.

It was time for SEOs to become:

Harder, Better, Faster, Stronger (2010 - 2014)

Harder shells, better results, faster updates, stronger SEOs

SEOs really did become stronger. The value of organic search started to attract the attention of big brands that were seeking a hero who could bring them free traffic from Google Search. Retailers and other legacy brands began to see a drop in the "given" traffic they normally gleaned from SEO as more online-focused, tech-savvy businesses emerged in the SERPs and climbed up the rankings. Everyone else had to catch up, and it became a race to win.

It was a difficult time to start a career as an in-house SEO. After all, they were tasked with trying to show the worth of SEO within a business that had preexisting values elsewhere and work on fixing issues that should've been resolved ten years ago and had already done much damage. It was a bit of a mess.

Daft Punk members with fire burning in front of them

Many SEOs carried the workload of an entire SEO team on their shoulders, struggling to prioritize their time and ongoing tasks. Proving the value of organic search to grow their teams and get buy-in was harder than ever - especially since there wasn't enough time in a day to gather reports and tie long-term SEO metrics to business outcomes.

Plus, Google released some of the biggest algorithm updates in history, which resulted in unexpected changes in rankings and hard-hitting penalties. "Content is king" - a phrase originally coined by Bill Gates - grew a dark underbelly as sites tried to keep up (and backtrack) to make sure their site's content was high quality and accepted by Google's standards.

Here are some of the updates that became part of the SEO ecosystem:

Panda (2011): Essentially comprised a series of questions that determined the quality of a site's content (from number of ads on the page to originality of content) to tie ranking signals with real searchers' feedback on content quality. The result? The "content farm" business model faded away, 12% of queries were impacted.

Penguin (2012): debuted as a means to catch and prevent sites from skewing search results through buying or obtaining links, keyword stuffing, and more.

Hummingbird (2013): allowed Google to better understand the semantics of search queries, impacting more than 90% of all searches worldwide.

Pigeon (2014): put a higher focus on local SEO by rewarding local businesses that had a strong organic presence with greater visibility in the SERPs.

These updates shook the SEO world, changing the game of (ro)bots to include the real human factor and make sites user-friendly. It was time to demand the attention SEO always deserved and show business that they needed to start:

Doin' It Right (2015 - Present)

If you do it right, let the robots go all night

This leads us into the current age of SEO, which is all about doing right by searchers and visitors. SEOs have come a long way from resorting to tedious work in a silo and only getting attention for the wrong reasons (like when something seems wrong on the surface). The industry has evolved into much more than that - with an amazing community, global events, and the tools and resources needed to help SEOs succeed and show the results of their work.

We are in the age where we saw;

The year of mobile in 2015 when mobile became a mainstream marketing solution

In 2016, Google announced that they would move all websites over to the MFI (mobile-first index) progressively. In March 2020, they announced that all sites will be officially migrated to the MFI by September 2020.

RankBrain in 2015 and more recently BERT in 2019 have changed the way search engines understand queries to help them serve more relevant answers - bringing the "thought processes" of humans and bots closer than ever before.

Also during this time, customer-first brands emerged (like Botify partners AB Tasty and WunderKind) that focused on the visitor experience and customer feedback.

We've reached an age where the value of SEO isn't questioned quite as much as it used to be, but buy-in for the execution of SEO tasks and investment in SEO software or teams is still a struggle.

It's not so much about explaining the technicalities of SEO (like communicating the importance of canonicals and hreflang tags), but more so about getting stakeholders to prioritize SEO and translating the implications of SEO's effects on all marketing channels (and vice versa).

When SEO is prioritized, it can drive long-term sustainable traffic and revenue. And, when it's prioritized across teams (from your developers to your paid marketers), it can maximize ROI.

The game changed from:

"Keywords" to "Queries"
"Keyword types" (such as navigational, informational, or transactional) to "Searcher Intent"
"Internal Linking" to "Navigation"
"Backlinks" to "User Journeys"

And the list goes on.

But the point is, SEO has evolved, in a sense, to User Acquisition Optimization. Now, the priority of SEOs is helping businesses adapt to an ecosystem that's ever-evolving and complex.

At Botify, we've seen brands unable to adapt their SEO processes, we've uncovered real data proving that Google misses 51% of a typical enterprise site, and Forrester Consulting has shown that executives who prioritize SEO call it a top revenue generator - yet only 14% of enterprise organizations are making that commitment.

When given the right tools, resources, and support, SEOs have real power. For one, when an entire team - from the stakeholders to the developers - prioritize SEO, it supports all of the organization's marketing efforts. Meanwhile, unified SEO software can help them make sense of their site's data, streamline processes, catch anomalies before they impact traffic, and reach solutions faster. SEO tools and software have given them the fuel they need to keep up with a continuously evolving industry (3200+ algorithm updates each year!) but also to show the efforts of their work.

While technical SEO still makes them tick, the SEO role is so much more than that. SEOs encompass the roles of teachers, evangelists, coders, copywriters, researchers - and the list goes on. To be truly successful in their role, they need to build collaborative relationships across their marketing departments and think about SEO comprehensively - to understand their site's performance through every phase of search.

On top of that, search is changing. New technologies, like Google Discover and Alexa, bring new complexities to the table - like anticipating a searcher's interests and comprehending voice search. And advancements like Google's BERT means we're that much closer to bridging the gap between bots and humans.

But then again, maybe Daft Punk predicted it all.