If your websites are like most, they include a fair amount of extra "stuff" in the title tags: things like your brand name or repeating boilerplate text that appears across multiple pages.
Should you include these elements in your titles automatically?
To be fair, most sites do.
Alternatively, could it help your SEO to actually include less information in your titles? (Or at least in specific circumstances?)
We know from a handful of studies that titles of a certain length tend to perform better. A now-famous study from the engineers at Etsy showed how shorter titles performed better than longer ones. SEOs speculate that this could be because shorter titles can have more focused relevancy (by focusing on core keywords), might earn higher click-through rates, or some other reason we can't imagine.
When choosing which part of a title to shorten, brand names and boilerplate text are obvious choices. But how do you determine if this is something you should consider for your own SEO?
Here's an example of a brand's site name at the end of every title:
We've all seen sites like this. Heck, most of us do this on our own sites. The question is, does having our brand/site name at the end of every title actually help, or hurt?
But first, we also have to consider other types of boilerplate.
What is boilerplate? Boilerplate simply means standardized, non-unique pieces of text that are used over and over again. This often includes things like categories, product categories, author tags, and taglines.
In this example below, the boilerplate text on every title includes "Tomatoes - Vegetable Seeds - Shop."
Sometimes boilerplate material can become quite long. The comic book review site Major Spoilers (awesome name!) often includes the same 65-character boilerplate on many pages:
"Major Spoilers – Comic Book Reviews, News, Previews, and Podcasts"
Of course, at this length, it's so long that Google truncates every single title:
The problems that boilerplate can cause your SEO are threefold:
Relevancy: Unnecessary words can make your title less relevant, both to search engines and users. For search engines, this could mean lower rankings. For users, this could result in fewer clicks.
Uniqueness: Titles that share the same repeating text, and only vary from one another by a word or two, aren't very unique. While this isn't necessarily a problem, it goes against most SEO best practices, where uniqueness is key.
Length: Boilerplate means you have less room to display other words in your title, and Google will often cut these off if they go beyond a certain length.
Experiment #1: Remove category from title
We decided to run a couple of boilerplate experiments here at Moz, to see if we could increase our rankings and traffic by removing some of the repeating parts of our titles.
We started with our Whiteboard Friday blog posts. Every time Moz publishes a new Whiteboard Friday, we traditionally include "Whiteboard Friday" in the title.
What would happen if we removed this from the titles?
Using an A/B split test methodology — where we rolled the test out on 50% of the titles and used the other 50% as a control — we saw an amazing 20% uplift from this experiment.
This chart represents the cumulative impact of the test on organic traffic. The central blue line is the best estimate of how the variant pages, with the change applied, performed compared to how we would have expected without any changes applied. The blue shaded region represents our 95% confidence interval: there is a 95% probability that the actual outcome is somewhere in this region. If this region is wholly above or below the horizontal axis, that represents a statistically significant test.
Honestly, the results surprised us. Whiteboard Friday is a popular brand (so we thought) but removing this boilerplate from our titles produced a significant uplift in traffic to those pages.
At this point, we got cocky…
Experiment #2: Remove brand from title
If removing the category name from Whiteboard Friday posts produced such a significant uplift, what if we removed our brand name from all titles?
For this A/B experiment, we did exactly that—removing the word "Moz" from 50% of our titles and measuring the results.
Crazy, right? If it worked by removing "Whiteboard Friday" would we see the same uplift by removing "Moz?"
Sadly, Google had other plans:
While this A/B test never reached full statistical significance, we actually saw a 4% decline in traffic by removing our brand from our title tags.
Boo!
So why did this test not produce the same gains? To be honest, I've removed the brand name from other site's titles and seen as much as a 20% uplift.
It turns out that whether or not removing brand/boilerplate will be beneficial to your SEO depends on a few key factors, which you can gauge in advance.
How to know if removing boilerplate may succeed
Over 10 years of experience and literally millions of title tags, I've found that there are basically four factors that influence whether or not removing boilerplate from your titles might be beneficial:
Brand Strength: Popular brand names in titles almost always perform better than unknown brands, even when people aren't searching for your brand specifically. Amazon's brand recognition, for example, likely gives a significant boost to including "Amazon" in every title, even when people aren't specifically searching Amazon. Less recognizable brands, however, don't always get the same boost, and can actually lead to fewer visits based on relevancy, length, and clickability (described next.)
Relevancy: Are your boilerplate/brand keywords relevant to what your users search for? For example, if you're site is about television repair, then boilerplate titles that say "Brad's TV Repair" are going to be much more relevant than boilerplate that simply say "Brads." (We'll explore a way to determine your boilerplate's brand strength and relevancy in the next section.)
Length: In general, long boilerplate has the potential to do more harm than short boilerplate/brand words. Long boilerplate can dilute the relevance of your titles. So if you include "Buy Brad's TVs, Television Repair, High Definition Servicing, Audio and Visual Equipment for Sale in Houston Texas and Surrounding Areas" - you may want to rethink your boilerplate.
Clickability: Sometimes, boilerplate can make your titles more clickable, even if they aren't terribly relevant. Words like "Sale", "Solved", "Free", "2020", "New", and many others can lead to an increase in click-through rates (CTR.) Sometimes you can't tell until you test, but in many cases even adding clickable elements to your boilerplate can lead to significant gains.
Simple technique for determining your brand strength and boilerplate relevancy
This simple technique will also show why removing "Whiteboard Friday" led to an increase in traffic while removing "Moz" from titles did not.
Here's what you want to do: for each piece of boilerplate, determine the number of URLs on your site that rank/receive traffic for those keywords.
Simply enter your boilerplate/brand as a query filter (you may need to break it into chunks for longer boilerplate) and see how many URLs receive traffic for queries that include that keyword.
When we filter for keywords that contain our "moz" brand name, we find thousands of ranking URLs.
People are searching for things like:
Moz DA Checker
Moz Pro
Moz SEO
Moz Blog
Etc., etc.
As our brand name is part of so many queries and leads to visits across thousands of pages, this tells us that "Moz" is a very strong brand, and we'd likely be smart to include it as part of our title tags.
"Moz" is also very short at only 3 characters, which doesn't hurt either.
So what happens when we try this same technique with "Whiteboard Friday" — the boilerplate that led to a 20% uplift when we removed it? We see a very different result:
In this case, almost all the traffic for "Whiteboard Friday" search terms goes to only one or two pages.
For most Whiteboard Friday posts, the term is simply irrelevant. It's not what people are searching for, and the brand isn't strong enough to produce additional uplift.
Also, at 17 characters long, this boilerplate added significant length to each of our titles, in addition to possibly diluting the relevancy for what the posts were ranking for.
Final thoughts + bonus free title tag webinar
These tips can't tell you definitively whether you should or shouldn't include boilerplate or brand in your title tags, but they should give you a pretty good idea of when you should test things out.
Remember: Always test and evaluate before making any SEO change permanent. At least know the impact of the change you are making.
Also, please don't be under the impression that you should always remove boilerplate from your titles. In some instances, actually adding boilerplate can produce an uplift, particularly when the boilerplate is:
Recognizable: For example a strong brand
Relevant: The right keywords
Clickable: Encourages a high CTR
Succinct: Not overly long
If you found value in the tips, and want to learn even more ways to optimize your title tags, we've made available a free webinar for you: SEO Master Class: Advanced Title Tag Optimization (For Any Site).
If you've got 40 minutes, it's definitely worth a watch.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Since Moz released the new Google Data Studio Connectors for STAT, you might be wondering how to best implement them for your reporting strategy. My colleagues at Path Interactive and I absolutely love how granular you can get with your reports in STAT, and finally having the ability to cleanly and effectively pull those reports into Data Studio (the tool we use for our own reporting) is a godsend.
While the Historical Keyword Rankings connector reports on rank over time, it may not be as obvious how to report on rank change over time. In this post, I'll give you step-by-step guidance on how to report on rank change — as well as a couple other filtering and reporting tips — while using the connectors within Google Data Studio.
If you aren't a STAT user yet but you want to know how it might fit into your SEO toolkit, you can take a tour of the product. Click on the button below to set one up!
Before you begin, you need to identify a few things to set up the connector: your STAT Keyword API Key, the Project ID, and your Site ID. If you don’t already know how to identify these via the STAT API, you can head over to STAT’s documentation here to learn more. After you’ve identified these, it’s time to connect your data source.
We’re going to be doing something a little out of the ordinary here, but stay with me — you’ll see why in just a second!
For this step, we'll be connecting two instances of the same source. Because our goal is to compare rank change over time, we’ll use the same source twice to identify those deltas.
When setting up your connector, be sure to name the source something that you’ll easily recognize:
In my case, I usually go with something simple such as “[client name] STAT Keyword Connector.” When this is complete, repeat the step above, but name it something different, e.g. “[client name] STAT Keyword Connector 2.”
Finally, make sure the metrics you plan on comparing have unique names for each connector. To do so, go into your data source. Click on the metric’s name so that you can rename it, and then rename it something unique. For this case we'll be doing it for “Google Base Rank,” since we're comparing ranks, but it can also be done for “Google Rank,” if we wanted to compare that. Again, I like to just keep it simple: for the first data source call it “Google Base Rank 1,” and then for the second data source call it “Google Base Rank 2.” When all is said and done, it should look something like this:
Building your table and blending data
Now we’ll start to get a bit more technical. Blending the data of the two connectors lets you compare two instances of rankings against each other. Your final result will produce a table showing the ranks of two given dates, as well as their rank change. The five-step process will look like this:
Blend data of keyword connectors one and two
Add in your common metrics for the two sources (keyword at the minimum, but you can also add in location, device, market, and search volume)
Add in the metric you'd like reported (Google base rank and/or Google rank)
Set date range
Apply “No Null” filter
1. Blend data of keyword connectors one and two
The first step here is to blend the two connectors so that you can compare two instances of ranks against each other.
First, you need to create a new report, or go into a report that's already set up. Next, select your data source. Here you'll select the first instance of the source that you set up earlier (if you’re starting on a fresh report, it'll ask you to add a data source immediately). Once selected, click on “Blend Data” underneath the data source on the right hand side of Google Data Studio, seen here:
This will bring you to the Blend Data source tool. From here you select to add another data source, being your second instance of the connector.
2. Add in your common metrics
Once you've chosen to blend both connectors, you need to set your metrics. Towards the top, you’ll see “Join keys.” This is in reference to what's going to be the same for both instances, so here at the minimum, you want to include “keyword.” Feel free to play around here with adding different metrics.
Note: We'll go over this later, but if you plan on having different graphs filtered by a certain tag or location, make sure to add these in here.
3. Add in the type of rank you want reported
After setting your metrics under “Join keys,” now select the metrics that will be unique for each date. Depending on what you want to compare, under “Metrics” you'll pick “Google Base Rank,” “Google Rank,” or both. You may also include “Date” here too if you'd like. Once done, click on “SUM” next to the metric name, and change this to “MIN.” You'll see why in just a moment.
At this point, your blended data should look something like this:
4. Set date range
Now you need to set the two date ranges you're comparing to each other.
To do this, under the first connection, set your first date: Under “Date Range,” click on “Custom,” then click on the field to select your date. Here you might see that there's an option for two dates, but for this solution, we're using the same date for each connector.
In the end, it'll be something like “Connector 1” selected for the “start date” and “end date” as the first of the month, and for “Connector 2,” the “start date” and “end date” will be the last of the month. This is essentially pulling in the rank for the first instance as well as the second instance, so you can compare the two.
5. Set “No Null” filter
The last step in setting up your blended data is creating a “No Null” filter. When the keyword connector reports on ranks that your site is not ranking for, it will return as "null." To avoid flooding your data with fluff, you need to create a filter removing instances of "null."
First, click on “Add A Filter” below where you selected the date range. Next, towards the bottom, click on “Create A Filter.” Set the parameters of the filter as “Exclude” > “Google Base Rank 1 (2)” > “Is Null.” Be sure to name the filter something identifiable such as “No Null.” It should look like this:
Applying rank change to your report
Now you can create a new field that will report on the rank change by making a calculated field to find the difference of the two ranks.
Under dimensions, select “Add Dimension,” and click on “Create Field.” You can name it “Rank Change,” but to create the field, start typing “Google Base Rank,” and you’ll see your instances from each connector come up. To make the calculated field, select your “Google Base Rank 1” and subtract it from “Google Base Rank 2,” so it should look something like this:
Hit apply, and now your rank change should be calculated!
There is also an additional way to get the same result, but with a few drawbacks, such as not being able to name the header, as well as not being able to filter or sort your rank change. The benefit to this approach is that it's easier to set up initially, as you don’t actually need to blend the data. However, not setting up the blended data will also forfeit having the initial rank visible. When in your edit view, set a custom date range that you're reporting on under “Default date range.” Here, you can then set a comparison date: if looking back a month, you can set this to the first. If you go with this option, it should look like this:
Head into the "Style" tab, where you can change the comparison to “Show Absolute Change” under “Metrics.” You can also change the colors of your positive and negative arrows to more accurately represent the movement (you can see from above that the “negative” change is a green arrow, this defaults to red).
Using filters
Applying filters to your data set can be extremely beneficial to making sense of your data! Using filters with the connector can help you segment out rankings for a particular location, or create charts that show rankings for a specific keyword group that you’ve set up using keyword tags.
Take a look at this report I set up as an example. Within STAT, I created keyword tags to target locations determined by what zip code they were. Then, I was able to create a filter for each chart targeting that keyword tag:
Setting filters up is extremely simple. First, go into edit mode. Next, scroll down the side until you find “Filter.” Then under Filter > Table Filter, click on “Add a Filter.” This will bring you to the filter picker. Toward the bottom, click on “Create a Filter.” Here you can set the parameters for the filter you'd like to show.
Some of my other favorites include filtering to only show the top few pages (filters out non-relevant and high ranks), using the keyword tag filter like I showed before, and also filtering by location. But you don’t have to stop there! Adding in the additional dimensions available to you in the connector, you can use the filter to show things such as desktop vs. mobile or how your keyword ranking performance does in different markets.
Blending your Google Analytics, Google Search Console, and STAT data
One of my favorite uses for the connectors is the ability to blend the data with your Google Analytics and Google Search Console data. By blending this data together, you’re able to directly tie keyword rankings with different metrics, such as clicks or goal completions.
You’re probably a pro at blended data at this point, but just for reference, the data blended should look like this:
A few things to note: it’s important what order you put the connectors in. I’ve found that adding the STAT connector first works best (i.e. if you put Google Analytics first, you’ll get a report with the infamous "not found" keyword). Additionally, to pull in Search Console data in order to match with your other connectors, using “Query” will have the same effect as “Keyword.”
The result would look something like this, but feel free to edit the design how you wish!
Now you can go even further with this and match up URLs, but this will require some RegEx.
You'll rename the “Google URL” field in STAT and “Landing Page” field in Google Search Console in order to match the URL structure convection within Google Analytics by taking out the domain portion of the URL. To do this, go into your data source for each STAT connector and Google Search Console, and click “Add A Field” in the top right.
Next, enter to following RegEx for the STAT connector:
REGEXP_REPLACE(Google URL, ".*[\\.]com", "")
And for Google Search console:
REGEXP_REPLACE(Landing Page, ".*[\\.]com", "")
Remember to name them something to differentiate from the default field. I use “Landing Page (no domain).”
When building a report, use these new fields for consistency across the URL structure so that, when you select them when blending data, they'll match.
Use this method in the same way as above to get the desired results of pulling in data from across all three connectors to match up with each other! In the end you should be able to find what keyword ranks for what URL, as well as have many sessions or clicks that are brought in as well as goal completions, or any other combination.
Well there you have it! Hope this was helpful to you. If you have any other questions you can comment below or find me on Twitter @ianpfister. Happy reporting!
To help us serve you better, please consider taking the 2020 Moz Blog Reader Survey, which asks about who you are, what challenges you face, and what you'd like to see more of on the Moz Blog.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
To anyone working in SEO, it’s fairly evident that this is a male-dominated industry. Although there are powerful women SEOs in the field (like Moz CEO Sarah Bird, for example), if you glance at a conference speaker lineup or peruse the bylines on search-related blogs, you’ll see that those who identify as female are few and far between. A recent list of the 140 most influential SEOs featured 104 men and just 36 women.
So how big is the gender gap? And how does it translate to tangible things like pay and job titles? To find out, we mined the data from our State of SEO 2020 survey, which featured 652 SEOs in 51 countries. Here are some of the things we learned.
But first, a mea culpa. If SEOs who identify as women have an uphill climb in this industry, there’s no doubt that female-identifying SEOs of color have a hill that is steeper still. I deeply regret not asking demographic questions on race and ethnicity which would have allowed us to analyze the disparate impacts that bias plays on BIPOC women SEOs. It was a missed opportunity. That said, we are currently running a survey on BIPOC in SEO that aims to cover those issues and more as we continue to take an introspective view of our industry.
Men outnumber women by more than 2 to 1 in SEO
Of the 652 SEOs who participated in the study, 191 identified as women (29.3%) and 446 identified as men (68.4%). Additionally, one identified as non-binary and 14 preferred not to say. Data was collected on a SurveyMonkey form. We reached out to our own database, purchased lists of SEOs around the world, and promoted the survey on social channels for respondents. We offered no compensation or reward for participating. Non-binary, persons who chose not to identify a gender by choosing “preferred not to say”, and SEOs from the African continent were underrepresented mostly due to the outreach database itself. Finally, respondents selecting non-binary and “preferred not to say” were not calculated in the men/women percentages.
A voluntary survey is not a scientific sampling, but those percentages mesh with previous studies by Moz that found those who identified as women made up 22.7% of SEOs in 2012, 28.2% in 2013, and 30.1% in 2015. In all four studies, men outnumbered women by more than 2 to 1.
Importantly, the new results suggest the gap hasn’t narrowed over the past five years.
This was not a surprise to many female-identifying SEOs who participated in the study.
“I started out in the SEO industry about 10 years ago. Compared to that, I do see more women at conferences, on online platforms, and in the day-to-day work with clients,” one said. However, she added that she hasn’t seen much progress in the last 5 years. “It’s like we are kind of stuck. I suspect it’s at least partly a visibility issue: Men have been there forever, building their reputation and expertise. It is hard to keep up with that if you had a late start.”
We interviewed more than a dozen female-identifying SEOs, most of whom asked not to be named. Although a few had supportive bosses, clients, colleagues, and mentors along the way, many shared experiences of being passed over for promotions, having to fight to be heard in meetings and, in some cases, being paid less than men for the same work.
“I think you can sum up the SEO industry by looking at speaker panels of all the major conferences. There is no equality. Are you a white male and 50+? You must be an expert! Are you a woman, 40, who’s been doing this since 2004? Oh, honey, go sit down. We have an expert already,” said one woman. She said she spent 13 years at a website development company being “constantly overlooked” before moving to a digital marketing agency where she is respected and valued.
The gender gap is widest in Latin America
Global internet usage has boomed over the past two decades, with more than 59% of the world’s population now online. Although internet penetration is highest in Europe and North America, more than three-quarters of global users live elsewhere. These growing markets are served by robust communities of SEO professionals on every continent.
Our study reached SEOs in 51 countries, which we grouped into 11 large regions. Participation was highest in the U.S. with 269 SEOs responding, but we also surveyed 113 SEOs in Western Europe, 85 in the U.K., 43 in the Eastern Europe/Balkans region, 39 in Australia and New Zealand, 30 in Asia, 21 in Canada, 18 in Scandinavia, 16 in the Middle East, 12 in Central and South America, and 6 in Africa.
We found that the gender gap is most pronounced in Latin America (83.3% who identified as men to 16.7% who identified as women) and Australia/New Zealand (82.1% who identified as men to 17.9% who identified as women).
The gender gap is least significant in Africa (although with an admittedly very small sample size due to the small number of African SEOs in our database) and Canada (52.6% who identified as men to 47.4% who identified as women). Under Prime Minister Justin Trudeau, a self-professed feminist who appointed a gender-balanced cabinet, Canada has made gender equality a priority, but progress has been uneven at times.
When it comes to gender diversity in SEO, the U.S., Asia, and the U.K. all trail behind Europe, Scandinavia, and the Middle East.
Female-identifying SEOs are more likely to freelance and specialize in content
Generally, the three most common career environments for SEOs are serving as an in-house expert at a single company, working in an agency setting, or operating independently as a consultant or freelancer. Each path has its own pros and cons. We found some interesting gender differences in where SEOs are working.
Male-identifying and female-identifying SEOs are equally likely to work in-house, with about 40% of both genders working inside a single business. And as we discuss below, both genders reported being satisfied with the working conditions and level of support they received in their roles.
Among those who practice their craft externally, men are slightly more likely to work in agencies than women (49.7% vs. 42.5%).
The biggest gap was among freelancers. Female-identifying SEOs are almost twice as likely to be contractors or freelancers as those who identify as men (17.7% vs. 10.6%). However, it’s unclear if female-identifying SEOs are heading out on their own because they don’t feel they can get a fair shake working for others, or if they're drawn to the freedom and flexibility of freelance work.
Full-time freelancing has grown steadily across the economic landscape in recent years. It also tends to draw more women than men. Part of the appeal may be flexibility around childcare, but control over income was also a factor for some of the SEOs we interviewed.
“I think a lot of women choose to do freelance because they want to be paid what they deserve, frankly,” said one 25-year-old female SEO in East Anglia, U.K.
However, another woman who works as an in-house SEO said, “When I got my start in marketing, most of the jobs offered to me were contractor roles, and it wasn’t clear how to become full time. It wasn’t by choice; it was what was available to me.”
Many female-identifying SEOs said it was hard for them to get hired or promoted, even with stellar track records.
“I’ve seen loudmouth, no-record, no-proof men be hired. It’s absolutely aggravating. At my old company, I was skipped by two men who had half the knowledge for supervisor positions. Each man left within months to different companies to the next title,” said a 41-year-old female SEO in Minnesota. She subsequently changed companies and found a much more welcoming environment.
In addition to career paths, there are noteworthy differences in the areas of the industry that male-identifying and female-identifying SEOs are most likely to specialize in. Most SEOs consider themselves generalists, but among those who profess a specialty, women are twice as likely to be content experts (17.6% to 7.7%).
On the other hand, male-identifying SEOs are nearly twice as likely to be technical experts (21.5% to 12.6%). It’s unclear if this is a result of choice, fallout from the gender gap in STEM occupations generally, or if those who identify as women feel unwelcome among tech SEOs.
Among the female-identifying SEOs we interviewed, several said they think early gender stereotyping played a role, from the toys little boys and girls are given to what each gender is encouraged to pursue as a career.
“It’s similar to why women are not often involved in engineering jobs. Technical roles are historically associated with developer training, and women are more likely to transition from the marketing side than the programming side,” one said.
Several women also said technical SEO, in particular, is a “boys club.”
“I participate in online forums for general Tech SEO and Women in Tech SEO, and the vibes are much different,” one woman said. “The male-dominated general forums are competitive. The female groups are more supportive, but again, we are trying to bring along and encourage women in the field.”
Another tech SEO who worked at an agency and in-house before going out on her own as a contractor said the culture can be intimidating: “I find that men are quicker to hop on and attack people about technical knowledge than women.”
Female-identifying SEOs generally charge less than men for their services
To find out more about the dollars and cents of SEO, we asked the agency and contract SEOs who participated in our study about their pricing models. In all, 261 SEOs were willing to share how they price their services and how much they charge.
The three most common pricing models are monthly retainers, per-project pricing, and hourly rates. Although there was a wide range of rates among male-identifyng and female-identifying SEOs, the medians were consistently lower for those who identified as women.
Among agency and contract SEOs, men are more likely to price their services with monthly retainers (59.1% of men vs. 39.4% of women). Women are more likely to charge per project (31.8% of women vs. 18.2% of men). About a quarter of both groups use hourly pricing.
But before we talk about prices...
Before we get into the details of how much male- and female-identifying SEOs earn, it’s important to note that we didn’t ask who actually set the prices. Depending on the size of an agency, SEOs who work there may have very little control over the pricing structure.
The agency’s rates might be standard, or they might vary depending on who does the work. One can assume that freelancers choose their own rates, although they might be responding to signals about what the market will bear and what clients are willing to pay.
Some studies have suggested that a variety of psychosocial factors lead female-identifying freelancers to charge less than their male counterparts. For instance, a Hewlett-Packard study identified a confidence gap in which women tended not to apply for a promotion unless they met all the qualifications, but men would go for it if they met 60 percent of the job requirements.
Conventional wisdom holds that women are more cooperative and men are more competitive. Whether or not that’s true, men initiate negotiations more readily than women and tend to ask for higher compensation.
In a future study, we will certainly ask who determined the service pricing. For now, we can only report what male-identifying and female-identifying SEOs told us they charge.
Retainers for those who identify as male are 28.6% higher than for those identifying as female
Our respondents included 138 agency and contract SEOs who use monthly retainers as their primary pricing model. These retainers ranged from less than $250 a month to more than $25,000 a month, but overall they were higher for men. At the midpoint of the ranges on our survey, those identifying as male charge a median retainer of $2,250 a month while those identifying as female charge a median of $1,750.
When we looked at agency SEOs and freelancers separately, the median for freelancers was much lower, but it was the same for both genders: $750 a month. However, the sample size was quite small. There were only 19 freelancers in the study who primarily use retainers. Among the 119 agency SEOs who use retainer pricing, the median retainer was $2,250 for those identifying as male and $1,750 for those identifying as female.
Project prices for men are 66.7% higher than for women
Our respondents included 54 agency and contract SEOs who typically charge on a per-project basis. The scope and cost of projects varied greatly, from less than $250 to more than $100,000. But the data showed that overall, men charge more per project — a median of $5,000 vs. $3,000 for female-identifying SEOs.
We decided to dig deeper and found an interesting exception when we looked at agency SEOs and freelancers separately.
The price gap was more than three times as wide among those who work in agencies. Our sample included 36 agency SEOs who use per-project pricing. Male-identifying SEOs reported that their agencies charge a median of $8,750 per project while those who identify as women said their agencies charge a median project fee of $2,250.
The reverse was true among independent SEOs. The sample size was small, so we’re not sure what to make of it, but among the 18 freelance or contract SEOs we polled who charge by the project, women had the edge. Female-identifying freelancers charge a median fee of $3,750 per project to $1,750 for male freelancers.
One contractor in her 50s hypothesized: “I think women may be more detail-oriented and spend more time with their project. Maybe men may charge less because they have more clients?”
Median hourly rates for male-identifying SEOs are 16.8% higher than for female-identifying SEOs
Our respondents included 57 agency and contract SEOs who typically bill by the hour. Among this group, the median rate is $125 for male-identifying SEOs vs. $107 for female-identifying SEOs. In this case, the difference is largely attributable to more women working as freelancers. The median rate for men and women SEOs at agencies was $125 an hour, and the median rate for both who work as contract or freelance SEOs was $88 an hour.
Many of the female-identifying SEOs we interviewed said women tend to undervalue themselves and need to be more assertive in negotiating prices.
“I think confidence and not being scared to charge what you’re worth comes into play for the higher rates,” said digital marketing and content specialist Kristine Strange.
Both men and women feel equally supported as in-house SEOs
Some good news for in-house SEOs: When asked about working conditions, frustrations, and pain points, both men and women had very similar responses. Both reported strong levels of interdepartmental cooperation and support for SEO priorities.
Female-identifying SEOs are slightly more satisfied than male-idneitfying SEOs with in-house SEO resources
The resources available to in-house SEOs are largely dependent on the size and fiscal health of the company that employs them.
Among in-house SEOs, women are as likely as men to work for enterprise-level companies. We found that 27.1% of male-identifying in-house SEOs and 24.8% of female-identifying in-house SEOs work for companies with more than 250 employees. And 72.9% of male-identifying and 75.2% of female-identifying SEOs work for companies with 250 or fewer employees.
In-house SEOs across the board rated engineering support as their biggest challenge. Female-identifying SEOs were generally more satisfied than their male peers with the expertise of their teams and their staffing levels. They were equally satisfied with other elements of their SEO programs.
Conclusion
Although there are some very prominent and talented female-identifying SEOs, they are still underrepresented. And when they do enter the field, they are often compensated at lower rates than men. There is no single solution to broadening the talent pool, but we have a few thoughts.
Welcoming industry: The overwhelming number of women who spoke to us about these findings wished to remain anonymous. We can only assume that means female-identifying SEOs do not feel safe openly discussing issues of gender within an SEO workplace. Silence only serves to bolster the status quo. We must foster an industry culture that does not punish the whistleblower but instead seeks to listen, understand, grow, and improve opportunities for all its members.
Training and mentoring: More than in many other industries, there isn’t one clear path to becoming an SEO. The STEM fields are one training ground, but many other SEOs learn the craft from mentors. To achieve more diversity, which is good for the industry and outcomes, it’s important for girls and those who identify as girls to be supported and welcomed into STEM classes during their student years.
As an industry, we need to take the job of mentorship seriously. Experienced SEOs can do more to mentor young talent, particularly those who identify as women. Agencies can do more to recruit and hire people with different backgrounds.
Several women whom we interviewed mentioned the importance of mentors and allies:
"I sit in countless calls where I say something and until my CTO repeats what I say, some clients don’t hear me. My CTO is so supportive and wonderful, and he will literally say, 'She’s right when she says, ‘Blah.’ She’s got 20 years under her belt… .' Then their light turns on."
"I’m good at learning complex software and doing complex technical tasks but wasn’t encouraged in this until my recent job — and even then, it wasn’t until I got a female manager that I was recognized for this ability and assigned those kinds of tasks on a regular basis."
"I spent the first two years double- and triple-checking all my work, backing everything with links from male experts in the industry. One day the CTO told me I didn’t need to do that. He trusted me. I found myself in the bathroom in tears. It took me a long time to stop sending links. (Sometimes I still send links, but only if I think he needs to read them to keep up with me!)"
Transparency about pay and pricing: The taboo about discussing fees and compensation keeps inequities hidden. It’s time to shatter that norm. Independent SEOs should run their pricing plans by mentors of all genders for perspective. Agencies should be sure that skill and experience, not gender, is the driving factor in pay and pricing.
Don’t undersell yourself: If negotiation doesn’t come naturally to you, spend extra time preparing proposals. Research your competitors and talk with mentors. Focus on the value you’re adding. Be sure to factor in your skill level and experience as it grows. Don’t fall into the confidence gap trap. Even if you don’t tick all of the boxes, if you have most of the qualifications, forge ahead to apply or submit a proposal.
I want to acknowledge the important role that several female-identifying SEOs played in the making of this article. First, I have the privilege of working with some amazing women every day in my SEO agency. Thanks to Cindy Glover, without whom I could not have produced this study. I also want to thank Areej AbuAli whose work in creating the Women in Tech SEO community has been an invaluable resource to the SEO industry and in particular, SEOs who identify as women. Women in Tech SEO not only helps to amplify the voices of those identifying as women within the community, but also helps connect them to each other.
If you wish to explore your own possible implicit bias around issues of gender and career, check out Harvard’s Gender-Career implicit bias test.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
We spend a lot of time discussing SEO tactics, but in a constantly changing industry and especially in times of uncertainty, the strategies agencies should employ in order to see success deserve more attention. In this popular (and still relevant) Whiteboard Friday, Russ Jones discusses four essential success tactics that'll ultimately increase your bottom line.
Russ also delved into the topic of profitability in his MozCon Virtual presentation this year. To watch his and our other amazing speaker presentations, you can purchase access to the 2020 video bundle here.
Click on the whiteboard image above to open a high-resolution version in a new tab!
Video Transcription
Howdy, Moz fans. I am Russ Jones, and I can't tell you how excited I am for my first Whiteboard Friday. I am Principal Search Scientist here at Moz. But before coming to Moz, for the 10 years prior to that, I was the Chief Technology Officer of a small SEO agency back in North Carolina. So I have a strong passion for agencies and consultants who are on the ground doing the work, helping websites rank better and helping build businesses.
So what I wanted to do today was spend a little bit of time talking about the lessons that I learned at an agency that admittedly I only learned through trial and error. But before we even go further, I just wanted to thank the folks at Hive Digital who I learned so much from, Jeff and Jake and Malcolm and Ryan, because the team effort over time is what ended up building an agency. Any agency that succeeds knows that that's part of it. So we'll start with that thank-you.
But what I really want to get into is that we spend a lot of time talking about SEO tactics, but not really about how to succeed in an industry that changes rapidly, in which there's almost no certification, and where it can be difficult to explain to customers exactly how they're going to be successful with what you offer. So what I'm going to do is break down four really important rules that I learned over the course of that 10 years. We're going to go through each one of them as quickly as possible, but at the same time, hopefully you'll walk away with some good ideas. Some of these are ones that it might at first feel a little bit awkward, but just follow me.
1. Raise prices
The first rule, number one in Let's Make Money is raise your prices. Now, I remember quite clearly two years in to my job at Hive Digital — it was called Virante then — and we were talking about raising prices. We were just looking at our customers, saying to ourselves, "There's no way they can afford it." But then luckily we had the foresight that there was more to raising prices than just charging your customers more.
How it benefits old customers
The first thing that just hit us automatically was... "Well, with our old customers, we can just discount them. It's not that bad. We're in the same place as we always were." But then it occurred to us, "Wait, wait, wait. If we discount our customers, then we're actually increasing our perceived value." Our existing customers now think, "Hey, they're actually selling something better that's more expensive, but I'm getting a deal," and by offering them that deal because of their loyalty, you engender more loyalty. So it can actually be good for old customers.
How it benefits new customers
Now, for new customers, once again, same sort of situation. You've increased the perceived value. So your customers who come to you think, "Oh, this company is professional. This company is willing to invest. This company is interested in providing the highest quality of services." In reality, because you've raised prices, you can. You can spend more time and money on each customer and actually do a better job. The third part is, "What's the worst that could happen?" If they say no, you offer them the discount. You're back where you started. You're in the same position that you were before.
How it benefits your workers
Now, here's where it really matters — your employees, your workers. If you are offering bottom line prices, you can't offer them raises, you can't offer them training, you can't hire them help, or you can't get better workers. But if you do, if you raise prices, the whole ecosystem that is your agency will do better.
How it improves your resources
Finally, and most importantly, which we'll talk a little bit more later, is that you can finally tool up. You can get the resources and capital that you need to actually succeed. I drew this kind of out.
If we have a graph of quality of services that you offer and the price that you sell at, most agencies think that they're offering great quality at a little price, but the reality is you're probably down here. You're probably under-selling your services and, because of that, you can't offer the best that you can.
You should be up here. You should be offering higher quality, your experts who spend time all day studying this, and raising prices allows you to do that.
2. Schedule
Now, raising prices is only part one. The second thing is discipline, and I am really horrible about this. The reality is that I'm the kind of guy who looks for the latest and greatest and just jumps into it, but schedule matters. As hard as it is to admit it, I learned this from the CPC folks because they know that they have to stay on top of it every day of the week.
Well, here's something that we kind of came up with as I was leaving the company, and that was to set all of our customers as much as possible into a schedule.
Annually: we would handle keywords and competitors doing complete analysis.
Semi-annually: Twice a year, we would do content analysis. What should you be writing about? What's changed in your industry? What are different keywords that you might be able to target now given additional resources?
Quarterly: You need to be looking at links. It's just a big enough issue that you've got to look at it every couple of months, a complete link analysis.
Monthly: You should be looking at your crawls. Moz will do that every week for you, but you should give your customers an idea, over the course of a month, what's changed.
Weekly: You should be doing rankings
But there are three things that, when you do all of these types of analysis, you need to keep in mind. Each one of them is a...
Report
Hours for consulting
Phone call
This might seem like a little bit of overkill. But of course, if one of these comes back and nothing changed, you don't need to do the phone call, but each one of these represents additional money in your pocket and importantly better service for your customers.
It might seem hard to believe that when you go to a customer and you tell them, "Look, nothing's changed," that you're actually giving them value, but the truth is that if you go to the dentist and he tells you, you don't have a cavity, that's good news. You shouldn't say to yourself at the end of the day, "Why'd I go to the dentist in the first place?" You should say, "I'm so glad I went to the dentist." By that same positive outlook, you should be selling to your customers over and over and over again, hoping to give them the clarity they need to succeed.
3. Tool up!
So number three, you're going to see this a lot in my videos because I just love SEO tools, but you've got to tool up. Once you've raised prices and you're making more money with your customers, you actually can. Tools are superpowers. Tools allow you to do things that humans just can't do. Like I can't figure out the link graph on my own. I need tools to do it. But tools can do so much more than just auditing existing clients. For example, they can give you...
Better leads:
You can use tools to find opportunities.Take for example the tools within Moz and you want to find other car dealerships in the area that are really good and have an opportunity to rank, but aren't doing as well as they should be in SERPs. You want to do this because you've already serviced successfully a different car dealership. Well, tools like Moz can do that. You don't just have to use Moz to help your clients. You can use them to help yourself.
Better pre-audits:
Nobody walks into a sales call blind. You know who the website is. So you just start with a great pre-audit.
Faster workflows:
Which means you make more money quicker. If you can do your keyword analysis annually in half the time because you have the right tool for it, then you're going to make far more money and be able to serve more customers.
Bulk pricing:
This one is just mind-blowingly simple. It's bulk pricing. Every tool out there, the more you buy from them, the lower the price is. I remember at my old company sitting down at one point and recognizing that every customer that came in the door would need to spend about $1,000 on individual accounts to match what they were getting through us by being able to take advantage of the bulk discounts that we were getting as an agency by buying these seats on behalf of all of our customers.
So tell your clients when you're talking to them on the phone, in the pitch be like, "Look, we use Moz, Majestic, Ahrefs, SEMrush," list off all of the competitors. "We do Screaming Frog." Just name them all and say, "If you wanted to go out and just get the data yourself from these tools, it would cost you more than we're actually charging you." The tools can sell themselves. You are saving them money.
4. Just say NO
Now, the last section, real quickly, are the things you've just got to learn to say no to. One of them has a little nuance to it. There's going to be some bite back in the comments, I'm pretty sure, but I want to be careful with it.
No month-to-month contracts
The first thing to say no to is month-to-month contracts.
If a customer comes to you and they say, "Look, we want to do SEO, but we want to be able to cancel every 30 days." the reality is this. They're not interested in investing in SEO. They're interested in dabbling in SEO. They're interested in experimenting with SEO. Well, that's not going to succeed. It's only going to take one competitor or two who actually invest in it to beat them out, and when they beat them out, you're going to look bad and they're going to cancel their account with you. So sit down with them and explain to them that it is a long-term strategy and it's just not worth it to your company to bring on customers who aren't interested in investing in SEO. Say it politely, but just turn it away.
Don't turn anything away
Now, notice that my next thing is don't turn anything away. So here's something careful. Here's the nuance. It's really important to learn to fire clients who are bad for your business, where you're losing money on them or they're just impolite, but that doesn't mean you have to turn them away. You just need to turn them in the right direction. That right direction might be tools themselves. You can say, "Look, you don't really need our consulting hours. You should go use these tools." Or you can turn them to other fledgling businesses, friends you have in the industry who might be struggling at this time.
I'll tell you a quick example. We don't have much time, but many, many years ago, we had a client that came to us. At our old company, we had a couple of rules about who we would work with. We chose not to work in the adult industry. But at the time, I had a friend in the industry. He lived outside of the United States, and he had fallen on hard times. He literally had his business taken away from him via a series of just really unscrupulous events. I picked up the phone and gave him a call. I didn't turn away the customer. I turned them over to this individual.
That very next year, he had ended up landing a new job at the top of one of the largest gambling organizations in the world. Well, frankly, they weren't on our list of people we couldn't work with. We landed the largest contract in the history of our company at that time, and it set our company straight for an entire year. It was just because instead of turning away the client, we turned them to a different direction. So you've got to say no to turning away everybody. They are opportunities. They might not be your opportunity, but they're someone's.
No service creep
The last one is service creep. Oh, man, this one is hard. A customer comes up to you and they list off three things that you offer that they want, and then they say, "Oh, yeah, we need social media management." Somebody else comes up to you, three things you want to offer, and they say, "Oh yeah, we need you to write content," and that's not something you do. You've just got to not do that. You've got to learn to shave off services that you can't offer. Instead, turn them over to people who can do them and do them very well.
What you're going to end up doing in your conversation, your sales pitch is, "Look, I'm going to be honest with you. We are great at some things, but this isn't our cup of tea. We know someone who's really great at it." That honesty, that candidness is just going to give them such a better relationship with you, and it's going to build a stronger relationship with those other specialty companies who are going to send business your way. So it's really important to learn to say no to say no service creep.
Well, anyway, there's a lot that we went over there. I hope it wasn't too much too fast, but hopefully we can talk more about it in the comments. I look forward to seeing you there. Thanks.
You'll uncover even more SEO goodness in the MozCon 2020 video bundle. At this year's special low price of $129, this is invaluable content you can access again and again throughout the year to inspire and ignite your SEO strategy:
21 full-length videos from some of the brightest minds in digital marketing
Instant downloads and streaming to your computer, tablet, or mobile device
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
As digital PRs we can often get stuck with our "campaign goggles" on, especially in the ideation and production stage of a creative campaign.
By this I mean, you have a preconceived idea of where you'd like your campaign to be featured, what kind of headlines you want it to achieve, and how people should read your data and story.
As we all know, we can't control the outcomes of a campaign, but we can certainly push them in the right direction.
To give your link building campaigns the best chance in the outreach stage, you need to make sure there is enough creative diversification during the production process, especially for data-led pieces and surveys. This opens up your "journalist pool" and gives you a ton more people to outreach to with a potential interest in your piece.
What is creative diversification?
Creative diversification is how you minimize the amount of risk in your link building campaign by ensuring your idea has enough breadth during the production process. It doesn't matter what format you’re using for each campaign — you always need to confirm it’s diverse enough to stand up in a changing news landscape. You want to develop an idea that can naturally explore multiple angles and sectors in the outreach phase. This flexibility needs to be set up before production, by exploring the potential outcomes and headlines you’re going after before you have them.
Find related topics
In the production stage, we obviously need to focus on our fundamental topic. This is often the domain's main reason for being. It could be finance, travel, fashion — you get the picture.
Then you want to start branching out and overlaying topics: finance + students, travel + safety, fashion + Elon Musk, and so on. You’re attempting to grab subtopics of interest.
Every link builder will have a different approach to discovering these topics, but the simplest way to get started is to grab a piece of paper and start scribbling ideas by word association. Just write as much as you can and you’ll find there’s lots of closely-related topic areas your content could delve into. (Tools like BuzzSumo would be invaluable here, but if you’re after a free alternative, I have been enjoying playing with AlsoAsked.com lately for related topic inspiration. Nothing is going to beat existing news content, though.)
It's also crucial to think about topic relevance, because if you question a tenuous link between your domain and topic matter, you can be certain journalists will, too. Link relevance is a whole other conversation to be had, but as long as it aligns with your client’s goals and you’re happy with showing them the link/coverage in full, you can’t go far wrong.
As a team at Root, we scrutinize our data points and approaches a lot in the production phase of each campaign and we find that championing personal expertise and curiosity often leads to some interesting statistics. My own passion for veganism gave us a unique angle which proved fruitful when we went out with a third round of outreach for our recent COVID-19 spending campaign.
Take off your campaign goggles
If the idea for your new campaign was born from your mind, you’re emotionally and personally invested whether you like it or not. You’ll need to put these feelings aside to engage with as many potential angles as possible from the start.
When I say you need to take off your campaign goggles, you need to (preferably with a colleague) tear apart the campaign and think about where you can add further value. It's best to approach this objectively, so if you can tackle a colleague’s campaign and vice-versa, even better.
Some link builders will look at their angles and opportunities only once the content has been created and consider it an outreach decision. Success is definitely possible this way, but you’re stopping yourself from being as successful as you might have been had you thoroughly drilled into your content before and during the production process.
Highlight the key areas and approaches you'd like to tackle beforehand and you can feed this into your outreach strategy later on.
Make sector-specific data for journalists
When creating media lists and discovering relevant journalists, link builders can often be encouraged to rush through and ignore the content itself. If you know what they’re writing about, both on Twitter and in publications, you can begin to think about what data you could craft specifically for them.
In the campaign I mention in this blog, we focused on side-hustle data related to the fundamental topic of how people are earning their money during the pandemic, which was directly influenced by journalists.
The journalist who covered this specific topic in USA Today fortunately tweeted a lot about the stories he was working on, so it made it incredibly easy for us to tailor some content toward his interest and later offer him the type of unique data he wanted.
Aside from keeping tabs on Twitter, you can also find out what they’re interested in through Google Discover and Reddit to understand what’s being talked about and what is topical.
I know many digital PRs review key publications directly on a regular basis and have big Feedly feeds or watch insight roundups on YouTube instead. Either way, thinking about what a journalist will need in the next few weeks is imperative to early planning and ensuring your campaign is diverse enough from the get-go.
Diversify outreach with hash URLs
Another way you can make certain your content is diversified and prepared for a breadth of outreach is through the use of URL fragments or "hash URLs". In the case of our coronavirus spending research campaign, we used article hooks on the page to provide anchor links from the table of contents at the top which then allowed us to offer another layer of personalization.
The key findings or headlines section in a table of contents is an essential piece to any long-form data campaign and makes it incredibly easy for journalists and readers to find the most relevant statistics to them in literally seconds.
If you’ve never implemented this yourself, there’s a simpler way than hooks — you just need to know your HTML basics. (Please excuse me if I butcher this description as a non-dev!) Place id="#subject" within the heading tag, so it would look like: <h2 id="#subject">.
In the example below, a BBC journalist used the URL with "#vegetarian" when referencing our statistics about plant-based food usage. This came from the ID tag and meant the journalist could link directly to the bits of research that was relevant to their article.
On top of that, we could send journalists semi-personalized links in our outreach, too. It’s a win-win and is best practice for users and search engine crawlers to navigate your long-form content anyway.
This is a literal manifestation of your creative diversification process early on, as it’s now been produced and each hash URL is an extra asset pointing journalists to the most relevant data for them.
Creative diversification in action
The campaign I’ve mentioned in this piece was a lengthy, yet simple, survey campaign for a fin-tech client asking Americans about their spending habits during the pandemic. We secured a range of coverage, but the three biggest placements we landed (BBC, CNBC, and USA Today) all covered different angles and data points from each other, but from this one survey, and that wasn’t an accident.
In the production stage, we knew we needed to focus on the campaign fundamentals: spending during the pandemic. Our related topics led us to grocery store spending and another leap encouraged us to look at food choices (were American’s eating more veg during lockdown? Hmm). These topics were still closely related to our core focus (finances) and therefore useful for our outreach in terms of securing relevant and high quality links.
When it came to the outreach strategy, we prioritized landing placements tied directly to the campaign fundamentals, then the related topics fed into the consecutive rounds which we chose depending on the strength of the data we received from the survey.
If you’re thinking in the production process that there’s too much going on with too many angles, you may have just created multiple mini content campaigns for yourself.
We’ve found time and time again that the simpler stories and slimmer, more targeted outreach emails will land placements way more often than bloated emails trying to offer up far too much content at once.
That’s not to say that you should automatically split up larger pieces of content, but your outreach should be the final step in diversifying your piece. A data analysis research piece that taps into multiple sectors should simply highlight the most relevant information to the journalist in bite-size sections. We gave grocery spend data to retail business journalists, vegan food consumption data to food writers, and side-hustle data to those writing on the latest employment trends.
The next time you’re creating a content campaign, have your team (even if that’s just you) ruthlessly find new sectors, journalists, and angles to target, to ensure your next piece is as diverse as possible. Creative diversification = more hooks and less risk.
To help us serve you better, please consider taking the 2020 Moz Blog Reader Survey, which asks about who you are, what challenges you face, and what you'd like to see more of on the Moz Blog.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Earlier this year (15 years after starting Distilled in 2005), we spun out a new company called SearchPilot to focus on our SEO A/B testing and meta-CMS technology (previously known as Distilled ODN), and merged the consulting and conferences part of the business with Brainlabs.
I’m now CEO of SearchPilot (which is primarily owned by the shareholders of Distilled), and am also SEO Partner at Brainlabs, so… I’m sorry everyone, but I’m very much staying in the SEO industry.
As such, it feels a bit like the end of a chapter for me rather than the end of the book, but it has still had me looking back over what’s changed and what hasn’t over the last 15 years I’ve been in the industry.
I can’t lay claim to being one of the first generation of SEO experts, but having been building websites since around 1996 and having seen the growth of Google from the beginning, I feel like maybe I’m second generation, and maybe I have some interesting stories to share with those who are newer to the game.
I’ve racked my brain to try and remember what felt significant at the time, and also looked back over the big trends through my time in the industry, to put together what I think makes an interesting reading list that most people working on the web today would do well to know about.
The big eras of search
I joked at the beginning of a presentation I gave in 2018 that the big eras of search oscillated between directives from the search engines and search engines rapidly backing away from those directives when they saw what webmasters actually did:
While that slide was a bit tongue-in-cheek, I do think that there’s something to thinking about the eras like:
Build websites: Do you have a website? Would you like a website? It’s hard to believe now, but in the early days of the web, a lot of folks needed to be persuaded to get their business online at all.
Keywords: Basic information retrieval became adversarial information retrieval as webmasters realized that they could game the system with keyword stuffing, hidden text, and more.
Links: As the scale of the web grew beyond user-curated directories, link-based algorithms for search began to dominate.
Not those links: Link-based algorithms began to give way to adversarial link-based algorithms as webmasters swapped, bought, and manipulated links across the web graph.
Content for the long tail: Alongside this era, the length of the long tail began to be better-understood by both webmasters and by Google themselves — and it was in the interest of both parties to create massive amounts of (often obscure) content and get it indexed for when it was needed.
Not that content: Perhaps predictably (see the trend here?), the average quality of content returned in search results dropped dramatically, and so we see the first machine learning ranking factors in the form of attempts to assess “quality” (alongside relevance and website authority).
Machine learning: Arguably everything from that point onwards has been an adventure into machine learning and artificial intelligence, and has also taken place during the careers of most marketers working in SEO today. So, while I love writing about that stuff, I’ll return to it another day.
History of SEO: crucial moments
Although I’m sure that there are interesting stories to be told about the pre-Google era of SEO, I’m not the right person to tell them (if you have a great resource, please do drop it in the comments), so let’s start early in the Google journey:
Google’s foundational technology
Even if you’re coming into SEO in 2020, in a world of machine-learned ranking factors, I’d still recommend going back and reading the surprisingly accessible early academic work:
If you weren’t using the web back then, it’s probably hard to imagine what a step-change improvement Google’s PageRank-based algorithm was over the “state-of-the-art” at the time (and it’s hard to remember, even for those of us that were):
Google’s IPO
In more “things that are hard to remember clearly,” at the time of Google’s IPO in 2004, very few people expected Google to become one of the most profitable companies ever. In the early days, the founders had talked of their disdain for advertising, and had experimented with keyword-based adverts somewhat reluctantly. Because of this attitude, even within the company, most employees didn’t know what a rocket ship they were building.
From this era, I’d recommend reading the founders’ IPO letter (see this great article from Danny Sullivan — who’s ironically now @SearchLiaison at Google):
“Our search results are the best we know how to produce. They are unbiased and objective, and we do not accept payment for them or for inclusion or more frequent updating.”
“Because we do not charge merchants for inclusion in Froogle [now Google shopping], our users can browse product categories or conduct product searches with confidence that the results we provide are relevant and unbiased.” — S1 Filing
In addition, In the Plex is an enjoyable book published in 2011 by Steven Levy. It tells the story of what then-CEO Eric Schmidt called (around the time of the IPO) “the hiding strategy”:
“Those who knew the secret … were instructed quite firmly to keep their mouths shut about it.”
“What Google was hiding was how it had cracked the code to making money on the Internet.”
Luckily for Google, for users, and even for organic search marketers, it turned out that this wasn’t actually incompatible with their pure ideals from the pre-IPO days because, as Levy recounts, “in repeated tests, searchers were happier with pages with ads than those where they were suppressed”. Phew!
Applied Semantics technology was integrated with their own contextual ad technology to form what became AdSense. Although the revenue from AdSense has always been dwarfed by AdWords (now just “Google Ads”), its importance in the history of SEO is hard to understate.
By democratizing the monetization of content on the web and enabling everyone to get paid for producing obscure content, it funded the creation of absurd amounts of that content.
Most of this content would have never been seen if it weren’t for the existence of a search engine that excelled in its ability to deliver great results for long tail searches, even if those searches were incredibly infrequent or had never been seen before.
In this way, Google’s search engine (and search advertising business) formed a powerful flywheel with its AdSense business, enabling the funding of the content creation it needed to differentiate itself with the largest and most complete index of the web.
As with so many chapters in the story, though, it also created a monster in the form of low quality or even auto-generated content that would ultimately lead to PR crises and massive efforts to fix.
The first forms of spam on the internet were various forms of messages, which hit the mainstream as email spam. During the early 2000s, Google started talking about the problem they’d ultimately term “web spam” (the earliest mention I’ve seen of link spam is in an Amit Singhal presentation from 2005 entitled Challenges in running a Commercial Web Search Engine [PDF]).
I suspect that even people who start in SEO today might’ve heard of Matt Cutts — the first head of webspam — as he’s still referenced often despite not having worked at Google since 2014. I enjoyed this 2015 presentation that talks about his career trajectory at Google.
Search quality era
Over time, as a result of the opposing nature of webmasters trying to make money versus Google (and others) trying to make the best search engine they could, pure web spam wasn’t the only quality problem Google was facing. The cat-and-mouse game of spotting manipulation — particularly of on-page content, external links, and anchor text) — would be a defining feature of the next decade-plus of search.
It was after Singhal’s presentation above that Eric Schmidt (then Google’s CEO) said, “Brands are the solution, not the problem… Brands are how you sort out the cesspool”.
Those who are newer to the industry will likely have experienced some Google updates (such as recent “core updates”) first-hand, and have quite likely heard of a few specific older updates. But “Vince”, which came after “Florida” (the first major confirmed Google update), and rolled out shortly after Schmidt’s pronouncements on brand, was a particularly notable one for favoring big brands. If you haven’t followed all the history, you can read up on key past updates here:
A real reputational threat
As I mentioned above in the AdSense section, there were strong incentives for webmasters to create tons of content, thus targeting the blossoming long tail of search. If you had a strong enough domain, Google would crawl and index immense numbers of pages, and for obscure enough queries, any matching content would potentially rank. This triggered the rapid growth of so-called “content farms” that mined keyword data from anywhere they could, and spun out low-quality keyword-matching content. At the same time, websites were succeeding by allowing large databases of content to get indexed even as very thin pages, or by allowing huge numbers of pages of user-generated content to get indexed.
This was a real reputational threat to Google, and broke out of the search and SEO echo chamber. It had become such a bugbear of communities like Hacker News and StackOverflow, that Matt Cutts submitted a personal update to the Hacker News community when Google launched an update targeted at fixing one specific symptom — namely that scraper websites were routinely outranking the original content they were copying.
Shortly afterwards, Google rolled out the update initially named the “farmer update”. After it launched, we learned it had been made possible because of a breakthrough by an engineer called Panda, hence it was called the “big Panda” update internally at Google, and since then the SEO community has mainly called it the Panda update.
I gave a presentation at SearchLove London in 2014 where I talked about the unbelievable growth and scale of mobile and about how late we were to realizing quite how seriously Google was taking this. I highlighted the surprise many felt hearing that Google was designing mobile first:
“Towards the end of last year we launched some pretty big design improvements for search on mobile and tablet devices. Today we’ve carried over several of those changes to the desktop experience.” — Jon Wiley (lead engineer for Google Search speaking on Google+, which means there’s nowhere to link to as a perfect reference for the quote but it’s referenced here as well as in my presentation).
This surprise came despite the fact that, by the time I gave this presentation in 2014, we knew that mobile search had begun to cannibalize desktop search (and we’d seen the first drop in desktop search volumes):
And it came even though people were starting to say that the first year of Google making the majority of its revenue on mobile was less than two years away:
Writing this in 2020, it feels as though we have fully internalized how big a deal mobile is, but it’s interesting to remember that it took a while for it to sink in.
Machine learning becomes the norm
Since the Panda update, machine learning was mentioned more and more in the official communications from Google about algorithm updates, and it was implicated in even more. We know that, historically, there had been resistance from some quarters (including from Singhal) towards using machine learning in the core algorithm due to the way it prevented human engineers from explaining the results. In 2015, Sundar Pichai took over as CEO, moved Singhal aside (though this may have been for other reasons), and installed AI / ML fans in key roles.
It goes full-circle
Back before the Florida update (in fact, until Google rolled out an update they called Fritz in the summer of 2003), search results used to shuffle regularly in a process nicknamed the Google Dance:
Most things have been moving more real-time ever since, but recent “Core Updates” appear to have brought back this kind of dynamic where changes happen on Google’s schedule rather than based on the timelines of website changes. I’ve speculated that this is because “core updates” are really Google retraining a massive deep learning model that is very customized to the shape of the web at the time. Whatever the cause, our experience working with a wide range of clients is consistent with the official line from Google that:
Broad core updates tend to happen every few months. Content that was impacted by one might not recover — assuming improvements have been made — until the next broad core update is released.
Tying recent trends and discoveries like this back to ancient history like the Google Dance is just one of the ways in which knowing the history of SEO is “useful”.
If you’re interested in all this
I hope this journey through my memories has been interesting. For those of you who also worked in the industry through these years, what did I miss? What are the really big milestones you remember? Drop them in the comments below or hit me up on Twitter.
If you liked this walk down memory lane, you might also like my presentation From the Horse’s Mouth, where I attempt to use official and unofficial Google statements to unpack what is really going on behind the scenes, and try to give some tips for doing the same yourself:
To help us serve you better, please consider taking the 2020 Moz Blog Reader Survey, which asks about who you are, what challenges you face, and what you'd like to see more of on the Moz Blog.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
