MozCon 2019 was an absolute blast. There were endless snacks. There were Roger hugs. There were networking opportunities and Birds of a Feather tables and search epiphanies galore. And there were a ton of folks in our community who watched it all unfold from the perspective of a Twitter hashtag — fun to follow along with, but not quite the same impact as seeing the talks unfold in real-time.
If you're still wishing you could've joined us in Seattle this past July, you’ll be happy to know that you can recreate the MozCon experience from the comfort of your home or office (or your home office, but hopefully not your office-home — seriously, Karen, the quarterly reports will still be there in the morning!).
Yep, you got it: the MozCon 2019 Video Bundle is available for your purchasing and viewing pleasure!
For those of you who attended in-person, good news: you've already got access! The video bundle is always included in the price of your MozCon ticket, so you can relive your three jam-packed days of learning as many times as you want — and if you aren't too bummed that they already made you share your MozCon swag with them, be sure to share the vids with your team!
For the rest of us, the video bundle lets us enjoy the presentations at our own pace. It's condensed MozCon-caliber information in a neat, on-demand package that you can — have we mentioned this? — share with your team. Seriously, we think they'll like it. We were humbled to host some of the very brightest minds in SEO and digital marketing on our stage. With topics ranging from content marketing to technical SEO, PPC to local SEO, and just about everything in between, there are presentations to inspire just about any role in marketing (and your web dev just might be interested in a few talks, too).
What's covered in the videos:
The Golden Age of Search, Sarah Bird
Web Search 2019: The Essential Data Marketers Need, Rand Fishkin
Human > Machine > Human: Understanding Human-Readable Quality Signals and Their Machine-Readable Equivalents, Ruth Burr Reedy
Improved Reporting & Analytics Within Google Tools, Dana DiTomaso
Local Market Analytics: The Challenges and Opportunities, Rob Bucci
Keywords Aren't Enough: How to Uncover Content Ideas Worth Chasing, Ross Simmonds
How to Supercharge Link Building with a Digital PR Newsroom, Shannon McGuirk
From Zero to Local Ranking Hero, Darren Shaw
Esse Quam Videri: When Faking it is Harder than Making It, Russ Jones
Building a Discoverability Powerhouse: Lessons From Merging an Organic, Paid, & Content Practice, Heather Physioc
Brand Is King: How to Rule in the New Era of Local Search, Mary Bowling
Making Memories: Creating Content People Remember, Casie Gillette
20 Years in Search & I Don't Trust My Gut or Google, Wil Reynolds
Super-Practical Tips for Improving Your Site's E-A-T, Marie Haynes
Fixing the Indexability Challenge: A Data-Based Framework, Areej AbuAli
What Voice Means for Search Marketers: Top Findings from the 2019 Report, Christi Olson
Redefining Technical SEO, Paul Shapiro
How Many Words Is a Question Worth?, Dr. Peter J. Meyers
Fraggles, Mobile-First Indexing, & the SERP of the Future, Cindy Krum
Killer E-commerce CRO and UX Wins Using A SEO Crawler, Luke Carthy
Content, Rankings, and Lead Generation: A Breakdown of the 1% Content Strategy, Andy Crestodina
Running Your Own SEO Tests: Why It Matters & How to Do It Right, Rob Ousbey
Dark Helmet's Guide to Local Domination with Google Posts and Q&A, Greg Gifford
How to Audit for Inclusive Content, Emily Triplett Lentz
Factors that Affect the Local Algorithm that Don't Impact Organic, Joy Hawkins
Featured Snippets: Essentials to Know & How to Target, Britney Muller
What you’ll get:
For just $299, you'll get all of the MozCon education and inspiration with none of the air travel or traffic. The bundle includes:
27 full-length presentation videos chock full of leading SEO innovations, thought leadership, and tips & tricks
Instant downloads and streaming to your computer, tablet, or mobile device
Downloadable slide decks for all presentations
If we could include a download of a Top Pot doughnut and some piping hot Starbucks, we would in a heartbeat. Alas, they don't have the technology for that... yet.
Free preview - Running Your Own SEO Tests: Why It Matters & How to Do It Right by Rob Ousbey
Speaking of doughnuts, we wouldn't expect you to buy a dozen sweet treats without taking a little taste first to see if you like 'em. It's important to know that your doughnuts are both delicious, shareable, and relevant to your everyday work as an SEO — almost exactly like the MozCon video bundle. And just like the feeling of warmth and goodwill you receive when you come back to the office with a fragrant baker's dozen, your teammates will thank you when you've got twenty-seven highly actionable talks to share with them — presentations that'll hone your skills and level up your understanding of modern SEO and digital marketing.
That's why we've released a talk we're super proud of as your free preview of all the juicy goodness you can look forward to in the video bundle: Running Your Own SEO Tests: Why It Matters & How to Do It Right, presentedby our very own Rob Ousbey.
Google's algorithms have undergone significant changes in recent years. Traditional ranking signals don't hold the same sway they used to, and they're being usurped by factors like UX and brand that are becoming more important than ever before. What's an SEO to do? The answer lies in testing. Sharing original data and results from clients, Rob highlights the necessity of testing, learning, and iterating your work, from traditional UX testing to weighing the impact of technical SEO changes, tweaking on-page elements, and changing up content on key pages. Actionable processes and real-world results abound in this thoughtful presentation on why you should be testing SEO changes, how and where to run them, and what kinds of tests you ought to consider for your circumstances.
Gather the team, grab some snacks, and get ready to binge these presentations Netflix-Original-Series-style.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Can franchises make good digital marketing agency clients? There are almost 750,000 of them in the US alone, employing some 9 million Americans. Chances are good you’ll have the opportunity to market a business with this specialized model at some point. In this structure:
The Franchisor grants permission to others to operate under its trademark, selling approved goods and services supported by an operating system and marketing.
The Franchisee is the person or group paying the franchisor for the right to use the trademark and the benefits of the operating system and marketing.
Seems simple enough. But it’s this structure that gives franchise marketing its unique complexities. For your agency, the challenge is that you can’t enter these marketing relationships equipped solely with your knowledge of corporate or local search marketing.
You need to deeply understand the setup to avoid bewilderment over why implementation bogs down with franchise clients and why players lose track of their roles, or even overwrite one another’s efforts.
In this post, we’ll give you some quick and useful coaching on the franchise model, but if your agency just got a phone call from Orangetheory or Smoothie King, you can get the bigger playbook right away.
Imagine a post-game locker room scene. On the field, all players seemed united by the goal of winning. But now, at different press conferences, the owner is saying the coach failed to meet standards, the coach is saying the owner should keep his opinions to himself, and several of the star players are saying they didn’t get the ball enough.
Franchises can be just like that when there’s confusion over roles and goals. Read on to get a peek into the playbook we've prepared to help the team as a whole work better together:
Franchise marketing is a unique kind of activity. It does share a lot of qualities with corporate marketing (on the awareness side) and with SMB marketing (on the local side) but as we noted earlier, it’s sort of a joint custody arrangement that — like all custody arrangements — can get contentious at times.
Everyone wants the best for the brand, but everyone’s “best” is very much a matter of their own perspective and goals. Typically in this arrangement, there are at least two stakeholders, though sometimes there are more. The stakeholders and their goals tend to play out as follows:
Corporate Franchisor goals
Creating a strong brand to license more franchisors.
Controlling that brand so it isn’t negatively impacted.
Supporting franchisees with strong branding and resources so they succeed.
Master Franchisor goals
Working with corporate to protect the brand.
Licensing more local franchisors.
Supporting franchisees with resources so they succeed.
Regional or Area Franchisee goals
Driving customer traffic and revenue at individual locations.
Growing their portfolio of locations.
Supporting location managers with resources so they succeed.
Owner/Operator Franchisee goals
Increasing location(s) foot traffic.
Increasing location(s) revenue.
Building customer loyalty at the location(s).
In what ways is franchise marketing different from corporate or standard SMB marketing? There are some unique challenges that franchisors and franchisees face which are worth unpacking. Some of them are:
Conflicting goals between franchisor/franchisee
Faster turnover of locations and addresses
Different opening hours, menus and promotions from location to location
Unique local sales and marketing opportunities and challenges
Competitors on both the brand side but also among local SMBs
Lack of clearly defined marketing roles causing work to be overwritten, duplicated, or even neglected
Your agency can be a better coach to franchises by having a playbook that respects how they differ from corporate or SMB clients at the very outset. But differences don’t have to equal weaknesses. Are you ready to draft a game plan that draws from the strengths of both franchisors and franchisees?
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
We couldn’t do it without you! In 2018, over 1,400 marketers responded to our State of Local SEO industry survey. We all learned so much from your responses about the day-to-day realities of marketing local businesses. This year, we can do even better because your answers will give us all valuable comparative data to analyze, YoY.
Who can take the survey?
Anyone who markets local businesses in any way is eagerly invited. Whether you market a single location, work for an agency with some local business clients, or are an in-house SEO for a brand with thousands of locations, we would love your participation! Whether you do just a little local search marketing or a lot, are a novice or an adept, your insights have value.
What is the survey about?
Unlike a typical local ranking factors poll, The Local Search Marketing Industry Survey digs deep into marketers’ experiences with tactics, challenges, clients, Google, and the working environment. For example, we learned last year that:
90% of respondents felt Google’s emphasis on proximity was detrimental to SERP quality
62% felt there aren’t enough quality local search marketing training materials available
60% lacked a comprehensive review management strategy
49% felt utilization of Google Business Profile features were impacting local rank
35% had no link building strategy in place
17% of enterprises had no in-house SEO staff
With your help, we’ll see what’s changed and what hasn’t. There are fresh questions, too, which we hope will uncover new stories to spark new strategies for local brands and their marketers.
There will be four lucky winners!
Everyone is a winner with access to the data we’ll be sharing from this large survey. But we’d like to offer a little extra thank-you for your time and knowledge.
Every respondent who completes the full survey will be automatically entered for a chance to win one of four $50 Visa gift cards. Winners will be selected at random, and we hope they will use these gift cards to shop someplace local and awesome this holiday season!
Look forward to seeing the results in early 2020, when we compile them into our State of Local SEO 2020 Industry Report. Curious about last year's insights? Check them out here, and thank you for participating!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Earlier this week, I hosted a webinar all about featured snippets covering essential background info, brand-new research we've done, the results of all the tests I've performed, and key takeaways. Things didn't quite go as planned, though. We had technical difficulties that interfered with our ability to broadcast live, and lots of folks were left with questions after the recording that we weren't able to answer in a follow-up Q&A.
The next best thing to a live webinar Q&A? A digital one that you can bookmark and come back to over and over again! We asked our incredibly patient, phenomenally smart attendees to submit their questions via email and promised to answer them in an upcoming blog post. We've pulled out the top recurring questions and themes from those submissions and addressed them below. If you had a question and missed the submission window, don't worry!Ask it down in the comments and we'll keep the conversation going.
If you didn't get a chance to sign up for the original webinar, you can register for it on-demand here:
And if you're here to grab the free featured snippets cheat sheet we put together, look no further — download the PDF directly here. Print it off, tape it to your office wall, and keep featured snippets top-of-mind as you create and optimize your site content.
Now, let's get to those juicy questions!
1. Can I win a featured snippet with a brand-new website?
If you rank on page one for a keyword that triggers a featured snippet (in positions 1–10), you're a contender for stealing that featured snippet. It might be tougher with a new website, but you're in a position to be competitive if you're on page one — regardless of how established your site is.
We've got some great Whiteboard Fridays that cover how to set a new site up for success:
2. Does Google provide a tag that identifies traffic sources from featured snippets? Is there a GTM tag for this?
Unfortunately, Google does not provide a tag to help identify traffic from featured snippets. I'm not aware of a GTM tag that helps with this, either, but would love to hear any community suggestions or ideas in the comments!
It's worth noting that it's currently impossible to determine what percentage of your traffic comes from the featured snippet versus the duplicate organic URL below the featured snippet.
3. Do you think it's worth targeting longer-tail question-based queries that have very low monthly searches to gain a featured snippet?
Great question! My advice is this: don’t sleep on low-search-volume keywords. They often convert really well and in aggregate they can do wonders for a website. I suggest prioritizing long tail keywords that you foresee providing a high potential ROI.
For example, there are millions of searches a month for the keyword “shoes.” Very competitive, but that query is pretty vague. In contrast, the keyword “size 6 red womens nike running shoes” is very specific. This searcher knows what they want and they're dialing in their search to find it. This is a great example of a long tail keyword phrase that could provide direct conversions.
4. What's the best keyword strategy for determining which queries are worth creating featured snippet-optimized content for?
Dr. Pete wrote a great blog post outlining how to perform keyword research for featured snippets back in 2016. Once you've narrowed down your list of likely queries, you need to look at keywords that you rank on page one for, that trigger a snippet, and that you don't yet own. Next, narrow your list down further by what you envision will have the highest ROI for your goals. Are you trying to drive conversions? Attract top-of-funnel site visitors? Make sure the queries you target align with your business goals, and go from there. Both Moz Pro and STAT can be a big help with this process.
A tactical pro tip: Use the featured snippet carousel queries as a starting point. For instance, if there's a snippet for the query "car insurance" with a carousel of "in Florida," "in Michigan," and so on, you might consider writing about state-specific topics to win those carousel snippets. For this technique, the bonus is that you don't really need to be on page one for the root term (or ranking at all) — often, carousel snippets are taken from off-SERP links.
5. Do featured snippets fluctuate according to language, i.e. if I have several versions of my site in different languages, will the snippet display for each version?
This is a great question! Unfortunately, we haven’t been able to do international/multi-language featured snippet research just yet, but hope to in the future. I would suspect the featured snippet could change depending on language and search variation. The best way to explore this is to do a search in an incognito (and un-logged-in) browser window of Google Chrome.
If you've performed research along these lines, let us know what you found out down in the comments!
6. Why do featured snippet opportunities fluctuate in number from day to day?
Change really is the only constant in search. In the webinar, I discussed the various tests I did that caused Moz to lose a formerly won featured snippet (and what helped it reappear once again). Changes as simple as an extra period at the end of a sentence were enough to lose us the snippet. With content across the web constantly being created and edited and deprecated and in its own state of change, it's no wonder that it's tough to win and keep a featured snippet — sometimes even from one day to the next.
The SERPs are incredibly volatile things, with Google making updates multiple times every day. But when it comes down to the facts, there are a few things that reliably cause volatility (is that an oxymoron?):
If a snippet is pulling from a lower-ranking URL (not positions 1–3); this could mean Google is testing the best answer for the query
Google regularly changing which scraped content is used in each snippet
Featured snippet carousel topics changing
The best way to change-proof yourself is to become an authority in your particular niche (E-A-T, remember?) and strive to rank higher to increase your chances of capturing and keeping a featured snippet.
7. How can I use Keyword Lists to find missed SERP feature opportunities? What's the best way to use them to identify keyword gaps?
Keyword Lists are a wonderful area to uncover feature snippet (and other SERP feature) opportunity gaps. My favorite way to do this is to filter the Keyword List by your desired SERP feature. We’ll use featured snippets as an example. Next, sort by your website’s current rank (1–10) to determine your primary featured snippet gaps and opportunities.
The filters are another great way to tease out additional gaps:
Which keywords have high search volume and low competition?
Which keywords have high organic CTR that you currently rank just off page one for?
8. What are best practices around reviewing the structure of content that's won a snippet, and how do I know whether it's worth replicating?
Content that has won a featured snippet is definitely worth reviewing (even if it doesn’t hold the featured snippet over time). Consider why Google might have provided this as a featured snippet:
Does it succinctly answer the query?
Might it sound good as a voice answer?
Is it comprehensive for someone looking for additional information?
Does the page provide additional answers or information around the topic?
Are there visual elements?
It’s best to put on your detective hat and try to uncover why a piece of content might be ranking for a particular featured snippet:
What part of the page is Google pulling that featured snippet content from?
Is it marked up in a certain way?
What other elements are on the page?
Is there a common theme?
What additional value can you glean from the ranking featured snippet?
9. Does Google identify and prioritize informational websites for featured snippets, or are they determined by a correlation between pages with useful information and frequency of snippets?
In other words, would being an e-commerce site harm your chances of winning featured snippets, all other factors being the same?
I’m not sure whether Google explicitly categorizes informational websites. They likely establish a trust metric of sorts for domains and then seek out information or content that most succinctly answers queries within their trust parameters, but this is just a hypothesis.
While informational sites tend to do overwhelmingly better than other types of websites, it’s absolutely possible for an e-commerce website to find creative ways of snagging featured snippets.
It’s fascinating how various e-commerce websites have found their way into current featured snippets in extremely savvy ways. Here's a super relevant example: after our webinar experienced issues and wasn't able to launch on time, I did a voice search for “how much do stamps cost” to determine how expensive it would be to send apology notes to all of our hopeful attendees.
This was the voice answer:
“According to stamps.com the cost of a one ounce first class mail stamp is $0.55 at the Post Office, or $.047 if you buy and print stamps online using stamps.com.”
Pretty clever, right? I believe there are plenty of savvy ways like this to get your brand and offers into featured snippets.
10. When did the "People Also Ask" feature first appear? What changes to PAAs do you anticipate in the future?
People Also Ask boxes first appeared in July 2015 as a small-scale test. Their presence in the SERPs grew over 1700% between July 2015 and March 2017, so they certainly exploded in popularity just a few years ago. Funny enough, I was one of the first SEOs to come across Google’s PAA testing — you can read about that stat and more in my original article on the subject: Infinite "People Also Ask" Boxes: Research and SEO Opportunities
When it comes to predicting the future of PAAs, well, we don't have a crystal ball yet, but featured snippets continue to look more and more like PAA boxes with their new-ish accordion format. Is it possible Google will merge them into a single feature someday? It's hard to say, but as SEOs, our best bet is to maintain flexibility and prepare to roll with the punches the search engines send our way.
11. Can you explain what you meant by "15% of image URLs are not in organic"?
Sure thing! The majority of images that show up in featured snippet boxes (or to be more accurate, the webpage those images live on) do not rank organically within the first ten pages of organic search results for the featured snippet query.
12. How should content creators consider featured snippets when crafting written content? Are there any tools that can help?
First and foremost, you'll want to consider the searcher.
What is their intent?
What desired information or content are they after?
Are you providing the desired information in the medium in which they desire it most (video, images, copy, etc)?
Look to the current SERPs to determine how you should be providing content to your users. Read all of the results on page one:
13. "Write quality content for people, not search engines" seems like great advice. But should I also be using any APIs or tools to audit my content?
The only really helpful tool that comes to mind is the Flesch-Kincaid readability test, but even that can be a bit disruptive to the creative process. The very best tool you might have for reviewing your content might be a real person. I would ensure that your content can be easily understood when read out loud to your targeted audience. It may help to consider whether your content, as a featured snippet, would make for an effective, helpful voice search result.
14. What's the best way to stay on top of trends when it comes to Google's featured snippets?
Find publications and tools that resonate, and keep an eye on them. Some of my favorites include:
Industry news publications like Search Engine Journal and, of course, the Moz Blog ;-)
Subscribing to SEO newsletters like the Moz Top 10
One of the very best things you can do, though, is performing your own investigative featured snippet research within your space. Publishing the trends you observe helps our entire community grow and learn.
Thank you so much to every attendee who submitted their questions. Digging into these follow-up thoughts and ideas is one of the best parts of putting on a presentation. If you've got any lingering questions after the webinar, I would love to hear them — leave me a note in the comments and I'll be on point to answer you. And if you missed the webinar sign-up, you can still access it on-demand whenever you want.
We also promised you some bonus content, yeah? Here it is — I compiled all of my best tips and tricks for winning featured snippets into a downloadable cheat sheet that I hope is a helpful reference for you:
Free download: The Featured Snippets Cheat Sheet
There's no reason you shouldn't be able to win your own snippets when you're armed with data, drive, and a good, solid plan! Hopefully this is a great resource for you to have on hand, either to share around with colleagues or to print out and keep at your desk:
Again, thank you so much for submitting your questions, and we'll see you in the comments for more.
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
Machine learning is only growing in importance for anyone working in the digital world, but it can often feel like an inaccessible subject. It doesn't have to be — and you don't have to miss out on the competitive edge it can give you when it comes to SEO task automation. Put on your technical SEO cap and get ready to take notes, because Britney Muller is walking us through Machine Learning 101 in this week's episode of Whiteboard Friday.
Click on the whiteboard image above to open a high-resolution version in a new tab!
Video Transcription
Hey, Moz fans. Welcome to another edition of Whiteboard Friday. Today I'm talking about all things machine learning, something, as many of you know, I'm super passionate about and love to talk about. So hopefully, this sparks a seed in some of you to explore it a bit further, because it is truly one of the most powerful things to happen in our space in a very long time.
What is machine learning?
So a brief overview, in a nutshell, machine learning is actually a subset of AI, and some would argue we still haven't really reached artificial intelligence. But it's just one facet of the overall AI.
Traditional programming
The best way to think about it is in comparison to traditional programming. So traditional programming, you input data and a program into a computer and out comes the output, whether that be a web page or calculator you built online, whatever that might be.
Machine learning
With machine learning, what you do is you put in the data and the desired output and put this into a computer, and you get a program, otherwise known as a machine learning model. So it's a bit flipped, and it works extremely well. There are two primary types of machine learning:
You have supervised, which is where you're basically feeding a model labeled training data,
And then unsupervised, which is where you're feeding a program data and letting it create clusters or associations between data points.
The supervised is a bit more common. You'll see things like classification, linear regression, and image recognition. Things like that are all very common. If you think about machine learning in terms of, okay, there's all of this data that you're putting into the model, data is the biggest part of machine learning. A lot of people would argue that if machine learning was a vehicle, data would be the fuel.
It's a really important part to understand, because unless you have the right types of data to feed a model, you're not going to get the desired outcome that you would like.
A machine learning model example
So let's look at an example. If you wanted to build a machine learning model that predicts housing prices, you might have all of this information.
You might have the current price, square foot of these homes, land, the number of bathrooms, the number of bedrooms, you name it. It goes on and on. These are also known as features. So what a model is going to try to do, when you put in all of this data, it's going to try to understand associations between this information and come up with a model that best predicts home prices in the future.
The most basic of these machine learning models is linear regression. So if you think about inputting the data where maybe you just put in the price and the square foot, and you can kind of see the data like this.
You see that as the square foot goes up, so does the price. A model over time, in looking at this data, is going to start to find the smoothest line through the data to have the most accurate predictions in the future.
What you don't want it to do is to fit every single data point and have a line that looks like that — that's also known as overfitting — because it doesn't play nice for new data points. You don't want a model to get so calculated to your dataset that it doesn't predict accurately in the future.
A way to look at that is by the loss function. That's maybe getting a bit deeper in this, but that's how you would measure how the line is being fit. Let's see.
What are the machine learning possibilities in SEO?
So what are some of the possibilities in SEO? How can we leverage machine learning in the SEO space?
Automate meta descriptions
So there are couple ways that people are already doing this. You can automate meta descriptions by looking at the page content and using a machine model to summarize the text. So this literally summarizes the content for you and pares it down to a meta description length. Pretty incredible.
Automate titles
You could similarly do this for titles, although I don't suggest you do this for primary pages. This isn't going to be perfect. But if you have a huge, huge website, with hundreds of thousands of pages, it gets you halfway there. It's really interesting to start playing around in that space with these large websites.
Automate image alt text
You can also automate alt text for images. We see these models getting really good at understanding what's in an image.
Automate 301 redirects
301 redirects, Paul Shapiro has an incredible write-up and basically process for that already.
Automate content creation
Content creation, and if that scares some of you or if you doubt that these models can currently create content that is decent, I challenge you to go check out Talk to Transformer.
It is a pared-back version of OpenAI, which was founded by Elon Musk. It's pretty incredible and a little scary as to how good the content is just from that pared back model. So that is for sure possible in the future and even today.
Automate product/page suggestions
In addition to product and page suggestions.
So this is just going to get better. Imagine us providing content and UX specifically for the unique users that come to our site, highly personalized content, highly personalized experiences. Really exciting stuff moving forward.
Resources
I've got some resources I highly suggest you check out.
Google Codelabs is one of my favorites, just because it walks you through the steps. So if you go to Google Codelabs, filter by TensorFlow or machine learning, you can see the possible examples there. Colab notebooks or Jupyter notebooks are where you'll likely be doing any of the machine learning that you want to do on your own.
Kaggle.com is the number one resource for data science competitions. So you get to really see what are the examples, how are people using machine learning today. You'll see things like TSA has put up over $1 million for a data science team to come up with a model that predicts potential threats from security footage.
This stuff gets really interesting really fast. It's also so important to have diversity and inclusion in this space to avoid really dangerous models in the future. So it's something to definitely think about.
TensorFlow is a great resource. It's what Google put out, and it's what a lot of their machine learning models is built off of. They've got a really great JavaScript platform that you can play around with.
Then Algorithmia is sort of a one-stop shop for models. So if you don't care to dip your toes into machine learning and you just want say a summarizer model or a particular type of model, you could potentially find one there and do a plug-and-play of sorts.
So that's pretty interesting and fun to explore. The last thing is a machine learning model is only as good as the data. I can't express that enough. So a lot of machine learning and data scientists, it's all data cleaning and parsing, and that's the bulk of the work in this field.
It's important to be aware of that. So that's it for Machine Learning 101. Thank you so much for joining me, and I hope to see you all again soon. Thanks.
If you enjoyed this episode of Whiteboard Friday, you'll be delighted by all the cutting-edge SEO knowledge you'll get from our newly released MozCon 2019 video bundle. Catch more useful technical tips in Britney's talk, plus 26 additional future-focused topics from our top-notch speakers:
We suggest scheduling a good old-fashioned knowledge share with your colleagues to educate the whole team — after all, who didn't love movie day in school? ;-)
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
I’m a self-funded start-up business owner. As such, I want to get as much as I can for free before convincing our finance director to spend our hard-earned bootstrapping funds. I’m also an analyst with a background in data and computer science, so a bit of a geek by any definition.
What I try to do, with my SEO analyst hat on, is hunt down great sources of free data and wrangle it into something insightful. Why? Because there’s no value in basing client advice on conjecture. It’s far better to combine quality data with good analysis and help our clients better understand what’s important for them to focus on.
In this article, I will tell you how to get started using a few free resources and illustrate how to pull together unique analytics that provide useful insights for your blog articles if you’re a writer, your agency if you’re an SEO, or your website if you’re a client or owner doing SEO yourself.
The scenario I’m going to use is that I want analyze some SEO attributes (e.g. backlinks, Page Authority etc.) and look at their effect on Google ranking. I want to answer questions like “Do backlinks really matter in getting to Page 1 of SERPs?” and “What kind of Page Authority score do I really need to be in the top 10 results?” To do this, I will need to combine data from a number of Google searches with data on each result that has the SEO attributes in that I want to measure.
Let’s get started and work through how to combine the following tasks to achieve this, which can all be setup for free:
Querying with Google Custom Search Engine
Using the free Moz API account
Harvesting data with PHP and MySQL
Analyzing data with SQL and R
Querying with Google Custom Search Engine
We first need to query Google and get some results stored. To stay on the right side of Google’s terms of service, we’ll not be scraping Google.com directly but will instead use Google’s Custom Search feature. Google’s Custom Search is designed mainly to let website owners provide a Google like search widget on their website. However, there is also a REST based Google Search API that is free and lets you query Google and retrieve results in the popular JSON format. There are quota limits but these can be configured and extended to provide a good sample of data to work with.
When configured correctly to search the entire web, you can send queries to your Custom Search Engine, in our case using PHP, and treat them like Google responses, albeit with some caveats. The main limitations of using a Custom Search Engine are: (i) it doesn’t use some Google Web Search features such as personalized results and; (ii) it may have a subset of results from the Google index if you include more than ten sites.
Notwithstanding these limitations, there are many search options that can be passed to the Custom Search Engine to proxy what you might expect Google.com to return. In our scenario, we passed the following when making a call:
start=1 – The index of the first result to return – e.g. SERP page 1. Successive calls would increment this to get pages 2–5.
Google has said that the Google Custom Search engine differs from Google .com, but in my limited prod testing comparing results between the two, I was encouraged by the similarities and so continued with the analysis. That said, keep in mind that the data and results below come from Google Custom Search (using ‘whole web’ queries), not Google.com.
Using the free Moz API account
Moz provide an Application Programming Interface (API). To use it you will need to register for a Mozscape API key, which is free but limited to 2,500 rows per month and one query every ten seconds. Current paid plans give you increased quotas and start at $250/month. Having a free account and API key, you can then query the Links API and analyze the following metrics:
The number of links (external, equity or nonequity or not,) to the URL
umrp**
16384
The MozRank of the URL, as a normalized 10-point score
umrr**
16384
The MozRank of the URL, as a raw score
fmrp**
32768
The MozRank of the URL's subdomain, as a normalized 10-point score
fmrr**
32768
The MozRank of the URL's subdomain, as a raw score
us
536870912
The HTTP status code recorded for this URL, if available
upa
34359738368
A normalized 100-point score representing the likelihood of a page to rank well in search engine results
pda
68719476736
A normalized 100-point score representing the likelihood of a domain to rank well in search engine results
NOTE: Since this analysis was captured, Moz documented that they have deprecated these fields. However, in testing this (15-06-2019), the fields were still present.
Moz API Codes are added together before calling the Links API with something that looks like the following:
For a great starting point on querying Moz with PHP, Perl, Python, Ruby and Javascript, see this repository on Github. I chose to use PHP.
Harvesting data with PHP and MySQL
Now we have a Google Custom Search Engine and our Moz API, we’re almost ready to capture data. Google and Moz respond to requests via the JSON format and so can be queried by many popular programming languages. In addition to my chosen language, PHP, I wrote the results of both Google and Moz to a database and chose MySQL Community Edition for this. Other databases could be also used, e.g. Postgres, Oracle, Microsoft SQL Server etc. Doing so enables persistence of the data and ad-hoc analysis using SQL (Structured Query Language) as well as other languages (like R, which I will go over later). After creating database tables to hold the Google search results (with fields for rank, URL etc.) and a table to hold Moz data fields (ueid, upa, uda etc.), we’re ready to design our data harvesting plan.
Google provide a generous quota with the Custom Search Engine (up to 100M queries per day with the same Google developer console key) but the Moz free API is limited to 2,500. Though for Moz, paid for options provide between 120k and 40M rows per month depending on plans and range in cost from $250–$10,000/month. Therefore, as I’m just exploring the free option, I designed my code to harvest 125 Google queries over 2 pages of SERPs (10 results per page) allowing me to stay within the Moz 2,500 row quota. As for which searches to fire at Google, there are numerous resources to use from. I chose to use Mondovo as they provide numerous lists by category and up to 500 words per list which is ample for the experiment.
I also rolled in a few PHP helper classes alongside my own code for database I/O and HTTP.
In summary, the main PHP building blocks and sources used were:
Google Custom Search Engine – Ash Kiswany wrote an excellent article using Jacob Fogg’s PHP interface for Google Custom Search;
Mozscape API – As mentioned, this PHP implementation for accessing Moz on Github was a good starting point;
Database I/O – PHP has excellent support for MySQL which I wrapped into classes from these tutorials.
One factor to be aware of is the 10 second interval between Moz API calls. This is to prevent Moz being overloaded by free API users. To handle this in software, I wrote a "query throttler" which blocked access to the Moz API between successive calls within a timeframe. However, whilst working perfectly it meant that calling Moz 2,500 times in succession took just under 7 hours to complete.
Analyzing data with SQL and R
Data harvested. Now the fun begins!
It’s time to have a look at what we’ve got. This is sometimes called data wrangling. I use a free statistical programming language called R along with a development environment (editor) called R Studio. There are other languages such as Stata and more graphical data science tools like Tableau, but these cost and the finance director at Purple Toolz isn’t someone to cross!
I have been using R for a number of years because it’s open source and it has many third-party libraries, making it extremely versatile and appropriate for this kind of work.
Let’s roll up our sleeves.
I now have a couple of database tables with the results of my 125 search term queries across 2 pages of SERPS (i.e. 20 ranked URLs per search term). Two database tables hold the Google results and another table holds the Moz data results. To access these, we’ll need to do a database INNER JOIN which we can easily accomplish by using the RMySQL package with R. This is loaded by typing "install.packages('RMySQL')" into R’s console and including the line "library(RMySQL)" at the top of our R script.
We can then do the following to connect and get the data into an R data frame variable called "theResults."
library(RMySQL)
# INNER JOIN the two tables
theQuery <- "
SELECT A.*, B.*, C.*
FROM
(
SELECT
cseq_search_id
FROM cse_query
) A -- Custom Search Query
INNER JOIN
(
SELECT
cser_cseq_id,
cser_rank,
cser_url
FROM cse_results
) B -- Custom Search Results
ON A.cseq_search_id = B.cser_cseq_id
INNER JOIN
(
SELECT *
FROM moz
) C -- Moz Data Fields
ON B.cser_url = C.moz_url
;
"
# [1] Connect to the database
# Replace USER_NAME with your database username
# Replace PASSWORD with your database password
# Replace MY_DB with your database name
theConn <- dbConnect(dbDriver("MySQL"), user = "USER_NAME", password = "PASSWORD", dbname = "MY_DB")
# [2] Query the database and hold the results
theResults <- dbGetQuery(theConn, theQuery)
# [3] Disconnect from the database
dbDisconnect(theConn)
NOTE: I have two tables to hold the Google Custom Search Engine data. One holds data on the Google query (cse_query) and one holds results (cse_results).
We can now use R’s full range of statistical functions to begin wrangling.
Let’s start with some summaries to get a feel for the data. The process I go through is basically the same for each of the fields, so let’s illustrate and use Moz’s ‘UEID’ field (the number of external equity links to a URL). By typing the following into R I get the this:
Looking at this, you can see that the data is skewed (a lot) by the relationship of the median to the mean, which is being pulled by values in the upper quartile range (values beyond 75% of the observations). We can however, plot this as a box and whisker plot in R where each X value is the distribution of UEIDs by rank from Google Custom Search position 1-20.
Note we are using a log scale on the y-axis so that we can display the full range of values as they vary a lot!
A box and whisker plot in R of Moz’s UEID by Google rank (note: log scale)
Box and whisker plots are great as they show a lot of information in them (see the geom_boxplot function in R). The purple boxed area represents the Inter-Quartile Range (IQR) which are the values between 25% and 75% of observations. The horizontal line in each ‘box’ represents the median value (the one in the middle when ordered), whilst the lines extending from the box (called the ‘whiskers’) represent 1.5x IQR. Dots outside the whiskers are called ‘outliers’ and show where the extents of each rank’s set of observations are. Despite the log scale, we can see a noticeable pull-up from rank #10 to rank #1 in median values, indicating that the number of equity links might be a Google ranking factor. Let’s explore this further with density plots.
Density plots are a lot like distributions (histograms) but show smooth lines rather than bars for the data. Much like a histogram, a density plot’s peak shows where the data values are concentrated and can help when comparing two distributions. In the density plot below, I have split the data into two categories: (i) results that appeared on Page 1 of SERPs ranked 1-10 are in pink and; (ii) results that appeared on SERP Page 2 are in blue. I have also plotted the medians of both distributions to help illustrate the difference in results between Page 1 and Page 2.
The inference from these two density plots is that Page 1 SERP results had more external equity backlinks (UEIDs) on than Page 2 results. You can also see the median values for these two categories below which clearly shows how the value for Page 1 (38) is far greater than Page 2 (11). So we now have some numbers to base our SEO strategy for backlinks on.
# Create a factor in R according to which SERP page a result (cser_rank) is on
> theResults$rankBin <- paste("Page", ceiling(theResults$cser_rank / 10))
> theResults$rankBin <- factor(theResults$rankBin)
# Now report the medians by SERP page by calling ‘tapply’
> tapply(theResults$moz_ueid, theResults$rankBin, median)
Page 1 Page 2
38 11
From this, we can deduce that equity backlinks (UEID) matter and if I were advising a client based on this data, I would say they should be looking to get over 38 equity-based backlinks to help them get to Page 1 of SERPs. Of course, this is a limited sample and more research, a bigger sample and other ranking factors would need to be considered, but you get the idea.
Now let’s investigate another metric that has less of a range on it than UEID and look at Moz’s UPA measure, which is the likelihood that a page will rank well in search engine results.
UPA is a number given to a URL and ranges between 0–100. The data is better behaved than the previous UEID unbounded variable having its mean and median close together making for a more ‘normal’ distribution as we can see below by plotting a histogram in R.
A histogram of Moz’s UPA score
We’ll do the same Page 1 : Page 2 split and density plot that we did before and look at the UPA score distributions when we divide the UPA data into two groups.
# Report the medians by SERP page by calling ‘tapply’
> tapply(theResults$moz_upa, theResults$rankBin, median)
Page 1 Page 2
43 39
In summary, two very different distributions from two Moz API variables. But both showed differences in their scores between SERP pages and provide you with tangible values (medians) to work with and ultimately advise clients on or apply to your own SEO.
Of course, this is just a small sample and shouldn’t be taken literally. But with free resources from both Google and Moz, you can now see how you can begin to develop analytical capabilities of your own to base your assumptions on rather than accepting the norm. SEO ranking factors change all the time and having your own analytical tools to conduct your own tests and experiments on will help give you credibility and perhaps even a unique insight on something hitherto unknown.
Google provide you with a healthy free quota to obtain search results from. If you need more than the 2,500 rows/month Moz provide for free there are numerous paid-for plans you can purchase. MySQL is a free download and R is also a free package for statistical analysis (and much more).
Go explore!
Sign up for The Moz Top 10, a semimonthly mailer updating you on the top ten hottest pieces of SEO news, tips, and rad links uncovered by the Moz team. Think of it as your exclusive digest of stuff you don't have time to hunt down but want to read!
