large scale seo - method to the madness

Method to the madness - Large Scale SEO

What’s the problem?

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

A “modern” website:

Community portal example.com/forums/

What could go wrong?• Support portal outranking the main website

• An error in the shop software causing the entire site to stop being indexed

• So much crawl budget being spent on community forums that shop doesn’t get crawled properly

• KB linking to shop everywhere, shop changes URL structure

• etc. etc. etc.

What’s the solution?

Modern websites cause a need for a different SEO

approach.

Method• Build properly

• Get all the needed data

• Optimize constantly

• Audit regularly

• Teach others

Do you know what’s on your site?

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

It can be as simple as this:

Community portal example.com/forums/

Method• Build properly

• Get all the needed data

• Optimize constantly

• Audit regularly

• Teach others

A comprehensive SEO data strategy• Service Oriented Approach to main sources of

information:

• Analytics

• Logs, specifically:

• Spider logs

• Error logs

• Internal Search

Analytics

Analytics• Often the only thing that’s already centralized.

• Far too often answers questions that you don’t need to answer.

Do you know what content on your site is

the most engaging?

Facebook stats?

Twitter stats?

Pinterest?

Video?

This can be done.

Engagement metrics

Engagement metrics

Good analytics is an aggegration of loads of

sources of data.

Logs & Search console

Googlebot stats before GSC tells youDo you need Google Search Console to tell you you’ve got errors on your site?

LogsCentralize all logs into one “stash”.

My personal preference: Logstash on ElasticSearsch.

Using Kibana, you can filter out and create specific dashboards for bot traffic and errors.

Googlebot stats that HELP

Some of my favorite viewsTop spidered URLs over the last 24 hours, by Google user agent.

Average response time, slow loading pages, slow loading page types.

Googlebot 4xx and 5xx errors over the last 24 hours.

Google Search Console errorsBe honest, how many of you have “accepted” a “background noise” of thousands or even 100k+ errors?

User errorsIs anybody at your company monitoring user “crashes”? Pages that 404 more than X times a day?

If engagement is king, errors are the

hangman.

Internal Search

Internal search

Internal search should tell you which content

you have.

Internal Search monitoring• Is someone / something monitoring the missed

keyword opportunities across your site?

• This could flow back into ElasticSearch / Kibana too

URLs

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

A link changes in your shop

Community portal example.com/forums/

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

What happens to the links pointing there?

Community portal example.com/forums/

URLs and links across your site• In an ideal world, you never embed complete links.

• You embed ID’s, which are translated to links based on one datasource.

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

One database that has all the URLs for:

Community portal example.com/forums/

All URLs• App deeplinks

• hreflang relations

• AMP URLs

• etc. etc.

XML Sitemaps & hreflang

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

You probably don’t have this:

Community portal example.com/forums/

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

Community portal example.com/forums/

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

Community portal example.com/forums/

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

Community portal example.com/forums/

Homepage example.com/

Support portal kb.example.com

Shop example.com/shop/

News section example.com/news/

But this, in 4, 5, 8, 20 languages:

Community portal example.com/forums/

Multiple Sitemap indexes• Don’t try to combine them!

Need to build XML sitemaps?• Base it off of your Internal Search API!

• Use a smart naming scheme to help you dive into sections of your site easily.

Method• Build properly

• Get all the needed data

• Optimize constantly

• Audit regularly

• Teach others

Use your internal search API for related content.

Feed engagement and commercial metrics in.

Method• Build properly

• Get all the needed data

• Optimize constantly

• Audit regularly

• Teach others

Site changes? Embed SEO into CI• Who knows what CI is?

Create CI tests for important stuff• Check canonicals

• Check hreflang

• Check that URLs stay the same

• Check HTTP header codes

• etc.

Regular crawls to identify issues• Screaming Frog, OnPage.org, DeepCrawl, etc.

• I don’t care which one you use, as long as you one.

Who am I kidding? Use OnPage.org ;)

Method• Build properly

• Get all the needed data

• Optimize constantly

• Audit regularly

• Teach others

Teach developers technical SEO.

Teach analytics people to look beyond your

main analytics package.

Teach!

Questions?Follow along:

• https://yoast.com/

• @jdevalk & @yoast on Twitter

• facebook.com/yoast