Ultimate SEO Audit | Smart Indexing

@tristan_pirouz

Tristan Pirouz, Head of BD EMEA, DeepCrawl

@tristan_pirouz

HOW TO POSITION YOUR YOURSELF AS A WEBMASTER‘Smart Indexing’

@tristan_pirouz

Over-In

dexing

Under-Ind

exing

You want to read more: http://info.deepcrawl.com/science-of-crawlspace

Smart Indexing!

Num

ber o

f Ind

exed

Pag

es

IndexingINDEXING IS THE CONTROL, PROCESS OR MEASURE OF WHICH PAGES ARE INDEXED

When showcasing your digital assets to Google you want your preferred pages to be seen.

@tristan_pirouz

Source: How to rank yours in Google search results, Google in-Depth Articles

Why we want to be indexed?

HOW OFTEN A BOT CRAWLS A SPECIFIC PAGE OR DIRECTORY PER HOUR OR DAY?If a page hasn’t been crawled recently, it won’t rank well.

If a page hasn’t been crawled within a certain time period then it receives less authority.

@tristan_pirouz

You want to read more: http://info.deepcrawl.com/science-of-crawlspaceSource: Crawl Optimization, blindfiveyearold

INDEXED PAGES ARE THOSE WHICH ARE RETURNED IN SEARCH RESULTSThe method to understand where you are is…

URL Analyze Validate Universe

@tristan_pirouz

INDEXED PAGES ARE THOSE WHICH ARE RETURNED IN SEARCH RESULTSThe method to understand where you are is this…

• Ranking checking

• Log analysis• Search console• Sitemaps• Organic visits• Site:• Info:

• Dedupe• Crawl with DeepCrawl

Compare this to the indexed pages count in search console

Analyze Validate:URL Universe:

@tristan_pirouz

HOW TO ACHIEVE SMART INDEXING?Achieving ‘Smart Indexing’

@tristan_pirouz

Over-IndexationIncorrect Canonical tagsLack of NoIndexLack of Canonical tags

Under-IndexationThin ContentUndiscovered PagesInefficient ArchitectureSlow PerformanceMade NonIndexable by mistake

Smart Indexing

EXAMPLES OF UNDER-INDEXING

@tristan_pirouz

PAGES MAY NOT BE DISCOVERED DUE TO INSUFFICIENT LINKING/ HIGH CRAWL DEPTHUndiscoverable pages

@tristan_pirouz

Googlebot may not access your content reach pages as they are hidden too far down your website

HOW CRAWLABLE IS YOUR SITE?Crawl efficiency

@tristan_pirouz

EXAMPLES OF OVER-INDEXING

@tristan_pirouz

A LACK OF NOINDEX CAN RESULTS IN AN INDEX BLOAT CAUSING A DILUTION OF LINKJUICE AND CONSTRAINED CRAWL BUDGET

Lack of noindex

@tristan_pirouz

DO INCLUDE A CANONICAL TAG ON EVERY PAGE, WITHOUT EXCEPTIONLack of canonical tags

Prevent any possible duplication. Even if there are no other versions of a page, then that page should still include a canonical tag that links to itselfSource: Cononical Tags: Easy Dos and Don’ts, DeepCrawl

@tristan_pirouz

CONTROLS

@tristan_pirouz

UNDERSTANDING HOW TO USE NOINDEX, DISALLOW & NOFOLLOW MEANS YOU CANADVISE SEs AROUND

What to consider when optimizing your crawl efficiency?

You want to read more: https://www.deepcrawl.com/knowledge/best-practice/robots-txt-noindex-the-best-kept-secret-in-seo/You want to read more: https://www.deepcrawl.com/knowledge/best-practice/noindex-disallow-nofollow/

Backlink Authority

Craw

l Effi

cienc

y

meta canonical

robots.txt disallow

meta noindex

rel nofollow

meta noindex + rel nofollow

robots.txt noindex

@tristan_pirouz

Cra

wl E

ffici

ency

Backlink Authority

DETERMINE YOUR PERFORMANCE RELATIVE TO YOUR COMPETITORSHow am I performing relative to my competitors?

Determine your performance relative to your competitors

@tristan_pirouz

IS THE ABILITY TO CONSTRUCT YOUR WEBSITE ON THE FORESIGHT OF THE MARKETLEADERS WEBSITE ARCHITECTURE SIZE

Concept of architecture

3 Values:Market Size: What is the average website size set by market leaders?Actual Size: What is the actual website size of the given site?Indexed Size: What are the total number of returned pages in search results?

@tristan_pirouz

WHEN YOUR WEBSITE IS BELOW THE EXPECTED MARKET SIZEUnder-architected

When does Under-Architected occur:

1.When there is a lack of inventory, products, content

2.Lack of relevant categorization and facets

@tristan_pirouz

WHEN YOUR WEBSITE IS ABOVE THE EXPECTED MARKET SIZEOver-architected

When does Over-Architected occur:

1.Low relevancy or low value pages without unique content created

2.Uncontrolled categorization

3. Incorrect management of expired content

@tristan_pirouz

THEREFORE…

@tristan_pirouz

HOW TO ACHIEVE SMART INDEXING?‘Smart Indexing’ | Checklist

Over-IndexationIncorrect Canonical tagsLack of NoIndexLack of Canonical tags

Under-IndexationThin ContentUndiscovered PagesInefficient ArchitectureSlow PerformanceMade NonIndexable by mistake

Smart Indexing

@tristan_pirouz

HOW YOU WANT YOUR WEBSITE TO LOOK LIKE?‘Smart Indexing’

@tristan_pirouz

MULTIPLE SCENARIOS YOU CAN FIND YOURSELF AS A WEBMASTER‘Super Smart Indexing’

ArchitectureIn

dexi

ngUnder

Und

erOver

Ove

r

@tristan_pirouz

Your Site Audit with DeepCrawl

Tristan PirouzHead of Business Development EMEA, DeepCrawl

Tristan@DeepCrawl.com

@tristan_pirouz