09/04/2023 www.rba.co.uk 1

Making Google Behave

Society of Indexers Annual ConferenceRoyal Agricultural University, Cirencester

5-7th September 2014

Karen BlakemanRBA Information Services

Karen.Blakeman@rba.co.ukwww.rba.co.uk twitter.com/karenblakeman

Slides available on authorSTREAM, Slideshare and http://www.rba.co.uk/as/

This work is licensed under a Creative Commons Attribution-ShareAlike License.

09/04/2023 www.rba.co.uk 2

Search Engine Market Share June 2014 http://theeword.co.uk/info/search_engine_market.html

09/04/2023 www.rba.co.uk 3

2014 Financial Tables – Investor Relations – Google https://investor.google.com/financial/tables.html

09/04/2023 www.rba.co.uk 4

Google Play App Revenue Doubles in a Year http://mobilemarketingmagazine.com/google-play-app-revenue-doubles

With increased revenue, wearables focus, Google takes fast lane to mobile transition http://tech.firstpost.com/news-analysis/increased-revenue-wearables-focus-google-takes-fast-lane-mobile-transition-227605.html

09/04/2023 www.rba.co.uk 5

Google Brings More "Now" To Search With New Quick Answers http://searchengineland.com/google-brings-more-now-capabilties-to-search-with-new-quick-answers-169658

09/04/2023 www.rba.co.uk 6

Google search

– is free text search

– is not aimed at the serious, in depth researcher

– does not do Boolean

– structured search is limited even in Scholar

– is personalised

– can change minute by minute

09/04/2023 www.rba.co.uk 7

It's Over: The Rise & Fall Of Google Authorship For Search Results http://searchengineland.com/goodbye-google-authorship-201975

09/04/2023 www.rba.co.uk 8

Google Authorship

Without Google Authorship

Five things you need to know about Google search

1. Google personalises your search

Personalises search based on– location – country, town

– past search history

– past browsing activity

– your activity in other areas of Google e.g. YouTube, blogs, images

– your activity on all your devices linked to your Google account

– what other people have clicked on for similar searches

– the device you are using

09/04/2023 www.rba.co.uk 9

09/04/2023 www.rba.co.uk 10

Private browsing - quickest way “un-personalise”search

Chrome - New Incognito window Ctrl+Shift+N

FireFox Ctrl+Shift+P

Internet Explorer Ctrl+Shift+P

Opera Ctrl+Shift+N

Will not remove country personalisation

Not search engine specific, built into the browser

09/04/2023 www.rba.co.uk 11

Five things you need to know about Google search

2. Google automatically looks for variations on your search terms and sometimes drops terms from your search

– Google now tells you which terms it has ignored (some of the time)

– “..” around terms, phrases, names, titles of documents does not always work

– To force an exact match and inclusion of a term in a search prefix it with ‘intext:’

intext:agricultural occupational asthma

– Use Verbatim for an exact match search

09/04/2023 www.rba.co.uk 12

Let’s throw it into the soup and see if the cat licks it up

Google introduces the “soft AND”

“When you do a multi-term query on Google (even with quoted terms), the algorithm sometimes backs-off from hard ANDing all of the terms together.......it’s clear that people will often write long queries (with anywhere from 5 to 10 terms) for which there are no results. Google will then selectively remove the terms that are the lowest frequency to give you some results (rather than none)....Soft AND is a way to reduce the overall frustration and give the searcher something to examine (and with luck, a chance to reformulate their query).”

Dan Russell

http://www.rba.co.uk/wordpress/2011/11/08/dear-google-stop-messing-with-my-search/#comments

09/04/2023 www.rba.co.uk 13

Google – missing terms

09/04/2023 www.rba.co.uk 14

09/04/2023 www.rba.co.uk 15

Google Verbatim

Choosing your search terms

Google automatically looks for synonyms and variations on your terms

– biofuels will find biodiesel, biogas, bio-ethanol etc.

– but do not get the same results if you use biodiesel instead of biofuels

– run separate searches using alternative terms

– no information on how the synonyms are identified or implemented

The terms you use can radically change results– copper mining north wales

– copper extraction north wales

09/04/2023 www.rba.co.uk 16

Choosing your search terms

09/04/2023 www.rba.co.uk 17

09/04/2023 www.rba.co.uk 18

Five things you need to know about Google search

3. Google web search does not search everything it has in its database

– two indexes: main, default index and the supplemental index

– supplemental index may contain less popular, unusual, specialist material

– supplemental index comes into play when Google thinks your search has returned too few results

– Verbatim and some advanced search commands seems to trigger a search in the supplemental index

09/04/2023 www.rba.co.uk 19

“Normal search”1,555,500

Search after Verbatim is applied35,500,000

09/04/2023 www.rba.co.uk 20

Five things you need to know about Google search

4. Google changes its algorithms several hundred times a year How Google makes improvements to its search algorithm

- YouTube http://youtu.be/J5RZOU6vK4Q

09/04/2023 www.rba.co.uk 21

Five things you need to know about Google search

5. We are all Google’s lab rats

Just Testing: Google Users May See Up To A Dozen

Experiments

http://searchengineland.com/just-testing-google-searchers-may-see-up-to-a-dozen-experiments-141570

Mostly minor effects on search but sometimes totally bizarre

results

Google decides that coots are really lions

http://www.rba.co.uk/wordpress/2011/02/12/

google-decides-that-coots-are-really-lions/

Update on coots vs. lions

http://www.rba.co.uk/wordpress/2011/02/21/

update-on-coots-vs-lions/

Hummingbird

Not just an update but a completely new algorithm

Tries to make “sense” of your query and put it into context, natural language queries

Not just search history but also your location, device being used

Announced 26th September 2013 but had already been implemented for about a month

Many aspects had been tested over the previous months and past year

09/04/2023 www.rba.co.uk 22

Menu options change depending on your search

09/04/2023 www.rba.co.uk 23

Google rewrites page titles

09/04/2023 www.rba.co.uk 24

Google's Matt Cutts: Why Google Will Ignore Your Page Title Tag & Write Its Own http://searchengineland.com/googles-matt-cutts-look-title-match-query-190039

http://googlesystem.blogspot.co.uk/2013/11/google-knowledge-graph-gets-confused.html

09/04/2023 www.rba.co.uk 25

EU - so called “right to be forgotten” ruling

09/04/2023 www.rba.co.uk 26

Edition of Monday, January 19, 1998, page 23 - Newspaper - Lavanguardia.es http://hemeroteca.lavanguardia.com/preview/1998/01/19/pagina-23/33842001/pdf.html

EU Court of Justice ruled that Google is a “data controller” under Data Protection legislation and must remove links to information that is “inadequate, irrelevant .... or excessive” from search results on a person’s name.

Information is NOT removed from the web

Subject can apply to have links in search results that point to specific information removed from the results

Not just Google – all search engines with an EU presence

Only applies to searches conducted in the EU + Norway, Switzerland, Iceland and Lichtenstein

Not automatic – subject has to apply and request will be assessed to see if the information is “inadequate, irrelevant or no longer relevant, or excessive in relation to the purposes for which they were processed.”

09/04/2023 www.rba.co.uk 27

How to get around it?

Google now removing results (and also adding back in results!) from searches in European country versions of Google

Indicates on the results page if information has been excluded

Google adds removal statement from all results for searches on personal names even if nothing has been removed

Use non-European Google to see all results e.g. Google.com, Google.ca - but will see country biased results

09/04/2023 www.rba.co.uk 28

09/04/2023 www.rba.co.uk 29

Google Knowledge Graph and carousel

09/04/2023 www.rba.co.uk 30

Google gets it wrong again

09/04/2023 www.rba.co.uk 31

Google gets it wrong yet again

09/04/2023 www.rba.co.uk 32

Google "Henry VIII wives": Jane Seymour reveals search engine's blind spots http://www.slate.com/blogs/future_tense/2013/09/23/google_henry_viii_wives_jane_seymour_reveals_search_engine_s_blind_spots.html

Image courtesy of Will Oremus

Nutrition facts

09/04/2023 www.rba.co.uk 33

Information from Wikipedia and USDA

Compare

09/04/2023 www.rba.co.uk 34

Google Quick Answers

Some Of The Weird Issues When Google's Quick Answers Come From Random Sources http://searchengineland.com/weird-issues-googles-quick-answers-comes-random-sources-197611

“Answers” appear at top of the results page and below the ads

Try– your favourite football club

– flight times

– symptoms of a medical condition

09/04/2023 www.rba.co.uk 35

09/04/2023 www.rba.co.uk 36

09/04/2023 www.rba.co.uk 37

09/04/2023 www.rba.co.uk 38

09/04/2023 www.rba.co.uk 39

And then on another day...

09/04/2023 www.rba.co.uk 40

And one of many wrong Quick Answers submitted to me by delegates of this conference

Many thanks to Philip Stirups for the example. About 24 hours after taking this screen shot Google corrected the error.

09/04/2023 www.rba.co.uk 41

Google commands

Speech marks around phrases or titles of articles “Geochemical evaluation of flowback brine from Marcellus gas wells”

Note: if Google finds no documents, or very few documents, containing your phrase it will ignore the speech marks

Verbatim – runs your search exactly as you have typed it

Search tools, All results, Verbatim (see earlier slide)

intext: before your term – term must be present and exactly as you have typed it

UK public transport intext:biodiesel

Google commands

Think file format– PDF for research papers, lengthy documents, government

reports, industry papers

– ppt or pptx for presentations, tracking down an expert on a topic

– xls, xlsx or csv for data and statistics

filetype: command zeolites environmental remediation filetype:pdf

"north sea" deep water drilling filetype:ppt

"north sea" deep water drilling filetype:pptx

09/04/2023 www.rba.co.uk 42

Google commands

Site searchFor searching large websites, or groups of sites by type for example government, NHS, academic

Can exclude sites using -site:

agricultural occupational asthma UK site:nhs.uk

agricultural occupational asthma UK site:ac.uk

agricultural occupational asthma UK site:gov.uk

agricultural occupational asthma UK site:gov.uk –site:hse.gov.uk

09/04/2023 www.rba.co.uk 43

Google commands

Numeric range search

Anything to do with numbers and quantities: years, temperatures, weights, distances, prices etc

Use the advanced search screen or type in your two numbers separated by two full stops as part of your search

  Rhodes zeolites environmental remediation 2005..2014

toblerone 1..6 kg

09/04/2023 www.rba.co.uk 44

09/04/2023 www.rba.co.uk 45

Google commands

Words in the title – can be single words or phrases

Ensures subject is the main focus of the article

Use advanced search screen or intitle:intitle:”diabetic retinopathy”

Words in the URL – can be single words or phrases

Use advanced search screen or inurl:inurl:”diabetic retinopathy”

There are allintitle and allinurl commands but they are unreliable and do not work with other commands

define

09/04/2023 www.rba.co.uk 46

09/04/2023 www.rba.co.uk 47

Google search options

Date

Restrict your results to information that has been published within the last hour, day, week, month, year or your own date range

Search tools, Any time and select an option 

09/04/2023 www.rba.co.uk 48

Google search options

Reading level

Changes the type of material that is returned

Run the search and from the menu above the results select Search tools, All results and then Reading level

09/04/2023 www.rba.co.uk 49

Google search options

Google Scholar

http://scholar.google.com/

“Google Scholar provides a simple way to broadly search for scholarly literature. From one place, you can search across many disciplines and sources: articles, theses, books, abstracts and court opinions, from academic publishers, professional societies, online repositories, universities and other web sites. Google Scholar helps you find relevant work across the world of scholarly research”.

 • Search all scholarly literature from one convenient place

• Explore related works, citations, authors, and publications

• Locate the complete document through your library or on the web

• Keep up with recent developments in any area of research

• Check who's citing your publications, create a public author profile

09/04/2023 www.rba.co.uk 50

09/04/2023 www.rba.co.uk 51

Google Scholar

Does not cover all key journals in all subjects – no source list

Top publications for subjects and languages under Metrics link on home page or http://scholar.google.co.uk/citations?view_op=top_venues&hl=en

Scholar indexes the full text but you may have to pay to view the whole article

Groups different versions of an article together

09/04/2023 www.rba.co.uk 52

Google Scholar

Includes open access material, pre-prints, institutional repositories (but not necessarily author self archived repositories)

Includes material that is NOT peer reviewed but is structured and looks like an academic article (title in large font, authors, affiliations, abstract, keywords, citations)

Pre-prints and IR copies may differ from final published version – charts and images may be redacted because of copyright restrictions

09/04/2023 www.rba.co.uk 53

Google Scholar

Does NOT use the publishers’ metadata

Date and author search looks in the area of the document where those elements are usually found

Page numbers, part of an address, data item may be mistaken for publication year

Sometimes gets the author wrong

09/04/2023 www.rba.co.uk 54

Google Scholar verified author and citations

09/04/2023 www.rba.co.uk 55

Google Scholar advanced search commands

Use advanced search screen or commands as follows:

+ sign before a search term to force an exact match, for example +norne

“....” around phrases for example “environmental remediation”

 

intitle: to search for a single word in the title, for example intitle:zeolites environmental remediation

 

allintitle: to search for all of your terms in the title, for example allintitle:zeolites environmental remediation

 

author: to search on an author’s name, for example

zeolites environmental remediation author:rhodes

 

site: to limit your search to specific institution for example

marcellus shale site:psu.edu

 

Commands can be combined for a precise search, for example

author:wolford site:psu.edu allintitle:marcellus shale

Patents

https://www.google.com/patents

Coverage:– US

– Canada

– European Patent Office (EPO)

– Germany

– China

– World Intellectual Property Organisation (WIPO)

Patents available in original language and English (Google Translate)

09/04/2023 www.rba.co.uk 56

books.google.com

Includes magazines, journals, archives of newspapers (mainly US)

09/04/2023 www.rba.co.uk 57

Google Books Ngram viewer

http://books.google.com/ngrams/

09/04/2023 www.rba.co.uk 58

Google Books Ngram viewer

http://books.google.com/ngrams/

09/04/2023 www.rba.co.uk 59

09/04/2023 www.rba.co.uk 60

Google Trends http://www.google.com/trends/

09/04/2023 www.rba.co.uk 61

Google Trends http://www.google.com/trends/

Google Cultural Institute http://www.google.com/culturalinstitute/about/

09/04/2023 www.rba.co.uk 62

Google Art Project http://www.googleartproject.com/ http://www.google.com/culturalinstitute/project/art-project

09/04/2023 www.rba.co.uk 63

Google ArtProject museum view

09/04/2023 www.rba.co.uk 64

09/04/2023 www.rba.co.uk 65

1. Important to keep up to date with what Google and the other search engines are doing, and with regulatory changes

Search Engine Land http://searchengineland.com/ Search Engine Roundtable https://www.seroundtable.com/Search Engine Watch http://searchenginewatch.com/

2. Get to know Google commands and where the search options are hidden

3. Never, never, never trust the “answers” on Google’s results pages