mustafa degerli - 2007 - usability testing of google search engine

MIDDLE EAST TECHNICAL UNIVERSITY

DEPARTMENT OF COMPUTER EDUCATION &

INSTRUCTIONAL TECHNOLOGY

CEIT 440

SPECIAL STUDIES IN COMPUTER EDUCATION

“USABILITY TESTING”

Image Taken from: cognetics.com

Usability Testing of Google Search Engine

Wonder about the Usability of Google Search Engine?

Submitted by: Mustafa DEĞERLİ

Submitted to:

M. Banu GÜNDOĞAN

Date: June, 2007

1

In this paper, I endow with the information on the subject of my Usability Testing (UT) of

Google Search Engine (GSE). Within this study, I had tested how usable the GSE is with

regard to certain parameters and definitions with seven participants, one for pilot UT and six

for actual UTs. All details of test procedures of UTs and UT definitions are provided in this

paper. What is more, I come up with various analyses about the Usability of GSE (UGSE).

Towards the end, I decide on the UGSE with reference to my analyses and findings from UTs.

2

Table of Content

Abbreviations Used in This Paper, pg. 4

What is Usability?, pg. 5

Why UT of GSE?, pg. 5

Which Aspects of Usability for GSE?, pg. 6

Accurateness of the Results, pg. 6

The Time Required to Reach the Correct and Expected Result, pg. 6

The Satisfactoriness, pg. 7

Definitions of UT of GSE, pg. 7

Process (Flow) of UTGSE, pg. 8

Results of UTGSE, pg. 9

Success Results for Test Items for the Users, pg. 9

Time Results for Test Items for the Users, pg. 13

Results for Number of Users vs. Test Items, pg. 18

Results for Total Time Spent by Users in UTGSE, pg. 19

Results for Total Time Spent by All Users for Each Test Item in UTGSE, pg. 19

Results for Time Variation for Each User in Each Test Item in UTGSE, pg. 20

Results for User vs. Success Variations in Test Items for Users, pg. 21

Users vs. Test Items Results Table, pg. 22

Users vs. Test Items Spent Time Table, pg. 22

Conclusions and Final Comments, pg. 23

GSE is Richer in English Content?, pg. 23

3

GSE is Open to Manipulation?, pg. 23

GSE Has Danger in Image Search?, pg. 24

GSE is fast Enough?, pg. 24

GSE’s Goo(…)oogle Navigation is not Something Making Much Sense?, pg. 25

GSE is Pleasing?, pg. 25

GSE Provides Both by Choose and by Chance?, pg. 25

GSE’s Results are Fully Credible?, pg. 26

Other Comments, pg. 26

Do it simpler!, pg. 26

Do not Index Anything!, pg. 27

Revisit the Strategy Used in Image Indexing!, pg. 27

Appendix A, Literature Review for Google Search Engine (GSE), pg. 28

Appendix B, Kontrat-Contract, pg. 35

Appendix C, Test Definitions (TDs), pg. 36

Appendix D, Restrictions and Directions, pg. 40

Appendix E, Specifications of the Computer to be Used in Tests, pg. 46

Appendix F, Project Flowchart, pg. 47

4

Abbreviations Used in This Paper

Followings are the some abbreviations used in this paper. These abbreviations are used

frequently in this paper. Hence, it is beneficial to review them.

GSE: Google Search Engine

UT: Usability Test(ing)

UGSE: Usability of Google Search Engine

UTGSE: Usability Testing of Google Search Engine

WS: Web Search

IS: Image Search

TDs: Test Definitions

WSn: nth Wes Search

ISn: nth Image Search

Un: nth User

5

What is Usability?

In order to judge about the UGSE, it is necessary to know, what this paper means for the term of

usability. “Usability is the measure of a product's potential to accomplish the goals of the user”

(www.cesa8.k12.wi.us/media/digital_dictionary.htm). This is the simple and most general

meaning for this term. Fortunately, this definition is taken into account in this study and this

paper.

Why UT of GSE?

The foremost raison d'être made me work on this topic, UGSE, is that, without a doubt,

everybody, at least me, asks any questions to the search engines on every occasion they have any

questions related to anything.

On the other hand, I needed to make it some more specific to handle more easily and let it

happen in a good manner. In view of that, I decided to work on GSE, which is used habitually by

anyone.

That is why; I judged that there is a UT requirement for this search engine. I evaluated that

should be aware of its usability. These rationales made this situation a usability case and

fortunately a topic work on for me.

Moreover, in this process, I did prepare a literature review of GSE in order to clarify its working

logic and let myself to have a general view of GSE. Literature review of GSE is added to this

report as Appendix A.

6

Which Aspects of Usability for GSE?

That’s again important to know which aspect of usability for GSE which were taken into account with

this UT and this paper. The aspects are accurateness of the results, the time required to reach the

correct and expected result and the satisfactoriness, respectively. Details concerning these are

provided one by one below.

Accurateness of the Results

Correct and expected results of the search, and relevance of the result to the

searched case.

Here, what is imperative is for user or participant to get correct and accurate results. The

participants were required to find what I expect of them with some criteria. Since in a

sense, usability means to get what of want using a tool, it is indispensable to have

accurate and expected results by using GSE. For this reason, it is important for GSE to

provide correct and accurate results. If it cannot, there will be some doubts about its

usability. I have determined five web searches and four image searches, nine searches in

total. What I wanted of my participants was to find the things, which were listed towards

the end of this paper, which I provided them with certain criteria

The Time Required to Reach the Correct and Expected Result

How it did take to reach the correct and expected answer.

Surely, time means money in current conditions, sorry to say. What is more, I judge most

of the people work with the classical idea of basic capitalism that is minimum work and

maximum profit. In this context, any device or system says that it is usable and its

7

usability is proven should provide users do their work in a minimum possible time

interval since usability requires people do their work with minimum effort and shortest

possible time interval. These facts let me include this kind of aspect in my UT.

The Satisfactoriness

Did they come up with what was expected of?

At this point, what I looked for was not the complete meaning of satisfaction. Yet, what I

looked for was whether the user succeeded the search or not. More accurately, at this

juncture, I recorded whether user find or come up with the required or expected result or

not. Advantageously, within my test TDs and directions and restrictions declarations I

had set these things. Meaning that, it was certain what I expected of users to come up

with at the end and I let them know any required directions and restrictions. This

information, whether they succeeded or not in every sub steps of step one have let me had

meaningful tricks and points to take care in composing this report and agree on the

usability of GSE.

Definitions of UT of GSE

I included one pilot UT and six actual UTs. Moreover, in UTs I included an interview about the

GSE searches with participants.

Before starting to UTs, I had signed a contract with my participants to let them know about the

aim of this work and let them relax for this process. The contract sheet is added as Appendix B.

Here, explicitly I provide information my test procedures and process. I did both, UTs with

participant by using GSE to find the things with some certain criteria that I provided and an

8

interview about the GSE and its usage and functionality. In step 1, which is making participant

search five web searches and four image searches, I wanted participant to come across what I

expected with certain limitations and restrictions.

In step 2, I did interview with participants about their comments about the GSE and how easy to

use GSE, how satisfactory the results that he or she find. Whether or not they were bored with

searches or they got pleasure. I also did let them say whatever comes to their min related to UT

of GSE or any further comments about this process.

Between step 1 and step 2, I did come up with a tea to my participant to let them motivated and

relax. Moreover, what I cared and did is taking notes on my own while observing the participant

about where he or she had difficulty or problem, where he or she did easily, where she or he lost

time, and what the general tendencies were the participant have in searching.

Moreover, I recorded the screens of the computer that they use in this period to use in final

evaluation and this UT reporting. What I kept in this period is also the interview notes I took

during interviews.

I provide my TDs as Appendix C, restrictions and directions as Appendix D and specifications of

the computer used in tests as Appendix E. It is beneficial to have a look at them to have full

understanding of this UT.

Process (Flow) of UTGSE

The process (flowchart) of the UTGSE is added as Appendix F in JPEG and HTML formats. If

you want to have a look at it, referring that appendix.

9

Results of UTGSE

Success Results for Test Items for the Users

Here, the success results of every test item for the users are provided in charts. There are nine

charts including information for each test item. In charts 1 means user succeeded in test and o

means user failed in test. The horizontal axis includes user abbreviations and vertical axis is for

their success in tests.

13

Time Results for Test Items for the Users

At this juncture, the time results of every test item for the users are provided in charts. There are

nine charts including information for each test item. The horizontal axis includes user

abbreviations and vertical axis is for their time used in test items. The zero value written on bars

means that the user did not succeeded in the search or left the related item after some time. All

other values written on the bars mean the time that users spent on related test item. These values

are in seconds.

18

Results for Number of Users vs. Test Items

The following chart shows the number of users versus test items chart.

19

Results for Total Time Spent by Users in UTGSE

The following chart shows the the total time spent by each user in UTGSE.

Results for Total Time Spent by All Users for Each Test Item in UTGSE

The following chart shows the the total time spent by all users for each test item in UTGSE.

20

Results for Time Variation for Each User in Each Test Item in UTGSE

The following chart shows the time variations for each user in each test item in UTGSE.

21

Results for User vs. Success variations in Test Items for Users

The following chart shows the user versus success variations in test items for users in UTGSE.

22

Users vs. Test Items Results Table

The following table shows the test item results for each user in UTGSE. Zeros means success

and ones do mean failures in UTGSE.

WS1 WS2 WS3 WS4 WS5 IS1 IS2 IS3 IS4

PU 1 0 1 0 1 0 1 0 1

U1 1 0 1 1 1 1 1 1 1

U2 0 0 1 0 0 1 1 0 0

U3 1 1 1 1 1 0 1 1 1

U4 1 0 1 0 1 1 1 1 1

U5 1 1 1 1 1 1 1 1 1

U6 1 0 1 1 1 1 1 0 1

Users vs. Test Items Spent Time Table

The following table shows the times spent by each user in each test items in UTGSE. Times are

given in seconds.

WS1 WS2 WS3 WS4 WS5 IS1 IS2 IS3 IS4

PU 83 0 80 0 390 0 78 0 132

U1 32 0 130 640 469 132 90 190 98

U2 0 0 142 0 0 77 66 0 0

U3 45 980 165 1190 323 0 130 78 66

U4 64 0 65 0 457 116 45 133 201

U5 78 1242 44 897 231 78 68 67 79

U6 45 0 98 650 97 98 77 0 95

23

Conclusions and Final Comments

After UTs and interviews with participants, and analyses of both these data, I have pointed out

some points. I have listed and explained these findings and comments one by one below.

GSE is Richer in English Content?

UTs apprehended by me in this process, interviews with UT participants and my own

experiences authenticated that concerning content, indexed content by GSE, GSE can be

evaluated as richer in English content. This is one possible reason of my participants searching

for something in Turkish have some intricacy in certain test items. When they tried the same

thing in English, they accomplished better. This may be just as a consequence of the obtainable

sources is English is all embracing when compared with the same thing in different languages.

The other possible reason may be GSE logic when indexing pages. It may have allocated more

indexing servers for English pages. Consequently, GSE is richer in English content and its

usability with respect to indexed content can be said as better when compared with other

language contents.

GSE is Open to Manipulation?

Unfortunately, the answer of this question is YES. As its ranking technology and other working

logic parameters (Explained in detail in Appendix A, GSE is likely to be manipulated by some

exterior factors. Just because GSE cares about how many links are related to the specified item

and list the item having more links first and less links later, there is a would-be risk that some

organization may provide counterfeit contents and let people have these via GSE, with no

trouble. At this juncture, what the sham organizations all have to do is just creating counterfeit

24

contents and relating them in millions of pages via links. GSE has to limit or deal with this

potential risk.

GSE has Danger in Image Search?

Sorry to say, the answer is again YES. UTs apprehended by me in this process, interviews with

UT participants and my own experiences authenticated that GSE fails to index and provide

contents that is expected and desired. GSE may provide much unrelated results for image

searches. People felt themselves a bit unpleased while dealing with image searches. They got

what they are not looking for. This is another thing that GSE has to deal with in order to provide

assurance about its usability.

GSE’s Goo(…)oogle Navigation is not Something Making Much Sense?

GSE provides millions of results for any searched items and let people navigate between pages

using the following image scheme.

It is again something making users unpleased having that much of results and this kind of

navigation. In general, users deal with the first ten result pages at most. Afterward, what is the

meaning of remaining that much of results? None of the participants agreed on that this is

something good; nonetheless, they noted that there is no need for this. Indeed, they experienced

25

and noted that the things listed on last pages are nearly neither here nor there. For these reason,

GSE should limit its number of results page. This does not let people know that GSE is

professional, rather in favor of some people; it is something shadowing GSE’s professionalism.

GSE is Fast Enough?

The participants and my answer to this question are YES to this question. As long as enter is

stroked GSE provides millions of results for the search items. GSE is fast as much as necessary

to index and list the results. Nevertheless, for the user to find something the time consideration is

a bit dissimilar. They also want to come across whatever they want in most probable shortest

time interval. Intended for this, it is better for GSE to revisit indexing and listing strategy to

eliminate irrelevant content and just provide most relevant pages rather than listing more ranked

and linked pages.

GSE is Pleasing?

UTs apprehended by me in this process, interviews with UT participants and my own

experiences authenticated that GSE’s simple interface is something making people pleased not

feared of. Its simplicity should never be hazarded. Obviously, there is always something to do.

GSE may revisit to make its interface much simpler including navigational parts just to let users

have more usable GSE.

GSE Provides Both by Choose and by Chance?

GSE provided millions or even billions of results for a search item. This point make people have

chance of both by choose or by chance. Users may choose the pages, scan the expected results or

just navigating to a page, and scan them than go the further other pages and deal with them. This

26

is not actually the matter of usability, in a sense; yet, this lets people a freedom to navigate over

million results just by chance or choose.

GSE’s Results are Fully Credible?

Given that GSE does not look at the pages’ definite content or the content’s truthfulness, it is

easier said than done to say that GSE provides wholly trustworthy content. GSE should do

something to ensure that the content it provides is correct or et least relevant to the searched

item. On the other hand, careful users just like my participants can scan the results and come up

with expected results.

Other Comments

Here I include my final other comments regarding GSE.

Do it simpler!

GSE may come up with a technology or line of attack that provides result in any common

languages determined by users. Within this, when user enters a search item and stroke the

enter GSE may automatically translate this into the other languages including default

language and, do searches with respect to these and, finally endow with them in different

categories. For instance, when we just enter “Yaşamada Başarı” to the search box and

stroke enter key, GSE should first list items in Turkish, then translate this to another

language, let us say English as “Success in Life” and list the relevant pages to the users. I

believe this is going to perk the UGSE up, in a sense. This is if truth be told a case can be

applied to have more power-driven search engine.

27

Do not Index Anything!

GSE’s another problematic point, in my opinion, is the point that it indexes anything on

the net. It does not show much interest in its content or accurateness. This makes GSE,

sometimes, come up with irrelevant, unbeneficial, hazardous and false information. For

that reason, GSE must come up with a solution for this case ahead of it is too late.

Revisit the Strategy Used in Image Indexing!

As my UTs’ results and interviews and, my experiences demonstrated, GSE fails in

image search. At this juncture, the main reason is the technology or strategy that GSE

uses while indexing images. This strategy or technology makes GSE provide, generally,

irrelevant and unexpected results. Therefore, it is well again for GSE to revisit image-

indexing strategy to assure and add to its usability.

28

Appendix A

Literature Review for Google Search Engine (GSE)

How Does GSE Work and What Make GSE Different?

In this literature review, I provide fundamental information on the subject of how Google Search

Engine (GSE) works and what makes GSE different. While doing so, I refer some literature

studies conducted on this manner. The aim of this paper is let reader know about the working

logic of GSE and its special features making GSE popular. I suppose this understanding is going

to make more sense while conducting and evaluating usability tests not only for me, as project

executor, but also for the Instructor or anyone who will have a look at or interested in my

usability test studies and results abut GSE.

First, we need to answer the question of how GSE creates the index and the database of

documents that it accesses when processing a query.

The logic is explained in GoogleGuide.com web site as follows. GSE runs on a distributed

network of thousands of low-cost computers and can thus carry out fast parallel processing.

Parallel processing is a method of computation in which many calculations can be performed

simultaneously, significantly speeding up data processing.

GSE has three distinct parts:

• The Googlebot, a web crawler that finds and fetches web pages.

• The Indexer, which sorts every word on every page and stores the resulting index of

words in a huge database.

29

• The Query Processor, which compares your search query to the index and recommends

the documents that it considers most relevant.

Unquestionably, here what we are dealing with is how GSE processes a query when we push the

Enter key of our keyboard. Please see Figure 1 to have an idea what is happening just after we

touch the enter key of the keyboard.

Figure 1: How GSE Processes a Query

Moreover, what makes Google special from all other search engines is that it makes use of the

link structure of the Web to calculate a quality ranking for each web page. This ranking is called

PageRank. Second, Google utilizes link to improve search results. Sergey Brin and Lawrence

30

Page explain the logic of these two fundamental concepts in their “The Anatomy of a Large-

Scale Hypertextual Web Search Engine” titled article as follows:

PageRank: Bringing Order to the Web

The citation (link) graph of the web is an important resource that has largely gone unused in

existing web search engines. We have created maps containing as many as 518 million of these

hyperlinks, a significant sample of the total. These maps allow rapid calculation of a web page's

"PageRank,” an objective measure of its citation importance that corresponds well with people's

subjective idea of importance. Because of this correspondence, PageRank is an excellent way to

prioritize the results of web keyword searches. For most popular subjects, a simple text-matching

search that is restricted to web page titles performs admirably when PageRank prioritizes the

results (demo available at google.stanford.edu). For the type of full text searches in the main

Google system, PageRank also helps a great deal.

Description of PageRank Calculation

Academic citation literature has been applied to the web, largely by counting citations or back

links to a given page. This gives some approximation of a page's importance or quality.

PageRank extends this idea by not counting links from all pages equally, and by normalizing by

the number of links on a page. PageRank is defined as follows:

We assume page A has pages T1...Tn which point to it (i.e., are citations). The parameter d is a

damping factor which can be set between 0 and 1. We usually set d to 0.85. There are more

details about d in the next section. Also C(A) is defined as the number of links going out of page

31

A. The PageRank of a page A is given as follows:

PR(A) = (1-d) + d (PR(T1)/C(T1) + ... + PR(Tn)/C(Tn))

Note that the PageRanks form a probability distribution over web pages, so the sum of all web

pages' PageRanks will be one.

PageRank or PR(A) can be calculated using a simple iterative algorithm, and corresponds to the

principal eigenvector of the normalized link matrix of the web. Also, a PageRank for 26 million

web pages can be computed in a few hours on a medium size workstation. There are many other

details which are beyond the scope of this paper.

Intuitive Justification

PageRank can be thought of as a model of user behavior. We assume there is a "random surfer"

who is given a web page at random and keeps clicking on links, never hitting "back" but

eventually gets bored and starts on another random page. The probability that the random surfer

visits a page is its PageRank. In addition, the d damping factor is the probability at each page the

"random surfer" will get bored and request another random page. One important variation is to

only add the damping factor d to a single page, or a group of pages. This allows for

personalization and can make it nearly impossible to deliberately mislead the system in order to

get a higher ranking.

Another intuitive justification is that a page can have a high PageRank if there are many pages

that point to it, or if there are some pages that point to it and have a high PageRank. Intuitively,

pages that are well cited from many places around the web are worth looking at. Also, pages that

have perhaps only one citation from something like the Yahoo! homepage are also generally

32

worth looking at. If a page was not high quality, or was a broken link, it is quite likely that

Yahoo's homepage would not link to it. PageRank handles both these cases and everything in

between by recursively propagating weights through the link structure of the web.

Anchor Text

The text of links is treated in a special way in our search engine. Most search engines associate

the text of a link with the page that the link is on. In addition, we associate it with the page the

link points to. This has several advantages. First, anchors often provide more accurate

descriptions of web pages than the pages themselves. Second, anchors may exist for documents

which cannot be indexed by a text-based search engine, such as images, programs, and

databases. This makes it possible to return web pages which have not actually been crawled.

Note that pages that have not been crawled can cause problems, since they are never checked for

validity before being returned to the user. In this case, the search engine can even return a page

that never actually existed, but had hyperlinks pointing to it. However, it is possible to sort the

results, so that this particular problem rarely happens.

This idea of propagating anchor text to the page it refers to was implemented in the World Wide

Web Worm especially because it helps search non-text information, and expands the search

coverage with fewer downloaded documents. We use anchor propagation mostly because anchor

text can help provide better quality results. Using anchor text efficiently is technically difficult

because of the large amounts of data which must be processed. In our current crawl of 24 million

pages, we had over 259 million anchors which we indexed.

As these two guys explained these two concepts basic logic above, they let the Google work and

continue to work.

33

In order to understand the GSE better, I evaluate it as functional to include the general

information provided in GSE pages related with Google technology.

The technology behind Google's great results

As a Google user, you are familiar with the speed and accuracy of a Google search. How exactly

does Google manage to find the right results for every query as quickly as it does? The heart of

Google's search technology is Pigeon Rank, a system for ranking web pages developed by

Google founders Larry Page and Sergey Brin at Stanford University.

Building upon the breakthrough work of B. F. Skinner, Page and Brin reasoned that low cost

pigeon clusters (PCs) could be used to compute the relative value of web pages faster than

human editors or machine-based algorithms. Moreover, while Google has dozens of engineers

working to improve every aspect of our service on a daily basis, Pigeon Rank continues to

provide the basis for all of our web search tools.

Why Google's patented Pigeon Rank works so well

PigeonRank's success relies primarily on the superior trainability of the domestic pigeon

(Columba livia) and its unique capacity to recognize objects regardless of spatial orientation. The

common gray pigeon can easily distinguish among items displaying only the minutest

34

differences, an ability that enables it to select relevant web sites from among thousands of similar

pages.

By collecting flocks of pigeons in dense clusters, Google is able to process search queries at

speeds superior to traditional search engines, which typically rely on birds of prey, brooding hens

or slow-moving waterfowl to do their relevance rankings.

When a search query is submitted to Google, it is routed to a data coop where monitors flash

result pages at blazing speeds. When a relevant result is observed by one of the pigeons in the

cluster, it strikes a rubber-coated steel bar with its beak, which assigns the page a Pigeon Rank

value of one. For each peck, the Pigeon Rank increases. Those pages receiving the most pecks

are returned at the top of the user's results page with the other results displayed in pecking order.

The tree well defined and explained, I believe, works provided above as to let others and me

have a general idea about GSE. At the end of this report, I have integrated all these three works

completely. They can be viewed and observed in a full manner. As I stated ahead of, these

information will be useful not only for me while conducting and evaluating test with GSE, but is

for instructor or any other interested people to understand this study, Usability Testing of

Google Search Engine, better.

35

Appendix B

Kontrat-Contract

Bu kullanılabilirlik testi BÖTE-440 Bilgisayar Eğitiminde Özel Çalışmalar: Kullanılabilirlik

Testi dersi kapsamında gerçekleştirilmektedir. Test sürecinde ve sonucunda elde edilen bilgi ve

oluşturulan belgeler tamamen eğitimsel çalışma amaçlı olarak kullanılacak ve üçüncü özel ve

tüzel hiçbir kişi ya da kurumla eğitimsel olmayan amaçlarla paylaşılmayacaktır. Aksi hasılda

oluşabilecek tüm rahatsız edici durum ve olayların sorumluluğunu ve oluşabilecek tüm bedelleri

çalışma ve testin sahip ve sorumlusu olan Mustafa DEĞERLİ karşılamayı şimdidedn taahüt

etmektedir. Katılımınız için teşekkür ederim.

This Usability Test (UT) is held for CEIT-440 Special Studies in Computer Education: Usability

Testing course. Any information and/or documents gotten or produced from this UT will be used

just for educational purposes and will not be shared with any people or organization for any

non-educational purposes. However, additionally, any unwelcomed, possible, inconvenient

events’ or cases’ results and costs will be compensated by Mustafa DEGERLI, responsible

operator of this UT. Thank you very much for your contribution.

Not/Note:

Bu sözlemenin bir kopyası katılımcı bir kopyası test sorumlusunda bulunacaktır. Sözleşmenin

her iki lisanda yazılmış hali de aynıdır. Aksi bir durumda katılımcının referans aldığı lisan baz

alınacaktır.

One copy of this document will be provided to the participant and of other copy will be kept by

UT responsible executor. Of the two parts of this document written in different languages do

mean the same things. Otherwise, the one which is preferred by participant will be taken into

account for any further implications.

Test Sorumlusu Test Katılımcısı

UT Responsible Executor UT Participant

Mustafa DEĞERLİ ..………………………..

36

Appendix C

Test Definitions (TDs)

1. Web Search (WS)

1.1 WS 1

1.1.1 Exact thing to find: Dünya’nın Yaşı

1.1.2 Possible-Forced Keywords: Dünya, tarih, yaş, kaç

1.1.3 Language of the Results: Turkish

1.1.4 Site: http://www.google.com.tr/

1.1.5 Criteria: Must be founded in an organizational and/or scientific page. Not in an

unofficial forums or pages.

1.1.6 Allowed Google Features, Restrictions and Directions: Type 1 (See the “Type

1 - Allowed Google Features, Restrictions and Directions” part at the end).

1.2 WS 2

1.2.1 Exact thing to find: General Info about Turkish Armed Forces

1.2.2 Possible-Forced Keywords: TSK, TAF, Türk, Silahlı, Kuvvetleri, Turkish,

Armed, Forces, General, Information

1.2.3 Language of the Results: English

1.2.4 Site: http://www.google.com

1.2.5 Criteria: General Info about Turkish Armed Forces (TAF) must be founded in

TAF’s official web site.



1.3 WS 3

1.3.1 Exact thing to find: Atatürk’ün Devrimleri

37

1.3.2 Possible-Forced Keywords: Atatürk, Devrim, İnklılap


1.3.4 Site: http://www.google.com.tr

1.3.5 Criteria: Information about “Atatürk’ün Devrimleri” must be founded in an

organizational or official web page.



1.4 WS4

1.4.1 Exact thing to find: Küresel ısınma hakkında bilgi veren bir makale

1.4.2 Possible-Forced Keywords: Küresel, ısınma, iklim, değişiklik, Dünya, son



1.4.5 Criteria: An article should be founded. It must be a scientific article. Not a paper

written by someone but by some scientist or expert people.



1.5 WS5

1.5.1 Exact thing to find: An article about Global Warming

1.5.2 Possible-Forced Keywords: Article, Global, Warming, Climate, Change



1.5.5 Criteria: An article should be founded. It must be a scientific article. Not a paper

written by someone but by some scientist or expert people.



38

2. Image Serach (IS)

2.1 IS 1

2.1.1 Exact thing to find: Atatürk’ün Cephede Bir Fotoğrafı

2.1.2 Possible-Forced Keywords: Atatürk, Cephe, Resim, Fotoğraf



2.1.5 Criteria: An image related with “Atatürk’ün Cephede Bir Fotoğrafı” must be

founded. Result must be credible but not necessarily to be from an official web

page.



2.2 IS 2

2.2.1 Exact thing to find: Türkiye’nin Coğrafi Bölgeleri

2.2.2 Possible-Forced Keywords: Türkiye, Coğrafya, Bölge, Kısım



2.2.5 Criteria: An image related with “Türkiye’nin Coğrafi Bölgeleri” must be

founded. Result must be credible but not necessarily to be from an official web

page.


3 - Allowed Google Features, Restrictions and Directions” part at the end)

2.3 IS 3

2.3.1 Exact thing to find: En Eski (İlk) Bilgisayar’ın Fotoğrafı

2.3.2 Possible-Forced Keywords: Bilgisayar, İlk, Eski, Birinci


39


2.3.5 Criteria: An image related with “İlk Bilgisayar’ın Resmi” must be founded.

Result must be credible but not necessarily to be from an official web page.



2.4 IS 4

2.4.1 Exact thing to find: First Computer Picture

2.4.2 Possible-Forced Keywords: Computer, First, Oldest, Image, Picture



2.4.5 Criteria: An image related with “First Computer Image” must be founded. Result

must be credible but not necessarily to be from an official web page.



40

Appendix D

Restrictions and Directions

There are four types of R’s and D’s that I make use of. Details are provided below.

1. Type 1

http://www.google.com.tr will be used. (Figure 1)

“Türkçe sayfalar” option box will be selected.

“Şansımı Denemek İstiyorum” option will be allowed.

Figure 1: GSE Turkish Interface

Users will be allowed to use some features of “Gelişmiş Arama” (Figure 2).

The allowed “Gelişmiş Arama” options are highlighted below which are “kelimelerin

hepsi”, “aynen girildiği gibi”, “kelimelerden herhangi biri”, “bu kelimeler hariç”

There are two imposed-fields which will be set as follows. 10 result per page “Sonuçları bul

� 10 sonuçlar”and lanquage as Türkçe (Dil � Sonuç sayfaları dili: Türkçe.

41

No other features and or ways to search are allowed.

Figure 2: “Gelişmiş Arama” Options

2. Type 2

http://www.google.com will be used. (Figure 3)

“I’m Feeling Lucky” option will be allowed.

Users will be allowed to use some features of “Advanced Research.”

42

Figure 3: GSE English Interface

The allowed “Advanced Research” options are highlighted below which are “with all of the

words”, “with the exact phase”, “with at least one of the word”, “without the words”

There are two imposed-fields that will be set as follows. 10 result per page “Find Results �

10 results” and language as English (Language � Return pages written in: English.


43

Figure 4: Advanced Research Options

3. Type 3

http://images.google.com.tr will be used. (Figure 5)

Figure 5: GSE Image Turkish Interface

44

Users will be allowed to use some features of “Gelişmiş Resim Arama.” (Figure 6)

The allowed “Gelişmiş Resim Arama” options are highlighted below which are “tüm

kelimelerle ilgili”, “tam cümlecik ile ilgili”, “kelimelerden herhangi biri ile ilgili”,

“kelimelerle ilgisiz”


Figure 6: Advanced Image Research Options

4. Type 4

http://images.google.com will be used. (Figure 7)

45

Figure 7: GSE Image English Interface

Users will be allowed to use some features of “Advanced Image Search.” (Figure 8)

The allowed “Advanced Image Search” options are highlighted below which are “related to

all of the words”, “related to the exact phase”, “related to any of the words”, “not related

to the words”


Figure 8: Advanced Image Research Options

46

Appendix E

Specifications of the Computer to be Used in Tests

I provided to UT participants a computer which had at least the following features and

specifications.

The specifications of the computer are listed below:

256 MB RAM

Microsoft Windows XP Professional OS + SP2

Microsoft Internet Explorer 7.0 Browser

64 MB Graphic Card

17 inch LCD Monitor

Intel Pentium IV, 2.66GHz Processor

20 GB Free Disc Space

Q Keyboard

Optic Mouse

Internet Connection

No other non-core applications are run during Searches.

47

Appendix F

PF.rar

mustafa degerli - 2007 - usability testing of google search engine

Design

usability of gse ugse

ut of gse

test items

results of utgse

time results

usability testing ugse

aspects of usability

term of usability