introduction to web scienceslide 1 of 51 what turns an area into a science? why is it „web...
TRANSCRIPT
Introduction to Web Science Slide 1 of 51 http://west.uni-koblenz.de
What turns an area into a science?
Why is it „Web Science“ and not „Web practice“ what we try to learn/teach/research here?
Web Science & Technologies
University of Koblenz ▪ Landau, Germany
Web Observatory
Steffen Staab
Introduction to Web Science Slide 3 of 51 http://west.uni-koblenz.de
Doing Science on the Web - Methods
Observations Web Data
• Snapshot– Google Cache– Your own crawl
• Diachronous– Internet Archive
Introduction to Web Science Slide 4 of 51 http://west.uni-koblenz.de
Wayback machine
Introduction to Web Science Slide 5 of 51 http://west.uni-koblenz.de
Web Graph
Slides by Maren van Stehen
Full slide deck and free book on
„Graph theory and Complex Networks“ available from
http://www.distributed-systems.net/gtcn
Introduction to Web Science Slide 6 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 7 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 8 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 9 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 10 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 11 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 12 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 13 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 14 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 15 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 16 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 17 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 18 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 19 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 20 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 21 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 22 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 23 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 24 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 25 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 26 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 27 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 28 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 29 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 30 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 31 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 32 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 33 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 34 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 35 of 51 http://west.uni-koblenz.de
Issues with crawling
Issues:duplicate pages (available under different URLs)deep web pagestemporarily available pagesclosed / semi-public pagesRestricted content
no robot Republishing
• Even Facebook-shares may lead to written warnings („Abmahnung“) with fees
Introduction to Web Science Slide 36 of 51 http://west.uni-koblenz.de
Other Observation Efforts
Introduction to Web Science Slide 37 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 38 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 39 of 51 http://west.uni-koblenz.de
Linked Open Data Cloud
Introduction to Web Science Slide 40 of 51 http://west.uni-koblenz.de
Web Science & Technologies
University of Koblenz ▪ Landau, Germany
Observing Users
Steffen Staab
Introduction to Web Science Slide 42 of 51 http://west.uni-koblenz.de
Doing Science on the Web - Methods
Observations Web Data
• Snapshot– Google Cache– Your own crawl
• Diachronous– Internet Archive
User Data• Web Site Operator
Introduction to Web Science Slide 43 of 51 http://west.uni-koblenz.de
Server Logs
Apache Common Log FormatExample:127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
127.0.0.1 (%h)IP address of the client (remote host) which made the request to the server (may be proxy!). - (%l) identity of client not availablefrank (%u)userid of the person requesting the document as determined by HTTP authentication
Introduction to Web Science Slide 44 of 51 http://west.uni-koblenz.de
Server Logs
Apache Common Log FormatExample:127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
[10/Oct/2000:13:55:36 -0700] (%t)The time that the request was received. "GET /apache_pb.gif HTTP/1.0" (\"%r\“)The request line from the client 200 (%>s)status code that the server sends back to the client. 2326 (%b)size of the object returned to the client
Introduction to Web Science Slide 45 of 51 http://west.uni-koblenz.de
Server Logs
Apache Combined Log FormatExample:127.0.0.1 - frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "http://www.example.com/start.html" "Mozilla/4.08 [en] (Win98; I ;Nav)“
The additional fields are:"http://www.example.com/start.html" (\"%{Referer}i\“)The "Referer" (sic) HTTP request header. "Mozilla/4.08 [en] (Win98; I ;Nav)" (\"%{User-agent}i\“)The User-Agent HTTP request header. Identifying information that the client browser reports about itself.
Introduction to Web Science Slide 46 of 51 http://west.uni-koblenz.de
AOL Search Query Log
Introduction to Web Science Slide 47 of 51 http://west.uni-koblenz.de
Introduction to Web Science Slide 48 of 51 http://west.uni-koblenz.de
AOL Query Log Mirrors
Introduction to Web Science Slide 49 of 51 http://west.uni-koblenz.de
Doing Science on the Web - Methods
Observations Web Data
• Snapshot– Google Cache– Your own crawl
• Diachronous– Internet Archive
User Data• Web Site Operator
• User Experiments (cf. Indexing quality)
Introduction to Web Science Slide 50 of 51 http://west.uni-koblenz.de
Current Research Challenge
Web Observatory Analogy to
Virtual Observatories
What should it include?
Web Science & Technologies
University of Koblenz ▪ Landau, Germany
Predicting Behaviour
Steffen Staab
Introduction to Web Science Slide 52 of 51 http://west.uni-koblenz.de
Example problems of predicting people behavior
Politics Rules and laws Social welfare
Economics Buying behavior Unemployment
Social science (in a more narrow sense) birthrates