w w w. n e s s t a r. c o m building a data library or data observatory on the web using nesstar...

w w w . n e s s t a r . c o m

Building a Data Library or Data Observatory on the Web

using Nesstar technology

Jostein Ryssevik

Margaret Ward

Cliff Dive

IASSIST/IFDO 2005

Edinburgh, Scotland

Workshop 4


Agenda

09.30 – 10.00 Overview of the Nesstar technology

10.00 – 10.30 Install and configure your server

10.30 – 10.45 Break

10.45 – 11.15 Install the Publisher and connect it to the Server

Customize your data library

11.15 – 11.45 Populate your data library

11.45 – 12.00 Break

12.00 – 12.30 Federate your data libraries


Nesstar - vision

To develop a truly distributed platform for electronic publishing of statistical data, building on object technology, open (metadata-) standards and lightweight Internet

protocols.

....or simply

To bring the models, technologies and collective energy of the Web to the world of statistics.


Nesstar - background

• Originally funded by EC under the Electronic Publishing Program

– NESSTAR (1998-1999), 4th Framework Program

– FASTER (2000-2001), 5th Framework Program

• Nesstar Limited established in June 2001

– Owned jointly by The Norwegian Social Science data Services (NSD) and the UK Data Archive (through the University of Essex)

– Offices in Colchester (UK), Bergen (Norway) and Ottawa (Canada)


Solutions for a variety of sectors

• Nesstar provides solutions for a variety of sectors and has already a fast-growing international client-base within areas like:

– Health (Health Canada, Norwegian Institute of Public Health, erpho)

– Transport (Transport for London, German Aerospace etc.)

– Official statistics (Statistics Canada, Statistics South Africa)

– Market and public opinion research (NORC (Chicago), Gallup Europe (Brussels))

– International organizations (World Bank, WHO, International Household Survey Network, African Census Project)

– Universities and independent research organizations (approx 25 universities in Europe and US/Canada)

– Data archives (most of the European data archives)

• Any organization with a need to share data internally among its experts and decision makers, or externally to a wider audience, is profiting from Nesstar’s technology.

• Nesstar is true data liberation


RetrieveLoad and manage

Metadata and data input

ExtractMetadata and data editing/transformation

Data and metadata retrieval and display

Nesstar – a bird’s eye view

Internett


Nesstar 3.0 - highlights

• Provide access to all your data through a single system: Unlike most data publishing systems, Nesstar 3.0 supports micro-data (rectangular and hierarchical) as well as aggregated data (multidimensional tables or cubes) and knowledge products.

• Let your data “sing and dance” on the Web or the corporate Intranet in hours: Rapid server installation and configuration, combined with easy and efficient data publishing through the Nesstar Publisher, provides the shortest possible route to the Web for the data resources within your organization.

• Manage servers and publish data over the Web: With version 3.0 of Publisher - the ETL-tool of the Nesstar product suite – you can publish data and manage your servers over the Web. Simultaneous publishing of data to multiple servers is also supported.

• Avoid expensive installation and maintenance of software components on end-user desktops: With Nesstar WebView – the new and innovative end-user client - advanced and powerful data analysis capabilities are delivered through a standard Web-browser.

• Release the creativity of your data users: Nesstar Webview offers a truly unique and intuitive interface to statistical data allowing your data users and analysts to focus on content and the creation of knowledge rather than techniques.

• Experience the speed and elegance of a truly efficient data analysis system: Robust and scalable components from top-to bottom secure remarkable performance also in situations with high volumes of data, high number of users and inexpensive hardware.


Nesstar 3.0 – highlights (cont.)

• Protect your data against unauthorized use: Version 3.0 of Nesstar comes with an even more flexible access control system allowing you to define access conditions down to the finest details. Even individual variables, metadata elements or statistical operations can have their own policy.

• Customize the end-user interface to fit the graphical profile of your organization: Nesstar 3.0 comes with configurable and replaceable GUI elements that easily will let the interface melt into the organization’s web-design. Further customisation can be achieved my amending templates.

• Integrate Nesstar with the Web-systems of your organization: Nesstar is from top-to-bottom a fully web-enabled technology building on lightweight web standards and protocols

• Manage your data assets: Nesstar is building building on rich metadata standards (DDI, Dublin Core etc.) supporting intelligent knowledge management and retrieval.


The Workshop CD• The Workshop CD contains all the software and

documentation you will need for the two Nesstar workshops.

• When loading the CD, a welcome page will be displayed in the browser . The welcome page includes links to installations and documents.

• If the welcome page do not open automatically, use the File Explorer (My Computer) to open the CD. Click on the Index.html file in the root folder to load the Welcome page.

• You can also start the installers and copy files directly from the CD using File Explorer.


Install and configure your server 1

• The aim of this hands-on session is to install the Nesstar server from the CD onto your workstation.

• You will also learn how to configure the server to meet your needs.• By the end of the session you will have a running server ready for look and

feel customisation and resource publishing


Install and configure your server 2• Task 1: Install your server

– Click on the Install Server link on the CD’s front-page or alternatively, use File Explorer to find the file Nesstar-Server-3.08b.exe under the catalogue Server on the CD and double-click on this file.

– This will start the preparation of the installation process.

– Click through the first few dialogues until you come to the Installation folder window. Just accept the proposed folder by clicking Next.

– Click Next on the Installation Option windows, and then Install on the Start Menu Folder windows.

– The installation of the server is now starting. The process might take a couple of minutes. Just wait until the Nesstar Server Configuration Tool pops up on your screen.


Install and configure your server 3• Task 2: Configure your server

– You should now see the Nesstar Server Configuration Tool on your screen.

– First you will have to enter the License Code that authorizes you to use the Nesstar technology. You will find a 30 days license key in the Resources directory on the CD. Open the document in Notepad and use cut and paste to enter the key into the Configuration Tool.

– You might give your server a new human readable name (no blank spaces, just alphabet characters, the name is case sensitive).

– Don’t change HTTP Port 80 and the IP address proposed by the Configuration Tool. REMEMBER to write down the IP-address. You will need it later.

– Choose a password for the administrator (and please REMEMBER it). The password is case sensitive.

– Make sure that the option Launch server welcome page is ticked. When done, click Apply settings and the configuration will be completed.

– The process might take a couple of minutes. You know that all is well when a browser pops up showing the Server Welcome page.

Before clicking Apply settings you might take a look at the options under Advanced mode. You should however not change any of the advanced settings. Note also that various aspects of WebView can be configured by clicking WebView settings.


Install and configure your server 4

• Task 3: Take a look at your server– You should now see the Server Welcome

page displayed in the browser.– The page has three essential links:– Nesstar WebView will let you browse

your new server using the WebView client.

– User Management Tool is a client that can be used to manage users and their rights as well as passwords.

– Technical resources opens another web page where you can get access to a variety of documentation and tools, amongst other the Nesstar Object browser that you will use later in this workshop.

– Open Nesstar WebView and take a look at your brand new server.


Customize your data library 1

• In this session you will learn how to customize the look and feel of your data library.

• You will learn how to change the top banner, create a front page and create descriptive pages for your various data catalogues.

• To achieve this, you will need the Nesstar Publisher, so installing the Publisher is also part of this task.



• Task 1: Install the Publisher– Click on the Install Publisher link on the CD’s front-page or

alternatively, use File Explorer to find the file NesstarPublisherInstaller_v3.06.EXE under the catalogue Publisher on the CD and double-click on this file.

– This will start the installation of the Publisher. Just click Next a few times and finally Finish and it should all be done.

– When prompted, copy the Publisher License key from the CD and paste it in.

• Task 2: Start the Publisher and connect to the server– Start the Publisher from the Windows Start menu or from the

Icon on the desktop– Click on Publishing in the top menu-line and then Add

server. – In the dialog, enter the IP-address of your server– Then enter the Userid (admin) and Password– Press OK– Again click on Publishing in the top menu – Your sever should now be visible in the list.



• Task 3: Create one or more data catalogues– Click on Publishing in the top menu of Publisher,

click on the name/IP of your server and then choose Manage server. This will open a dialog looking like the one to the right

– Highlight the yellow folder (which is the root catalogue of your content tree), and hit the + sign to the right to create a sub-catalogue. Give the catalogue a name

– Create a few more catalogues and note that they can be arranged in a hierarchy like in File Explorer.

– Use WebView to see the changes you are making to your server. Remember to refresh the browser to see the updates.

– Note that you can create a front-page for each catalogue by adding HTML-code to the Comment field at the bottom of the Manage Server dialog box.

Example:<center><H2>Health data</H2>This catalogue contains Health Surveys from various countries. </center>


Customize your data library 4• Task 4: Modify the top banner

– Open the Server Configuration Tool from the Windows Programs menu. Choose Customisation in the top-menu, then Customise and finally WebView Header Source.

– Click on HTML source and observe a dialog looking like the one below.– Find the line with the reference to the image file Nesstar_World.gif and change this

filename to Left_topbar.gif. Keep the path as is.– Then copy the file Left_topbar.gif from the Resources directory of the CD into the

following directory on your server: …\jboss\server\default\deploy\webview.war\skins\default\images

- Check in Webview that the result looks OK


Customize your data library 5• Task 4: Modify the Welcome screen

– Open the Server Configuration tool from the Windows Programs menu. Choose Customisation in the top-menu, then Customise and finally WebView Welcome Source.

– Click on HTML source and observe a dialog looking like the one below.– Find the line with the reference to the image file nesstar_open.gif and change this

filename to welcomepage.gif. Keep the path as is.– Then copy the file welcomepage.gif from the Resources directory of the CD into the

following directory on your server: …\jboss\server\default\deploy\webview.war\skins\default\images

- Check in Webview that the result looks OK


Populate your data library 1

• The aim of this hands-on session is to learn how to populate your on-line library with data and other resources

• This is all done through the Nesstar Publisher and various test-data are available on the CD.

• Please note that nuts and bolts of data publishing will be explored in further depth in the follow-up workshop.



• Task 1: Publish a fully documented datafile to your server

– Start the Publisher from the Windows Programs menu, click on the File menu and then Open.

– In the Open dataset dialog, find the Resource directory on the CD and click on the file called demo-bes87.nsdstat. This is a small test-dataset from the British Election Study 1987.

– When the file is loaded, click on Publishing in the top menu, then on the name/IP of your server and finally on Publish data and metadata

– In the dialogue that now opens, check the catalogue where you want the file to be published and click Publish. Note that a single file can be placed in more than one catalogue. The publishing process may take a minute or two.

– Look for the dialog to the right to pop up– Click Open in Web client to test the result– Try publish other nsdstat-files from the CD



• Task 2: Publish an SPSS-file to your server– In Publisher, click on the File menu and then on Import dataset.– In the Import dataset dialog, find the Resource directory on the CD and click on the file

called 1991 U.S. General Social Survey.sav. This is one of the test-datasets distributed with SPSS.

– Note that when the file loads, Publisher is calculating statistics for the variables. The statistics are added to the metadata.

– Open the Study Description tab in the Publisher and give the study a name and an ID.– Fill in other metadata fields if you want to.– Open the Variable Groups tab and see if you are able to group the variables to make it

easier to navigate the dataset.– Publish the dataset to the server and check in WebView if it looks OK.– Make changes in Publisher and republish if necessary.– Try publish other SPSS-files from the CD


Federate your data libraries 1

• In this last hands-on session you will learn how to interconnect your servers to create a distributed virtual data library.

• By the end of the session we will probably have several partly overlapping networks available allowing you to search for data across servers and to explore the data and resources published by your neighbours.


Internet

Nesstar allows you

to build an integrated

data portal/archive even in situations

where data are distributed

and maintained on several distributed

servers




• Task 1: Find the name of your server object

– One way to find this name is to use the Nesstar Object Browser. This is a simple Web client that allows you to browse through the object hierarchy of your server. The Object Browser is also allowing you to view the properties of the objects and to invoke object methods.

– For this task we will only use the Object Browser to find the name of the object that sits at the top of the hierarchy, the server object.

– Enter the IP-address of your server into the address line of your browser. This will open the Welcome page of your server.

– Click on the link called Access to other Technical resources. On the next page under the section called Useful Tools, click on one of the links called browse objects.

-You have now entered the Object Browser and you will see the name of the server object in brackets in the very first line of the page (starting with http://….). -Keep the page open or make a note of the name. In the next task you will hand this name over to your neighbour(s). Please note that the name is case sensitive.

Federate your data libraries 4• Task 2: Edit the script that links in

external servers– Use the File Explorer (click on My

Computer in Windows) to find the Nesstar server directory on your workstation. Find the sub-directory called config, and under this directory the file called localdb.rdf. Open this file using Notepad.

– The bottom of this file looks like the extract to the right (top). Ignore the section at the bottom that is commented out (in blue) and focus on the section in red. This one is pointing to your own server

– To link in other external servers, you will simply have to duplicate this section, one for each external server.

– First, duplicate the section using copy/paste.

– Second, in the section you just pasted, replace the text between the quote marks with the name of the sever object of your neighbour’s server (Task 1). See the example to the right (bottom).

– Repeat the last two points for each external server you will link in.

– Save the localdb.rdf file

<c:Server rdf:about="&url;cServer/MyServer"><rdfs:label>MyServer</rdfs:label><rdfs:comment>The Open MyServer Server.</rdfs:comment></c:Server>  </rdf:RDF>

<c:Server rdf:about="&url;cServer/MyServer"><rdfs:label>MyServer</rdfs:label><rdfs:comment>The Open MyServer Server.</rdfs:comment></c:Server>

<c:Server rdf:about=“http://192.168.10.10/obj/cServer/MyWorld"><rdfs:label>MyServer</rdfs:label><rdfs:comment>The Open MyServer Server.</rdfs:comment></c:Server>  </rdf:RDF>



• Task 3: Restart your server– In order for the changes you just made to take effect, you will have to restart your

server. – Use the Programs menu in Windows, find the Nesstar server and then the Stop

program. You will also find this Stop program in the top directory of the Nesstar server (on your disk).

– Stop the server (this will take at least 10 seconds).– Then start your server (this might take a couple of minutes).– When the Welcome page pops up, open WebView and observe that your browse-

list now includes more than one server. – Congratulation, you have just created your first distributed virtual data library!



• Task 4: Play with your neighbours servers– Browse your neighbour’s server. Do some analysis. – Search for data across servers.– Open the advanced search dialog and notice that you now can decide which

servers to include in your search.– Publish data to your neighbour’s server.

w w w. n e s s t a r. c o m building a data library or data observatory on the web using nesstar...

Documents

data library

data observatory

data libraries

data sing

data resources

n e s s t

efficient data publishing

data publishing systems