counter: towards reliable vendor usage statistics progress to date (june 2004) peter shepherd...
TRANSCRIPT
COUNTER: towards reliable vendor usage statistics
Progress to date (June 2004)
Peter ShepherdProject DirectorCOUNTER
SSP 26th Annual Meeting, June 2004
Background
Goal: credible, compatible, consistent publisher/vendor-generated statistics for the global information community
Libraries and consortia need online usage statistics
To assess the value of different online products/services To support collection development To plan infrastructure
Publishers need online usage statistics To experiment with new pricing models To assess the relative importance of the different
channels by which information reaches the market To provide editorial support To plan infrastructure
COUNTER: strategy
Respond to the requirements of the international librarian, publisher and intermediary communities
An open, inclusive and interactive process Representation of all three communities on COUNTER
Limit scope of Release 1 to journals and databases Systematically extend scope of the Code of Practice
Horizontally, to cover other content types, such as e-books Vertically, to provide more detailed statistics on journals
A cost effective-process for all parties involved
Code of Practice, Release 1:Main Features
Definitions of terms used Specifications for Usage Reports Data processing guidelines Auditing Compliance Maintenance and development of the Code of
Practice Governance of COUNTER
Summary
COUNTER Code of Practice released January 2003
Which vendors are now compliant?
What issues and challenges has compliance raised?
Results of library and vendor research
Auditing
COUNTER Code of Practice Release 2
Priorities for 2004 and beyond
Compliant vendor growth
Compliant vendors Jan 2003: 2 Compliant vendors Dec 2003: 12 Compliant vendors May 2004: 30
Over 50% of annual total of new articles covered by ISI now come from COUNTER compliant publishers.
Register of Compliant Vendors
Achieving compliance - hurdles and issues
Cost of development
Concern re fulltext request count reductions
Need for more guidelines e.g. searching, sessions, timeouts, overlapping IPs
Need for a “Guide to interpretation” of COUNTER reports? Like National Rail or BookScan
Testing the COP via market research
Library Focus Groups:- San Diego 14 September 2003 London 16 September 2003 Elsinore 23 October 2003 Charleston 6 November 2003
International Advisory Board Round Table, London Dec 2003
Library pilot testing programme from February 2004:- Cornell University Cranfield University GlaxoSmithKline University of California University of Leicester
le
What the market research has told us
PUBLISHERS If you want to maximise compliance, don’t
make it excessively complex, demanding, and expensive - and don’t keep changing it!
LIBRARIES Keep reports simple and basic Postpone development and implementation
of Release 2 till we have more feedback on Release 1
Put future releases in draft form on the web for a period of public comment
What the market research has told us
“Level 2 reports, especially Journal Report 3, contain too much data to be useful”
“Two levels of compliance are unnecessary” Add a “Publisher” column (helpful with
aggregator reports) Allow removal of zero usage journals in
aggregator reports Divide Table of Definitions into 2, separating
terms used in the reports from the rest One report per file please No punctuation in data ISO date format
What the market research has told us
Preconstructed reports preferred to “on the fly”. One user found only 25% of usage reports generated live actually worked.
Consortia need a way to derive an aggregated summary report cf. compiling it institution by institution.
Make clearer the protocols for measuring usage when intermediaries are involved
Source ofpage
Responsibility forrecording usageand reporting tocustomer
Reportzerousage
Comments
Direct fromvendor’sserver
Vendor Yes Delivery of content to the user is from thevendor’s own service/site, to which the userhas direct access.
Direct fromanaggregator
Aggregator No Delivery of content to the user is from anintermediary (a gateway that is also a host),using its own store of publishers’ content.Gateway is responsible for recording andsupplying usage statistics for full-textrequests direct to the customer and also,where contractually permitted to do so, tothe vendor. (In this case the vendor may notadd the ‘gateway’ usage figures to thoserecording usage of content delivered by thevendor direct to the customer)
Referral fromanaggregatoror gateway
Vendor Yes Delivery involves the gateway sending theend user from the gateway’s site to thevendor’s site for the requested content.Vendor is responsible for recording andsupplying full-text usage statistics to thecustomer. Gateway may also supply usagestatistics to the customer, but must reportthem separately from those covering itsdelivery of full-text direct to the customer
Via agateway
Gateway No Delivery of content is via a gateway, whichrequests the content from the publisher anddelivers it to the user in the context of thegateway service. Responsibility forcollecting and supplying usage statistics tothe customer is the responsibility of theGateway.
Referral toanaggregatoror gateway
One of Vendor,Aggregator orGateway
In this case an index or abstract servicerefers the customer to the gateway for full-text. In this case the full-text is deliveredaccording to one of scenarios listed above,and the recording and supplying of usagestatistics to the customer is as specified ineach of these cases.
What the market research has told us
Provide a toolkit to allow customers to combine usage statistics from different vendors automatically
Issues Role of COUNTER cf agents, library systems suppliers, and
libraries themselves. Role of XML DTD (machine readable)
Develop a separate COP for e-Books and reference works
Audit is critically important – should be credible but not so rigorous it causes publishers to raise prices
What the market research has told us
Library test sites
Monitor compliant vendors and highlight problems. Results include:-
Some differences between compliant submitted reports and actual ones (e.g. different number of columns, ISSNs with leading zeros missing)
Difficulties locating and identifying COUNTER reports amongst others
Lack of historical data for comparison limits usefulness initially
Problem of knowing when compliant data starts. Add “Compliant from” column to the register
Suggest ‘product’ rather than ‘vendor’ be ‘compliant’
Auditing Auditing by a Chartered Accountant (UK), a
Certified Professional Accountant (USA), or its equivalent elsewhere, or by another, suitably-qualified COUNTER-approved auditor, will be required to validate the usage reports
Auditing principles have now been agreed Draft test scripts have now been written and are
being discussed by the Audit Task Force with professional feedback from Deloitte & Touche and others
RFPs for other COUNTER-approved auditors ready to go out to candidates once scripts finalised.
Taken longer than anticipated – complexity revealed as scripts written
COUNTER COP Release 2
Published April 2004 in draft and placed on the website for six months for comment
Specific questions asked in an introduction Should the definition of ‘turnaways’ be broadened?
Final version Jan 2005 and valid version Jan 2006. More prescriptive re formatting Make changes minimal cost as far as possible A new Journal Report 1a, which reports usage
statistics for html and PDF full-text requests separately, but with “health warning”.
A Table of terms and definitions specifically relevant to the Usage Reports contained in Release 2.
Priorities for 2004
Publish draft of Release 2
Solicit feedback on Release 2 draft
Implement auditing
Publish e-Books draft COP mid 2004
Encourage and assist growth in compliance
Reach target of 150 members
COUNTER Membership
Member Categories and Annual Fees
Publishers/intermediaries: £500 Library Consortia: £333 Libraries: £250 Industry organization: £250 Library affiliate: £100 (non-voting
member)
Benefits of full membership Owner of COUNTER with voting rights at
annual general meeting, etc. Regular bulletins on progress Opportunity to receive advice on
implementation
Membership at March 2004
Members May 2004: 130
Vendors 35%
Libraries 28%Consortia 25%Industry Orgs 10%Lib affiliates 2%
Target for 2004: 150 Join us too and help influence the future
development of usage reporting standards
For more information……….
http://www.projectcounter.org
Thank you!
Peter Shepherd (Project Director)[email protected]