DIY Search ToolsPractical Applications of Sierra DNA and REST API

Cameron Schrode – ILS Administrator

cschrode@akronlibrary.org

Cheryl Engel - Technical Services Manager

cengel@akronlibrary.org

Akron-Summit County Public Library

Introduction

• Sierra DNA (SQL)• Cannot edit or add records

• Useful for Decision Making and Routines

• Today's Examples:• Community Engagement Statistics

• On Order with Holds

• eNewsletter Subscribing

• Sierra API (Programmatic)• Fetch, Edit, and Add Records

• Useful for Scheduled Tasks, Rapid Update, Complex I/O

• Today's Example: Email Address Repair

Business Intelligence

• What is business intelligence?1. Metrics and Benchmarks

2. Measure Input / Output of Processes

3. Reporting

4. Collaborative

5. Knowledge Management

6. (optional) Real-Time Data

• Metabase• Alternatives

• Tableau

• Power BI for Office 365

• pgAdmin

Dashboards• Acquisitions / Cataloging metrics

• Branch Audience reports

• Cardholder metrics

• Circulation metrics

• Demographic metrics

• Community Engagement

• Marketing Campaign effects

Components• Question => Answer

• Collection

• Information Architecture

• Dashboard

• Visual Representation

• Pulse

• Reporting and Distribution

Metabase

Simple Reports: Tablesit Statistics

• Goal1. Metrics and Benchmarks

• Patron Count, Waives, Waived Amount, Registrations

2. Measure Input / Output of Processes

• Geographically, over time, by branch

3. Reporting

• Process• Create a new patron template and Sierra login for each branch's tablesit

• Make the user & template identical to their in-house one

• Add a unique note field

• Use tablesit logins for tablesits

• Harvest info from SQL based on Sierra username

• Create dashboards to compare and contrast between branches

Other Reports

• Generalized queries can be used

• For example:

• No. of Fines Waived

• Cost of Fines Waived

• No. of Patrons Assisted

• Next Steps: Community Engagement

• Which communities are active

• Potential locations for table sits

• Marketing locations with less activity

• Staffing tablesits according to potential

• Registration drives, points and prizes

• Board reports and advocacy

Transactional Reports: On Order w/Hold

• Goal1. Collaborative, multi-purpose data model

• Either by Create List file or manually specifying record IDs

2. Knowledge Management• Combines disparate data points into practical knowledge

3. Real-Time Data

• Process• Create a user interface (Submission Form), data model (Metabase Question),

and simple file processor (programming language of choice)

• File processor uses information from Create List export to feed the model

• The model plugs the information into a SQL query and prepares the result

• The user interface takes the file from a browser and passes it to file processor

Usage

• Use Python, PHP, or Perl to create Transactional Reports

• i.e. Data Input, Data Export

• Process for On Order with Holds

1. Make Sierra Create List export (see pic)

2. Submit to PHP script

3. PHP reads, space-delimits bib IDs

4. Redirect, feeding bib IDs to Metabase

5. Metabase inputs to SQL

6. Metabase feeds results back as XLSX

Result

Workflow for Data Science Projects

1. Define your question

2. Define your ideal data set, brainstorming must have / nice to have data points

3. Determine what data you can access

4. Obtain the data

5. Clean the data

6. Explore the data set, either with the end user or by understanding in-depth

7. Statistical prediction and modeling

8. Interpret the results

9. Challenge the results

10. Synthesize and write-up results

11. Create reproducible code

• Use the quantity to determine urgency or quality of service.

• Reports without action taken should be eliminated or re-purposed.

• Respect privacy! Anonymize your data sets, especially if sent by email.

• Know Your Audience. If you don't know enough about the problem-space, make time to learn!

• Charts and graphs are great for trending and comparisons.

• Always be on the look-out for improvements to queries, take advantage of version control.

• Transparency is key! More eyeballs on the report leads to more accurate reports.

Tips for Synthesizing Results

How We Handle Transparency

Problem Space• Begins with data entry & human fallibility

• Timeliness is key

• Branch-specificity and hyper-localization

• Deals with the heart of our mission: Programs & Services, Branch-Specific Collections

• Must be simple for business continuity

• Must be smart enough to correct bad data but also learn by human supervision

• Multiple systems / APIs, basically Systems Integration

Complex Report: eNewsletter Auto-Enroll

Basic Process Outline

1. Use SQL to collect name, email addresses, home library code if notice preference is Email

2. Feed email addresses through Repair Tool (see 2017 DIY Search Tool on IUG Website)

3. Enroll to General Mailing List for everyone via MailChimp API

4. Enroll to Branch Specific Mailing List by tying home library code to mailing list ID

Initial Questions to Ask

What version of Sierra are we on?

• Sierra API is available for Sierra ~2.4+

Do we have a capable server?

• Requirements:• PHP & php-zip, php-pgsql, php-curl, php-xml modules

• Composer (http://getcomposer.org/)

Can we write our own?

• Sure! If you use PHP, our framework is easy to plug-in queries and logic

• If you're using a different language, use the process breakdown on the "Process" slide

Autocorrect Process / Algorithm• Add domain to "known good" list if MX record exists, do not prompt for change

• If no MX record detected:1. Auto-correct mistakes that hamper analysis (e.g. comma to period and 2 to @)

2. Run similarity checks against "known good" list

3. Auto-correct to an exact match, if the host is already in "known good"

• If similarity check doesn't match any host exactly:1. Return top two matches, along with "other" to enter your best guess & "keep" to skip record

• If email was (auto-)corrected:1. Add user ID and corrected email address to "changed patrons" list

• Finally:1. Cycle through "changed patrons" list

2. PUT patron/[id] - no brackets - with the 'emails' field populated by list (array) of corrected email address(es)

3. (Archive &) Clear "changed patrons" list once all patrons are finished

Autosubscribe Process / Algorithm• Check each record for Home Library Code, assign each Branch a yes or no for its List ID

• General Interest / Global Mailing List1. * = Yes by default, enroll to the General Interest List

• Branch Interest1. Toggle the Yes/No field for the interest of branch

• Finally, submit information through MailChimp API:1. Cycle through patron list with associated branch interests

2. POST list/[list-id]/members - no brackets

1. Status = subscribed

2. Email_Address = email

3. Email_Type = html

4. Merge_FIelds = (FNAME => First Name, LNAME => Last Name)

5. Interests = ({LIST_ID} => True/False)

3. POST list/[list-id]/members/[md5sum(lowercase(email)))]/notes

1. Note = AutoSubscribed (vYYYY.MM.DD)

Working Repairs• test@example,ocm, test@exa...

• test@example,ocm

• test2example.com

• test2#example.com

• test example.com

• test@example .com

Output• Pending Changes

• Common misspellings

• Quality assurance dataset

• Long-term memory

• Known hosts

• Quality assurance dataset

• Short-term memory

Quirks• Not all MX records are good

• Popular hosts learn fastest

• Internal DNS maybe filtered

• Big dataset is more thorough

• Defunct companies

Learning API• Obtain "API Key" from Admin Corner

• Use config.json template

• Use developers.iii.com to find API endpoint

• Fill in config.json key and secret

• Use the sandbox (Swagger) to figure out fields and values for desired outcome

• Try to replicate GET queries in scripts

• *Remember* GET queries do not modify records

• Everything else will modify records

Sierra API

• https://developer.iii.com/• Requires registration

• Fantastic for understanding these modules

• http://bit.ly/ascpl-2016-iug• Configurable swiss-army tool to get running with SQL and API quickly (if using PHP)

Online Resources

Questions?

Technical + ServicesHow APIs make workflows better for our customers

Technical Services at Akron Summit County Public Library• Sierra Library since 2007

• Receive, invoice, catalog and process 15,000 items each month

• Order 500-700 titles each week

• Manage and prioritize the workflows for 3000-5000 items each day (60% arrive with holds)

Before the Widget

• Difficult to sort incoming items by demand

• Public Service Staff wanted to know what was going to arrive each week from Tech Services

• Need to anticipate what was coming to plan workflows

• Need to know which and whether orders were backordered too long

• Need materials for featured lists on our website

• Difficult to find this information using create lists

The Solution : Cam’s Widget

How it works for me: (1-2-3)

1. Create a list of desired bibliographic records

2. Export the records into a file

3. Dump the file into the widget

4. Optimize the results and create happy customers!

The List (Example 1)

Outcome #1:

Ability to know what releases next week, whether it is here and (if so)

Where is it in our workflow?

Outcome #2:

Ability to send out an email to the public service staff so that they know the Tuesday releases and have talking points with customers.

Outcome #3:

Ability to alert Shipping (they open our boxes and put items on trucks for us) what to expect and what to expedite too us.

Outcome #4

Ability to know what’s hasn’t come and figure and troubleshoot accordingly

• Publication date change?

• Vendor backorder?

• Ask vendor to expedite processing based on our customer demand?

• Is there another way to get the product or do we need to cancel, restore the funds and get it off our system?

Outcome 5:

Ability to use the holds and receipt information in selecting titles to display, promoting our content.

Questions?