(ATS3-PLAT07) Pipeline Pilot Tips, Tricks and Challenges

Ton van DaelenSr. Director, Platform Product Management

ton.vandaelen@accelrys.com

The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.

• Target audience– Protocol authors

• Personal productivity• Protocol deployment

• Many tips contributed by support• Mix of What’s New and Tips and Tricks

• Other relevant sessions– (ATS3-PLAT10) Win Matt's Money

Introduction

1. Pro-client what’s new2. Protocol search3. Local short-cuts4. Recursion5. Keyboard shortcuts6. Design mode7. Component update8. Merge/join9. RTC subprotocols10. Debugging

10 Tips to Raise Your PP Game

Protocol Search

• Tab– Move between tabs, retaining

search string

• Ctrl-Alt-F– Search utilities, prototypes,

internals– Be careful: it’s hidden for a reason!

• Search string– Tip: type first letters for keywords,

e.g.: “HTM Vie Mol”

Craig Shepherd - Support, San Diego

Favorites Bar

• What– Folders– Most used

• Why– Find components quickly– Accelerate protocol

building

Enhanced Tool Tips

• What– Tooltip explaining preventing a

protocol from running

• Why– Lower learning curve for novice users

• What– Display processing time for each component (Ctrl-T)

• Absolute compute time (sec)• Compute time as percentage of total execution time

• Why– Identify protocol bottlenecks

Component Profiling

PilotScript Context Sensitive Help

• What– Multiple references to same

subprotocol within a protocol

• Why– Reduce protocol size– Re-use of subprotocol defintions

Local Shortcuts

Design Mode

• What– Inspect data stream as

you build your protocol

• Why– Speed up protocol

development time >10x

Design Mode

• Rapid protocol building and debugging– Monitor data in real-time– Immediately see affect of component and parameter changes

• Particularly powerful for Reporting and Imaging applications

Using Design Mode

toggle design mode design on

Turn on design mode ...... puts data caches (“checkpoints”) at each (non-writer, non-viewer) component... after first run, data caches get filled , protocol may re-run from any of these... changes of components invalidate data caches... clicking on components shows data records in AutoWatch window

... From component context-menu: create individual windows; F8: on/off

Update data caches

Design Mode: Statistics

View cache information

Throughput statistics

Design Mode: Preferences

Design Mode: Watch Window

Toggle Default View / Hierarchical View

Toggle Show / Hide all imagesFilter by data to view types

Design Mode: Multiple Watch Windows

• Watch data at multiple stages in the pipeline• Add watch windows using F8 key

Protocol Recovery

• What– Keep copy of unsaved

protocol• Why

– Don’t lose work when client crashes

Saved in the Windows user’s temp directory, e.g.:C:\Users\tvandaelen\AppData\Local\Temp\scitegic\PipelinePilot\8.5.0.200 Default saving frequency is 5 minutes (value saved in registry): HKEY_CURRENT_USER\Software\SciTegic\Pipeline Pilot_8_9_1\Options\AutoSaveMinutes (If you don’t see it in your registry, open Options in Pro Client, click OK, then refresh reg key

Pipeline Pilot Quick Reference Card

• Grab a copy• My favorites– CTRL-DblClick– CTRL-M– CTRL-SHIFT

I/P/F– F7– F10

Stephane Vellay - Support, Cambridge

Looping in Pipelines

• DON’T DO IT!!!• But if you do, be very, very careful• It can work, but we don’t officially support it• Why? Datapipelining is about “data flow”.

Looping pipes is about “control flow”. Mixing these can be very problematic– e.g., a checkpoint in a loop will alter behavior– A generator inside a loop can be

“troublesome…”• Any looped protocol can be re-written

without loops• Control better exerted within components

(e.g., PilotScript), or across pipelines

Ian Kerman - Support, San Diego

Looping

• Calculate factorials– Fac(3) = 3*2*1

• Make sure there is a exit point!

Recursion

• Avoids looping• Again, make sure

your component exits

Factorial *= FacNumber; FacNumber--;

Iteration

• Use Generate Empty Data to create data stream with fixed number of records

Display Globals

Component Update

• Useful for:– Component

upgrades– Regular protocol

building

Craig Shepherd - Support, Cambridge

• Takes all streaming records with identical key values and collapses them into a single record

Merge Data

Key = ID

Join Data From File

• For streaming records, finds related information in reference file (based on identical key values)

Key = IDID Exp1A 4B 4C 1D 3E 2F 1

ID ActivityA 4.1B 2.1C -0.5A 4.3Q 3.0

ID Activity Exp1A 4.1 4B 2.1 4C -0.5 1A 4.3 4

ID ActivityQ 3.0

These records not seen in the output

Reference File

Merge All Data

• Merges properties of all input data records into a single output data record– Performs faster than Merge Data for the special case where all

input records are to be merged into one

Merge/Group/Pivot – Data Are Sorted

• New parameter Data Are Sorted

• Specifies whether the input data are already sorted by the Merge property. If so, then the merge can be done much faster with less memory impact

Quick Reference Guides

RTC Subprotocols

• In a RTC subprotocol, all internal components are executed for all incoming data records

• Use sparingly…• Relevant when there are initialization and/or finalization

components in subprotocol• Often used on “grouped” data

– Group prior to RTC SP > Ungroup in RTC SP > process

• In 8.0: Use a category/tag to auto-group data records and apply the RTC subprotocol

Run to Completion (RTC) Subprotocols

Ordinary data flow

Reader

Reader

Group Ungroup MCSS

Group Data By Tag aggregates the dataand “boxes” it

When all of the data arrives, the MCSScomponent acts on the data

Data moves as far to the right as possible

Subprotocol

Data flow through a subprotocol

Reader

Reader

Group Ungroup MCSS

Subprotocol is a way to visually hide work – No change in behavior

Process is repeated independently for each group

RunToCompletion Subprotocol

Data flow with RunToCompletion

Reader

Reader

Group Ungroup MCSS

Data is aggregated and partitioned by the“Group data by tag” component

MCSS component “sees” and actson only one box of data

Inside a RunToCompletion subprotocol, asingle data record (a “box”) is unpacked

Run to Completion – Simplified Interface

• Grouping / ungrouping obsolete in most cases

• Grouping behavior set as subprotocol parameter

Debugging Tips

Displaying Globals and Data Records

What’s my Data?

Don’t know what the syntax is for a property name?

Find out: Put in a viewer!

Why did I close that output Window so quickly?

• Never fear! Recover results from the Jobs window.

Checkpoints and Watch Windows

• Method to cache the data flowing through a pipeline

• Re-run a job from a specific location in the pipeline

• Skip time consuming steps when debugging

• View the contents of a checkpoint cache using watch windows

Checkpoints and Watch Windows (cont.)

• Watch can be added on to any component containing a cache

• Max of 3 Watch windows• Auto-watch displays currently selected

component cache• Multiple views

– Default View– Hierarchical View– Molecular Images View

• Data Type Filtering– All Data– Pass Data – Fail Data– Global Variables

• Provides a method to debug protocols at run time

• Displays a Debug Messages window that shows the results from debug message commands– DebugMessage: Displays a debug text message in the Debug Messages

window– DebugMessageError: Displays a debug text message (in red) in the Debug

Messages window– DebugBreak: Pauses protocol execution and displays a message in a dialog

box. Processing continues when you click the Resume button

• Commands available in Pilotscript, Java and Perl

• Debug Parameters Window– Not limited to components that use PilotScript, Java or Perl expressions.– Allows the addition of debug statements that test the outputs for any

component type

Running with Debugging

• Want to read in a data file quickly?– Just drag any file into an open protocol window to

automatically invoke its reader type (SD, Mol2, PDB, etc)– Drag any file onto an existing Reader to append that file to

the data stream• Want to use a protocol quickly that you get in an email?– Drag any protocol (e.g. email attachments, protocols in other

directories, etc) onto the protocol window to open them

Rapidly accessing files and protocols

• Lots of new productivity features in 8.0 and 8.5• Resources– Community forum

• Share your insights!

– Support hotline– Training

Summary

The information on the roadmap and future software development efforts are intended to outline general product direction and should not be relied on in making a purchasing decision.

For more information on the Accelrys Tech Summits and other IT & Developer information, please visit:https://community.accelrys.com/groups/it-dev