enterprise guide 4.3 ( sas 9.2 ts2m3) · software used for presentation windows 7 ultimate sas 9.2...
TRANSCRIPT
ENTERPRISE GUIDE 4.3
( SAS® 9.2 TS2M3)
Edmonton SAS User Group
May 11, 2011
John Kirtz
Snowy Owl Software
® SAS is the registered trademark of SAS Institute Inc, North Carolina, USA
Implementing an Automated Analytic Process
1
INTRODUCTION Presentation Objectives
Encourage traditional SAS programmers to explore enterprise guide 4.3
“Real Programmers use the SAS editor!”
Come over to the dark side, succumb to the power of 4.3
Why I actually started to use 4.3
Who is using EG 4.3 ?
Who is aware of new features in EG 4.3 ?
Demo EG can be used for more than developing SAS programs
Why I should have started to use EG earlier
No need to be a systems or SAS expert
Daily / weekly data updates
Analysis projects that need to be repeated
2
INTRODUCTION Presentation Outline
Introduction
1. Review EG and SAS configuration used
2. Some features of Enterprise Guide 4.3
Come over to the dark side
3. Implement a sample automated analytic process using EG 4.3
Scenario
Data processing ( hook in external SAS programs)
Analysis / Report Output products ( EG Tasks – no code)
Product Delivery ( EG Report and Email objects – no code)
Automate (EG Schedule process – no code)
4. Closing Remarks
3
SAS And Enterprise Guide Installation
Software Used For Presentation
Windows 7 Ultimate
SAS 9.2 M3 Installation
Foundation :Base / Graph / QC
Enterprise Guide 4.3
default configuration installed with SAS Foundation
Strictly using features easily available to SAS programmers
No need for IT support
No SAS BI Software
No Stored Processes
Many more capabilities with SAS BI installed
Access to metadata
Centralized data / projects
Roles based processing
Full use of Stored Processes
4
Some Features Of Enterprise Guide 4.3
Come over to the dark side Intellisense
Very efficient for slow typist and bad speller
now use much longer more descriptive datasets names
Allows easy exploration of all possible choices
New formats, informats, functions always being added
New procs and procs I had forgotten about
Hard to remember options and syntax for procs I was familiar with
Break old habits
Found I always tended to use what I already knew
Use favourite colours as opposed to checking available
Allowed me to update my coding elements
More powerful influence than it seems on the surface
Examples
Just show what is available
No attempt to detail how to access
5
Some Features Of Enterprise Guide 4.3
Come over to the dark side Integrated Syntax Help
6
Some Features Of Enterprise Guide 4.3
Come over to the dark side Syntax Suggestion and Auto-complete
Available Functions and Context Based Parameter Help
7
Some Features Of Enterprise Guide 4.3 Come over to the dark side
Available Libraries and Data Members
Parentheses Matching
8
Some Features Of Enterprise Guide 4.3 Come over to the dark side
Macro Variables and Macro Functions
9
Some Features Of Enterprise Guide 4.3 Come over to the dark side
Colour Selection Format Selection
10
Some Features Of Enterprise Guide 4.3 Come over to the dark side
Other Useful Features
Finding Errors In Log
Ctrl+E : next error
Ctrl+Shift+E : previous error
Analyse Program Execution
Detailed Report
Procedure and DATA steps
File and data sources
Process flow representation
so-so results in this release
Clean Up Program Formatting Problems
see below
11
Some Features Of Enterprise Guide 4.3 Come over to the dark side
Format Program Code (before)
12
Some Features Of Enterprise Guide 4.3 Come over to the dark side
Format Program Code (after)
13
Sample Automated Process Scenario
Snowy Owl Feed produces hog feed on five production lines at their Fort Saskatchewan Plant.
Decided to test implementing NIR based QA/QC on one of the production lines
Machine was installed on a line and set-up to test a sample every 3 minutes
At the end of each hour the machine outputs a “batch” jdx file containing scan and constituent data
The data in this file is to be used to generate distinct reports to NIR research and Product QA/QC dept
If test is successful then NIR machines will be installed on all five lines.
14
Sample Automated Process Process Flow
NIR
Machine
Consti-
tuent
Dataset
Product
Sampled
( 3 min)
Const
Listing
Import
Data
File
Batch
JDX File
( 1 hour)
EG
Report
QA/QC
Scan
Dataset
EG
Analytic
Tasks
Shew
Hart
Scan
Plots
Scan
CV
EG
Report
NIR
Support
Batch
Dataset
Problem
with
Process
Data Acquisition Analysis and Reports Report Distribution Exception Handling
Upload
Data
QA/QC
Data
Marts
Extract
Report
Data
EG
EG
15
Sample Automated Process SAS Tasks
Create an Enterprise Guide project
Develop SAS program to read JDX data into SAS datasets
Batch, Constituent Concentration, Nir Scan data
External program so it can be used elsewhere
Develop SAS program to load batch datasets into historical SAS data marts
Code separate as may switch to relational database later ( blob structure for NIR data)
External program so it can be used elsewhere
Develop SAS program to extract report data from historical data marts
Code separate as may switch to relational database later
External program so it can be used elsewhere
Implement required analysis reports
Use analytic EG tasks
Implement required report distribution
Use EG Email objects
Separate email task for each dept
Test the process manually
Automate the project
Use project schedule object
16
Sample Automated Process Create Project : Add Autoexec Process
17
Sample Automated Process Data Acquisition : Data File Structure
Jcamp-Dx Format ( see handout)
Documented at http://www.jcamp-dx.org/
ASCII
Divided into sections
1. File Header
contains batch-file info
For each Sample there is a distinct section
2. Sample header
Identify sample
Scan parameters
3. Sample Constituent Data
Predicted from scan
Equations for constituents can be purchased or developed internally
4. NIR Scan Data
value for each wavelength captured by the machine
array of values defined by sample header
18
Sample Automated Process Data Acquisition : Information to be Captured
2. Sample Header ##TITLE= UltraHog1A
##JCAMP-DX= 4.24
##DATA TYPE= NEAR INFRARED SPECTRUM
##ORIGIN= Unity InfoStar Converter
##OWNER= FortSaskLineA3
##DATE= 05/05/2011
##TIME= 12:03:00
##SAMPLE DESCRIPTION=FortSaskLineA3
##SPECTROMETER/DATA SYSTEM= UNITY
##XUNITS= NANOMETERS
##YUNITS= ABSORBANCE
##XFACTOR= 1
##YFACTOR= 0.0000001
##FIRSTX= 680
##LASTX= 2498
3. Sample Constituent
##CONCENTRATIONS= (NCU)
(<Protein >, 12.342947, ?)
(<Starch >, 59.667493, ?)
(<Energy >, 4043.35476, ?)
(<DEC >, 3088.24806, ?)
(<Lysine >, 0.388842, ?)
(<TotalFiber >, 18.936235, ?)
1. File Header
##TITLE= UltraHog1A
##JCAMP-DX= 4.24
##DATA TYPE= LINK
##BLOCKS= 50
##ORIGIN= Unity InfoStar Converter
##OWNER= FortSaskLineA3
##DATE= 05/05/2011
##TIME= 12:00:00
4. Scan NIR Data
##DELTAX= 1
##XYDATA= (X++(Y..Y))
680 -973670 -989915 -990243 -1002979 -1011126 -1017817
686 -1030018 -1041358 -1047110 -1059269 -1071373 -1076076
692 -1088688 -1094826 -1110564 -1118795 -1121593 -1126253
698 -1133244 -1143778 -1147941 -1161458 -1164918 -1167587
…
2474 3588844 3548895 3509170 3467828 3426212 3385334
2480 3348105 3306655 3269560 3234108 3193521 3155512
2486 3109822 3078577 3035928 2997463 2958904 2924827
2492 2885571 2852518 2819183 2782962 2753934 2710218
2498 2673369
##END=
19
Sample Automated Process Data Acquisition : Data Capture and Storage
Develop programs
Link in execution order
20
Sample Automated Process Analysis and Reports : Extract Report Data
Develop program to extract required data from data marts
Link to UpdateDatabase program to insure correct execution order
21
Sample Automated Process Analysis and Reports : Extract Report Data
Execute HogFeedQAQC process flow to create analysis datasets ( needed to develop analytic tasks)
22
Sample Automated Process Analysis and Reports : Batch Constituent Listing
Select ConstituentWideX dataset, click on Data List task
23
Sample Automated Process Analysis and Reports : Constituent Shewhart Charts
Select ConstituentX dataset, click on Individual Measurements Chart task
24
Sample Automated Process Analysis and Reports : NIR Scan Plots
Select NIRWAVELENGHTX dataset, click on Line Plot task
25
Sample Automated Process Analysis and Reports : NIR Wavelength Statistics
Select NIRWAVELENGHTX dataset, click on Summary Table task
26
Sample Automated Process Analysis and Reports : Analysis Tasks Implemented
27
Sample Automated Process Analysis and Reports : QA/QC Dept Report
Using Report Object
28
Sample Automated Process Analysis and Reports : NIR Dept Report
Using Report Object
29
Sample Automated Process Report Distribution : Create PDF Version of QA/QC Report
Use Export Option of QA/QC Report menu to create a PDF file
30
Sample Automated Process Report Distribution : Create PDF Version of NIR Report
Use Export Option of NIR Report menu to create a PDF file
31
Sample Automated Process Report Distribution : Ready to Implement Emails
32
Sample Automated Process Report Distribution : Email Administration
33
Sample Automated Process Report Distribution : Email QA/QC Report
Use Send To Option of QAQC Depart Report.pdf object “Email Recipient as step in project”
34
Sample Automated Process Project : Complete
Use Export Option of NIR Report menu to create a PDF file
35
Sample Automated Process Project : Schedule
36
Closing Remarks Not Perfect (Yet)!
SAS putting a lot of effort into EG
still a few bugs but hot fixes are on the way.
No task or method to shut down automated EG before it has completed the process
no equivalent to ENDSAS statement
need to set up a global “status” macro to conditionally execute steps
no date for a fix for this
An automated project that contains a “conditional task” may hang
requires a Task Manager shutdown
hot fix on the way in May (2011)
Icon in process flow window indicates if there was an error but process cannot access this information
SAS is aware of this and are planning to remedy it but no date available
Problems with:
DDE
X statement and systask (possible permissions issue)
SAS statements that do not work well in a SAS batch program
Interactive environments such as the Report window
SAS/AF Applications
%WINDOW Statements
DATA step debugger
37
Closing Remarks Not Perfect (Yet)!
Add-ins
Export to Excel 2007-2010
works if you do not have Access to pc file formats
Access to ODS Graphics Designer
allow you run the designer from within an EG session
Custom Add-ins
can develop your own using Visual Basic or C#
38
MANAGING A PROCESS WITH E.G. 4.3
John Kirtz
Snowy Owl Software
780-998-7455
The End!
39