quest q3 tips from the techies - sas institute...sep 04, 2014 · tips from the techies tip1 –di...
TRANSCRIPT
Tips from the TechiesLeonard Scuderi
4th September 2014
About Me
Contracting in the Business Intelligence / Data
Warehousing space for the last 12 years in Brisbane,
Sydney and London.
Experience in Banking, Telecommunications, Insurance,
Government, and Loyalty Marketing industries.
Training Courses This Year
"Tackling the Challenges of Big Data“ - Massachusetts Institute of
Technology
"Data Vault Modelling" - Data Vault Academy
Tips are some of the things I have picked up from client
sites and blogs.
Tips from the Techies
Tip1 – DI Studio Performance Tweak
Tip2 – DI Studio User Written Transforms
Tip3 – Using Unicode variables in your Reports
Tip4 – Using Index in SAS Datasets
Tip 1- Studio DI
Performance Tweak
Tip 1 – DI Studio Performance Tweak
Table 20GB in size with 120 Million Observations.
Taking 1 hour with table Loader to append 100,000 Observations.
Dataset is not compressed so it SHOULD use fast append.
Tip 1 – DI Studio Performance Tweak
SAS 9.2 the Table Loader
sometimes in inserts this
code to create a view of
your input dataset
before the append to
your final dataset.
View prevents Fast
Append.
We need to get rid of
this view so that SAS will
use Fast Append.
Tip 1 – DI Studio Performance Tweak
Change Table loader to
“User Written Body” and
change the VIEW to a
TABLE.
Job ran in 5 minutes to
append since fast append
is available.
Remember to add a NOTE
to your job to document
WHY you have done it!
Tip 2- DI Studio
User Written Transform
Tip 2 – DI Studio User Written Transforms
Seen on previous sites where coders have not using macro variable available in
your user written transforms in DI Studio (when you HAVE to write SAS code)
SAS temporary table names are coded the UWT. (eg W6E64OBZ)
data TRANSACTION_CODE_NO_DUPS;
set W6E64OBZ;
by TRANSACTION_CODE;
if first.TRANSACTION_CODE;
run;
Tip 2 – DI Studio User Written Transforms
UWTs have the following
macro variables available to
hook up your job.
_INPUTn and _OUTPUTn where
n is the number of input and
output nodes.
data &_output1.;
set &_input1.;
by TRANSACTION_CODE;
if first.TRANSACTION_CODE;
run;
Tip 2 – DI Studio User Written Transforms
Also Column Variables
%let _OUTPUT1_col0_name =
TRANSACTION_CODE;
So I could have actually coded
data &_output1;
set &_input1;
by &_OUTPUT1_col0_name;
if first.&_OUTPUT1_col0_name;
run;
Only works if your key field is
the first in your dataset
Tip 2 – DI Studio User Written Transforms
Now when we have our job we can just attach the UWT as it if is a normal DI transform without having change the names of the work datasets
Tip 3- SAS Reports
Using Unicode Values
Tip 3 – Adding Unicode Values to Reports
SAS Reports can use Unicode values to enhance reports.
Unicode numerical values to symbols (similar to your
Webdings Winding which are available on your Microsoft
products) which are in HEX format.
Examples
is Unicode 2605
is Unicode 2713
Tip 3 – Adding Unicode Values to Reports
• Example of the “Miscellaneous
Symbols”
• Thousands check out
http://www.unicode.org/charts/
Tip 3 – Adding Unicode Values to Reports
How is this useful?
Using the some code link this
we can use these Unicode
Symbols with colours.
We can use a format to replace
our values with the Unicode
Symbols.
Give Numbers and Descriptions
Colours and Symbols e.g. star
ratings or green tics and red
crosses for targets.
Tip 3 – Adding Unicode Values to Reports
/* ODS EXCAPECHAR needed to set style/unicode cues */
ods escapechar='~';
/* Captured these in macro variables for readability and */
/* easy maintenance */
%let Sunny = ~{style [color=red] ~{unicode '2600'x}};
%let Cloudy = ~{style [color=Grey] ~{unicode '2601'x}};
%let Rainy = ~{style [color=Blue] ~{unicode '2602'x}};
%let Snowy = ~{style [color=Black ] ~{unicode '2603'x}};
%let Hot = ~{style [color=red] ~{unicode '26AB'x}};
%let Comfortable = ~{style [color=orange] ~{unicode '26AB'x}};
%let Cool = ~{style [color=yellow] ~{unicode '26AB'x}};
%let Cold = ~{style [color=blue] ~{unicode '26AB'x}};
proc format lib=work;
value $Weather
'Sunny' = "&sunny"
'Cloudy' = "&Cloudy"
'Rainy' = "&Rainy"
'Snowy' = "&Snowy";
value Temp
0 - 10 = "&Cold"
11 - 20 = "&Cool"
21 - 30 = "&Comfortable"
30 - 40 = "&Hot";
title "Weather Report for 04SEP2014";
proc print data=work.Weather noobs;
format Weather $Weather. Temp Temp.;
run;
Tip 4 – Recreating Indexes
Tip 4 – Re-Creating Indexes
Modify an existing data set.
Delete Duplicate Observations
Remove Erroneous Rows
SAS data set has Index on City that we need to preserve.
Tip 4 – Re-Creating Indexes
Use PROC SQL statement using a Delete Step (to
preserve index) however you end up with Deleted
Observations in your SAS Dataset.
Tip 4 – Re-Creating Indexes
Use a Data Step instead to delete data.
Yes - I don’t have deleted observation.
Oh no - I have lost the indexes that were on the SAS Dataset.
Tip 4 – Re-Creating Indexes
Need to recreate the
index.
If you have DI Studio
you’re lucky you can
just go back to your
DI studio job
The Table Loader or
SCD Loader step in
the code gives you
the code to re-create
your index.
Questions
References
http://www.unicode.org/charts/
http://blogs.sas.com/content/sasdummy/2014/06/10/sas-stars-on-thars/
Leonard Scuderi
Kalebo Komputing