code generation. 2 overview of presentation goal background dynamic sql method examples

Code Generation

2

Overview of presentation

Goal Background Dynamic SQL Method Examples

3

Goal

The goal of this presentation is to cover several methods of having SAS write code for you

The method I will use is to cover the concept, followed by some examples, then some tips

Data is faked up or at the least obfuscated As much as possible all ideas have been stolen,

borrowed, downloaded from the internet or otherwise cribbed

Think about it: would you trust a presentation on “not writing your own code” from someone who actually came up with all of the ideas themselves?

4

Background

I’ve been programming SAS for years, but prior to that I spend a lot of time dealing with Oracle SQL SQL programming has little or no macro facility To do task that are simple in SAS you have to be

able to code Dynamically Dynamic coding translates will into dealing with

SAS macros, but still comes up on its own accord The next few slides are an overview of

Dynamic SQL as a background.

5

SELECT ALM_KEY, 1 Cycle , Prefix,Current_account_number

FROM OPS$U35234 . DEBIT_ACT_CARD_ALM_JUN_1999

where rownum < 11 and statement_cycle_number = 1

order by Prefix,Current_Account_Number;









.

.

.

.

SPOOL OPTIONS

spool code.sql

Select

'SELECT ALM_KEY,',rownum,' Cycle , Prefix,Current_account_number' coll1,

'FROM &owner . &ALM_table' coll2,

'where rownum < 11 and statement_cycle_number = '||rownum coll3,

'order by Prefix,Current_Account_Number;'

FROM snapshot where rownum < 29;

spool off

Dynamic SQL: Example 1 get 10 accounts of each type

6

/* -------------------------------------------------------------------- */

/* Spool a file of SQL code to be run later */

/* -------------------------------------------------------------------- */

spool code.sql

SELECT

'spool '||AGENCY||'.lst ' CC,

'select r1_account_num||account_prefix||account_number ' CC,

'from nbr_co_seg1 ' CC,

'where exclude_flag is null and move = 1 ' CC,

'and agency = '''||AGENCY||''' ' CC,

'and final_level in (201,301,202,203,302,303) ' CC,

'and keep_flag in (2,3,4); ' CC,

'spool off ' CC

FROM NBR_CO_SEG1

WHERE EXCLUDE_FLAG IS NULL

AND FINAL_LEVEL IN (201,301,202,203,302,303)

AND MOVE = 1

AND KEEP_FLAG in (2,3,4)

GROUP BY AGENCY;

spool off

/* -------------------------------------------------------------------- */

Dynamic SQL: Example 2 spool a separate file for each AGENCY

7

The Principles: How it works

Using SAS as a way to write other code is something that is most effective in situations meeting :

1. You have a lot of repetitive tasks2. These core details that change from task to task can be

expressed in a dataset3. The dataset needed is easy to create4. The code to complete these tasks needs to be saved and/or the

macro facility by itself cannot complete the task

For the sake of clarity let us establish some names:Program 1: The sas code we write in order to produce program 2Program 2: The sas code Program 1 writes in order to do stuff

[Program 1] [Program 2]

Did I mention that Program 2 does not need to be SAS code?

8

What is the difference between this and writing macrosThe interaction between using SAS generated

SAS code and a SAS macro is a little complicated. Sometimes they are two ways to solve the same

issue. Often SAS written code is a tactic in macro

programming, either explicitly or via the CALL EXECUTE. Oddly enough, code generating seems a bit more stable and platform independent

9

Basic outline of program 2

1. Set-up / Initialization Could include library statements, options, etc. Could include ODS statements

2. List of actions to be taken Could be a data steps for each of 100 situations Could be a report procedure for each combination of variables Could be a sas macro for each variable Could be a simple assignment statement or if/then

3. Clean-up Delete any temporary datasets Close out any ODS files Restore normal system options

10

Basic outline of program 1

1. Initialize options and set up

2. Gather information about each instance of the task

3. Organize data set of information for each task

4. Employ a data null step to fill in constants (i.e. the tasks) in between the variables (i.e. the specifics of each situation)

5. Clean up

11

Examples

Proc Google Getting a format from proc rank Generating SAS code from Specs Using SAS’s built in coding features

12

Proc Google

While this is technically not a SAS thing per say if searching for your problem on the internet solves the issue, who cares? I’ve lifted lots of good stuff over the years, but I want to share a great example next because it involves generating SAS code.

Originally it generated a proc format statement, but I modified it for other stuff…

13

Getting a format from proc rankFor the purposed of reporting and analysis we use proc rank all the

time. It is perfect for computing quantiles such as deciles or twentile. Sometimes we either do not want to add the ranked variables to our

dataset, or more likely we want to look at a number of variables from a number of datasets broken down the same way.

Attempt 1: check the SAS documentation to see if proc rank will produce a format instead of a variable. It seems reasonable, but the attempt is in vain

Attempt 2: Check Google: search “proc rank format” (2nd link is a macro written by Ken Moore that does the trick)

Attempt 3: Write the code yourself (thankfully we are spared)…well I did modify this to produce the SAS code to produce the ranges

14

Getting a format from proc rankLet’s look at an outline of how this process works:

1. Rank the data into a temporary data set

2. Run proc means on this data set, with the ranked variable as the only class variable. Output to a dataset

3. Adjust the data so that the Min of Rank N becomes the Max of Rank N-1. Treat the first and last rankings specially (along with missing values. All of this is in one dataset.

4. Input this summary dataset into proc format to create a format

5. Create a summary report using proc freq and the format we just created

6. Use a data _null_ step to write the code for the format

15

Getting a format from proc rank: Program 1

As an example I’m using the dataset sashelp.prdsal2%include "path\rankformat.sas";%FORMAT(SASHELP.Prdsal3 , PREDICT , PF , 10 );

Input dataset

Variable to rank

Output format name

Number of groups needed

16

Getting a format from proc rank: Program 2

As an example I’m using the dataset sashelp.prdsal2

PROC FORMAT; VALUE PF . -< . ="Missing " LOW -< 174.000 ="< 174.000 " 174.000 -< 361.000 ="174.000 -< 361.000 " 361.000 -< 547.800 ="361.000 -< 547.800 " 547.800 -< 728.000 ="547.800 -< 728.000 " 728.000 -< 916.300 ="728.000 -< 916.300 " 916.300 -< 1098.900 ="916.300 -< 1098.900 " 1098.900 -< 1304.000 ="1098.900 -< 1304.000 " 1304.000 -< 1557.000 ="1304.000 -< 1557.000 " 1557.000 -< 1855.000 ="1557.000 -< 1855.000 " 1855.000 - HIGH ="1855.000 + ";RUN;

%macro PREDICT; if PREDICT = . then PREDICTr =0 ; else if PREDICT <= 174.000 then PREDICTr =1 ; else if PREDICT > 174.000 and PREDICT <= 361.000 then PREDICTr =2 ; else if PREDICT > 361.000 and PREDICT <= 547.800 then PREDICTr =3 ; else if PREDICT > 547.800 and PREDICT <= 728.000 then PREDICTr =4 ; else if PREDICT > 728.000 and PREDICT <= 916.300 then PREDICTr =5 ; else if PREDICT > 916.300 and PREDICT <= 1098.900 then PREDICTr =6 ; else if PREDICT > 1098.900 and PREDICT <= 1304.000 then PREDICTr =7 ; else if PREDICT > 1304.000 and PREDICT <= 1557.000 then PREDICTr =8 ; else if PREDICT > 1557.000 and PREDICT <= 1855.000 then PREDICTr =9 ; else if PREDICT > 1855.000 then PREDICTr =10 ;%mend;

Output format statement

Output macro statement

17

Getting a format from proc rankHow the program works

******************************************************************************;* Program: format.sas *;* Author: Ken Moore *;* Created: 08/28/03 *;******************************************************************************;%MACRO FORMAT(DATA, /* INPUT DATA SET */ VAR, /* CONTINUOUS NUMERIC VARIABLE TO BE FORMATTED */ FMT, /* FORMAT NAME */ GROUPS /* NUMBER OF GROUPS */ ); **********************************************************************; * ASSIGN RANK FOR NON-MISSINGS ; **********************************************************************; proc rank data=&DATA (keep=&VAR where=(&VAR ne .)) out=rank groups=&GROUPS; var &VAR; ranks RANK; run;

18


*********************************************************************; * OBTAIN GROUP MEANS FOR FORMAT BOUNDARIES ; *********************************************************************; proc means data=rank missing noprint; var &VAR; class RANK; output out=mnrank (where=(RANK ne .)) min=min; run; data mnrank; set mnrank; RANK_=_N_; run; data getrange; set mnrank; RANK_=RANK_-1; run; proc sort data=getrange; by RANK_; run; proc sort data=mnrank out=smnrank; by RANK_; run;

19


*********************************************************************; * CREATE FORMAT DATASET ; *********************************************************************; data miss; format START END $10. LABEL $20.; FMTNAME="&FMT"; START='.'; END='.'; LABEL='Missing'; run;

data getrange2; format START END $10. LABEL $20. MIN MAX 8.3; merge getrange (in=a rename=(MIN=MAX_)) smnrank (in=b rename=(MIN=MIN_)); by RANK_; retain FMTNAME "&FMT" SEXCL EEXCL 'Y'; MAX=MAX_; MIN=MIN_; if RANK_>0; START=PUT(MIN,8.3); END=PUT(MAX,8.3); LABEL=left(compbl(put(MIN,8.3)||' -< '||put(MAX,8.3))); if RANK=0 then do; START='LOW'; HLO='L'; SEXCL='N'; EEXCL='N'; LABEL=compbl('< '||put(MAX,8.3)); end; if RANK=%eval(&GROUPS-1) then do; END='HIGH'; LABEL=compbl(put(MIN,8.3)||' +'); HLO='H'; SEXCL='N'; EEXCL='N'; end; run;

data getrange3; set miss getrange2; run;

proc format cntlin=getrange3; run;

20


*********************************************************************; * OUTPUT SUMMARY TABLE ; *********************************************************************; title "VAR: %UPCASE(&VAR.), FORMAT NAME: %UPCASE(&FMT.), GROUPS: &GROUPS "; proc freq data=&data.; tables &var. / missing; format &var. &fmt..; run; title; *********************************************************************; * OUTPUT PROC FORMAT TO PRINT ; *********************************************************************; data _null_; set getrange3 end=eof; file print; TO=' -< '; if END='HIGH' then TO=' - '; SEMI=''; if eof then SEMI=';';

if _n_=1 then put "PROC FORMAT;" / " VALUE %quote(&FMT.)" / " " START TO END '="'LABEL'"'; else put " " START TO END '="'LABEL'"' SEMI; if eof then put "RUN;"; run;

21


*********************************************************************;* OUTPUT CODE IN SAS TO PROGRAM ;*********************************************************************;data _null_; set getrange3 end=eof; file print;

R=_N_-1;if _N_=1 then put '%macro ' "&VAR.;" /

" if &VAR. = . then &VAR.r =" R "; " ;else if START="LOW" then put " else if &VAR. <= "END" then &VAR.r =" R "; " ;else if END="HIGH" then put " else if &VAR. > "START" then &VAR.r =" R ";" ;else put " else if &VAR. > "START" and &VAR. <= "END" then &VAR.r =" R ";";if eof then put '%mend;';

run;

*********************************************************************;* CLEANUP ;*********************************************************************; proc datasets library=work nolist;delete rank mnrank smnrank getrange miss getrange2 getrange3;run; quit; %MEND FORMAT;

22

Generating SAS code from SpecsThis example came from having to have code written in SAS and

other languages from programming specs All of the specs involve one of two structures

1. A single variable cut into ranges

2. A matrix of two variables cut into ranges The first step in this process was to standardize the specs:

A B C1 range min max2 0 496 5313 1 532 5494 2 550 5605 3 561 5686 4 569 5737 5 574 5798 6 580 5839 7 584 58710 8 588 59211 9 593 611

A B C D E F G1 99 0 1 2 3 4

2 99

3 0 1 1 1 2 2 2

4 1 2 2 2 2 2 2

5 2 3 2 2 2 3 3

6 3 3 2 3 3 3 3

7 4 3 2 3 3 3 3

8 5 3 2 3 3 3 3

Ranges Matrix (truncated: actually 20 by 20 is used today)

23

Generating SAS code from SpecsProgram 1: outlineNow that we have standard specs, we can outline a process to

generate the code:1. Specify metadata

Pathnames for specs and for macros Names of Range1-RangeN Names of Matrix1-MatrixM Variable ranges to be used in each matrix

2. Read Specs Read each range %Import_range Read each matrix %Import_matrix

3. Create code Code for each range %Code_ranges Code for each matrix %Code_matrix

4. Output code Code in SAS data _null_ . . . Code in other languages data _null_ . . .

5. Clean up proc datasets

%Run_Autocode

24

Generating SAS code from SpecsProgram 1: Specify metadata%Let Path1 ={c:\...\}\Code;%Let Path2 ={c:\...\}\Specs;

%Let Num_Ranges =16;%Let Range1 =Score_0; %Let var1 =Score ;%Let Range2 =Severity_0; %Let var2 =Severity ;

%Let Range3 =Score_1; %Let var3 =Score ;%Let Range4 =Severity_1; %Let var4 =Severity ;

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . %Let Range15 =Score_91; %Let var15 =Score ;%Let Range16 =Severity_91; %Let var16 =Severity ;

%Let Num_matrixes =8;%Let Matrix1_var1=Score_0; %Let Matrix1_var2=Severity_0;%Let Matrix2_var1=Score_1; %Let Matrix2_var2=Severity_1;

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . %Let Matrix7_var1=Score_90; %Let Matrix7_var2=Severity_90;%Let Matrix8_var1=Score_91; %Let Matrix8_var2=Severity_91;

25

Generating SAS code from SpecsProgram 1: %Run_autocode%macro Run_autocode;%do index = 1 %to &Num_Ranges.; %Import_range(&&Range&index..); %end;%do index = 1 %to &Num_matrixes.; %Import_matrix(&&Matrix&index._var1,&&Matrix&index._var2) %end;

%do index = 1 %to &Num_Ranges.; %Code_ranges(&&Range&index..,&&var&index..); %end;%do index = 1 %to &Num_matrixes.; %Code_matrix(&&Matrix&index._var1,&&Matrix&index._var2); %end;

data _null_;set sas_code;file "&Path1.\RangeMatrix.sas" ;put @1 code $120.;run;

data _null_;set bds_code; file "&Path1.\bds_code.txt" ;put @1 code $120.;run;

proc datasets; delete MTX: (gennum=all); delete Range: (gennum=all); delete Sas_code: (gennum=all); delete bds: (gennum=all);

quit;%mend Run_autocode;

26

Generating SAS code from SpecsProgram 1: %Import stuff;%macro Import_range(range);PROC IMPORT OUT= WORK.&range. DATAFILE= "&Path2.\&range..xls" DBMS=EXCEL REPLACE; SHEET="Sheet1$"; GETNAMES=YES; MIXED=NO; SCANTEXT=YES; USEDATE=YES; SCANTIME=YES;RUN;

%mend Import_range;

%macro Import_matrix(var1,var2);PROC IMPORT OUT= WORK.MTX_&var1._&var2._defn DATAFILE= "&Path2.\matrix_&var1._&var2..xls" DBMS=EXCEL REPLACE; SHEET="Sheet1$"; GETNAMES=YES; MIXED=NO; SCANTEXT=YES; USEDATE=YES; SCANTIME=YES;RUN;

%mend Import_matrix;

27

Generating SAS code from SpecsProgram 1: %Code_ranges;%macro Code_ranges(range,var);data &range._sascode (keep=code);set &range. nobs=n;where min*1 ne . and max*1 ne .;length code $ 120.; if _n_=1 then code=" "; if _n_=1 then output; if _n_=1 then code='%'||"macro CodeRange&range.; "; if _n_=1 then output; if _n_=1 then code="*----------| Sas Code for &range. using &var. |---*;"; if _n_=1 then output; if _n_=1 then code=" &range. =99;"; if _n_=1 then output; code=" if &var. >="||compress(min)||" and &var. <= "||compress(max)||" then &Range. ="||compress(range)||";"; output; if _n_=n then code="%mend CodeRange&range.;"; if _n_=n then output;run;

proc append base=sas_code data = &range._sascode force;run;%mend Code_ranges;

28

Generating SAS code from SpecsProgram 1: %Code_Matrix;%macro Code_Matrix(v1,v2);data MTX_&v1._&v2._defn2; set MTX_&v1._&v2._defn (keep=_numeric_); run;

data MTX_&v1._&v2._code (keep=code );length code $ 120.;set MTX_&v1._&v2._defn2 nobs=n;array columns[*] F: ;

dim=dim(columns);Range1=columns[1];

if _n_=1 then code=" "; if _n_=1 then output; if _n_=1 then code='%'||"macro CodeMatrix_&v1._&v2.; "; if _n_=1 then output; if _n_=1 then code="*------|Matrix of variables &v1. and &v2. |----*;"; if _n_=1 then output; if _n_=1 then code=" Matrixout =99;"; if _n_=1 then output;

do i= 2 to dim;if i=2 then Range2=99;if i>2 then Range2=i-3;RiskRank=columns[i]; code=" if &v1.="||compress(range1)||" and &v2. = "||compress(range2)||" then Matrixout="||

compress(RiskRank)||";";if RiskRank ne . then output;

end;

if _n_=n then code="%mend CodeMatrix_&v1._&v2.; "; if _n_=n then output; if _n_=n then code="*--------------------------------------------------*;"; if _n_=n then output;run;

proc append base=sas_code data = MTX_&v1._&v2._code force;run;%mend Code_Matrix;

29

Generating SAS code from SpecsProgram 2: The Actual Code Ranges

%macro CodeRangeScore_0; *----------| Sas Code for Score_0 using Score |---*; Score_0 =99; if Score >=0 and Score <= 589 then Score_0 =0; if Score >=590 and Score <= 601 then Score_0 =1; if Score >=602 and Score <= 610 then Score_0 =2; if Score >=611 and Score <= 618 then Score_0 =3; if Score >=619 and Score <= 626 then Score_0 =4; if Score >=627 and Score <= 662 then Score_0 =5; %mend CodeRangeScore; %macro CodeRangeSeverity_0; *----------| Sas Code for Severity_0 using Severity |---*; Severity_0 =99; if Severity >=0 and Severity <= 4994.6326 then Severity_0 =0; if Severity >=4994.6327 and Severity <= 5000 then Severity_0 =1; if Severity >=5000.0001 and Severity <= 7651.3413 then Severity_0 =3; if Severity >=7651.3414 and Severity <= 11027.44 then Severity_0 =4; if Severity >=11027.4401 and Severity <= 15335.9424 then Severity_0 =5; if Severity >=15335.9425 and Severity <= 20625.4816 then Severity_0 =6; if Severity >=20625.4817 and Severity <= 28399.5717 then Severity_0 =7; if Severity >=28399.5718 and Severity <= 41538.5343 then Severity_0 =8; if Severity >=41538.5344 and Severity <= 10000000 then Severity_0 =9; %mend CodeRangeSeverity;

30

Generating SAS code from SpecsProgram 2: The Actual Code Matrix

%macro CodeMatrix_Score_0_Severity_0; *------|Matrix of variables Score_0 and Severity_0 |----*; Matrixout =99; if Score_0=0 and Score_0 = 1 then Matrixout=3; if Score_0=0 and Score_0 = 2 then Matrixout=3; if Score_0=0 and Score_0 = 3 then Matrixout=3; if Score_0=0 and Score_0 = 4 then Matrixout=3; if Score_0=0 and Score_0 = 5 then Matrixout=3; if Score_0=0 and Score_0 = 6 then Matrixout=2; if Score_0=0 and Score_0 = 7 then Matrixout=2; if Score_0=0 and Score_0 = 8 then Matrixout=1; if Score_0=0 and Score_0 = 9 then Matrixout=1; if Score_0=0 and Score_0 = 10 then Matrixout=1; if Score_0=1 and Score_0 = 1 then Matrixout=4; if Score_0=1 and Score_0 = 2 then Matrixout=3; if Score_0=1 and Score_0 = 3 then Matrixout=3;. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . if Score_0=5 and Score_0 = 2 then Matrixout=4; if Score_0=5 and Score_0 = 3 then Matrixout=4; if Score_0=5 and Score_0 = 4 then Matrixout=4; if Score_0=5 and Score_0 = 5 then Matrixout=4; if Score_0=5 and Score_0 = 6 then Matrixout=4; if Score_0=5 and Score_0 = 7 then Matrixout=4; if Score_0=5 and Score_0 = 8 then Matrixout=4; if Score_0=5 and Score_0 = 9 then Matrixout=4; if Score_0=5 and Score_0 = 10 then Matrixout=3; %mend CodeMatrix_Score_0_Severity_0; *--------------------------------------------------*;

31

Using SAS’s built in coding featuresThere are several SAS features that will write code for

you: www.sas.com Import/Export wizard SUGI papers that are online Enterprise Guide

1. Click your way through your task

2. View the code

3. Fix the code to get what you really want

4. Save

32

Useful tools and tips

Dimensionless arrays Proc SQL select into External files Proc Contents

code generation. 2 overview of presentation goal background dynamic sql method examples

Documents