hana bootcamp sqlscript and abap integration
DESCRIPTION
HANA Bootcamp SQLScript and ABAP IntegrationTRANSCRIPT
Building SQL Script based Applications
Christian Behrens, Mirco SternJune 2012
© 2012 SAP AG. All rights reserved. 2
The Customer
Headquarter: Vernier Switzerland (Geneva) Fragrance and flavor industry Global leader
Highly regulated products Compliance checks
© 2012 SAP AG. All rights reserved. 3
The Problem: Compliance Checks
Imagine being one of 100 creators
You just happened to compose this great new fragrance
Then:
Enter the composition into “Sphinx”
AND: Check compliance
(> 1/2 hr, i.e.: Have lunch!)
The case for SAP HANA
© 2012 SAP AG. All rights reserved. 4
Requested Solution
Engine to check formulas against compliance rules
“Compliance Engine 2”, powered by SAP HANA
Business Facts
Phase I (“Fragrance”) Phase II (“Flavor”)
Project Start July 2011 February 2012
Go-Live November 2012 September 2013
Project Effort ~2200 Person Days ~4000 Person Days
© 2012 SAP AG. All rights reserved. 5
Agenda
Introduction
Reference Architecture
Architecture & Design of Compliance Engine 2 High-Level Architecture Best Practices & Lessons Learned: Building Applications in SQLScript
Lifecycle Management, System Landscape & Transports
Integration with ABAP Invoking SQLScript-Procedures from ABAP
Conclusion
Reference Architecture
© 2012 SAP AG. All rights reserved. 7
SAP Custom Development
Make customers’ core business processes real real-time
≠ Reporting
Provide customer-specific in-memory solutions
– Enhance SAP standard applications
– Develop next generation applications
For Compliance Engine 2 Enhancement of EHS (SAP Standard) SAP HANA solution is accessed from EHS
Custom DevelopmentOfferings for
SAP HANA
Accelerators
Support
Enhance-ments
Applications
© 2012 SAP AG. All rights reserved. 8
SAP HANA as a Secondary DB
Side-by-side scenario Replicate data into SAP HANA
ABAP Platform
SAP HANADBSLT
SAP Suite New App
© 2012 SAP AG. All rights reserved. 9
Next Generation ApplicationsThere’s more to it than just changing the underlying database
SQLScript: Move calculations into DB – Benefit from power of SAP HANA DB
Traditional Model: “Data to Code”
DB Server
Application
Server Code
New Model: “Code to Data”
SAP HANA DB
Application
Server
Code
© 2012 SAP AG. All rights reserved. 10
Reference Architecture (SAP HANA as a Secondary DB)
Side-by-side scenario Replicate data into SAP HANA
ABAP Platform
HANADBSLT
SAP Suite New App
New App
SQLScript
Architecture & Design of Compliance Engine 2
High-Level Architecture
Best Practices & Lessons Learned: Building Applications in SQLScript
© 2012 SAP AG. All rights reserved. 12
SAP HANA
Calculation results
SAP Application ServerKernel 7.20
3rd Party
Master Data
High-Level Architecture
Legacy Systems
Mid
dle
wa
re
EH&S
ComplianceEngine
DB
SAP HANAInterface
Proxy
SLT
Constraints
SubstancesCompositions
CheckResults
CompositionsWhere Used
Check Execution
R
R
Integration
Architecture & Design
Architecture & Design of Compliance Engine 2
High-Level Architecture
Best Practices & Lessons Learned: Building Applications in SQLScript
© 2012 SAP AG. All rights reserved. 14
Bring Code to Data: SQLScript
CREATE PROCEDURE getOutput (IN cnt INTEGER, OUT output_pubs tt_publishers, OUT output_year tt_years) LANGUAGE SQLSCRIPT READS SQL DATA ASBEGIN
big_pub_ids = SELECT publisher FROM books -- Query Q1 GROUP BY publisher HAVING COUNT (isbn) > :cnt;
big_pub_books = SELECT title, price -- Query Q2 FROM :big_pub_ids, publishers, books WHERE pub_id = pid AND pub_id = publisher;
output_pubs = SELECT SUM(price) -- Query Q3 FROM :big_pub_books GROUP BY publisher;
output_year = SELECT SUM(price) -- Query Q4 FROM :big_pub_books GROUP BY year;
END;
SQL ScriptCompiler
Publisher
PUBS Year
Q3 Q4
Q2
Q1
© 2012 SAP AG. All rights reserved. 15
SQLScript: SQL vs. Build-Ins
ot_books_agg = SELECT pub_year, COUNT(title) AS cnt FROM :lt_books GROUP BY pub_year;
lt_books_proj = CE_PROJECTION(:lt_books, ["TITLE", "PUB_YEAR"]);ot_books_agg = CE_AGGREGATION(:lt_books_proj, [COUNT("TITLE") AS "CNT"], ["PUB_YEAR"]);
Disadvantages of Build-Ins
Development effort increases
Optimizer is bypassed
Advantages???
© 2012 SAP AG. All rights reserved. 16
SQLScript: Imperative vs. Declarative Statements
lt_remaining_reals = …
SELECT COUNT(*) INTO lv_remaining_reals FROM :lt_remaining_reals;
WHILE lv_remaining_reals > 0 DO lt_remaining_reals = …
SELECT COUNT(*) INTO lv_remaining_reals FROM :lt_remaining_reals;
END WHILE;
Avoid imperative statements! – Negative impact on performance
© 2012 SAP AG. All rights reserved. 17
Data Flow for Compliance Checks
Application is a huge data flow
Thin orchestration layer Bars in the diagram
Aggregate Results
Extract Constraints
Composition 2Composition 1 Composition n
Check Type 1 Check Type 2 Check Type m…
…
© 2012 SAP AG. All rights reserved. 18
Best Practices I
Manage Complexity
Encapsulate as usual using procedures
Structure as usual in packages
Parallelization
Writing Data breaks parallelization
Encapsulate writing and non-writing code separately
© 2012 SAP AG. All rights reserved. 19
SQLScript: ReadOnly vs. ReadWrite
SQL Script Client
ReadWrite Procedure
ReadOnly Procedure
© 2012 SAP AG. All rights reserved. 20
Best Practices II
Client Handling
Encapsulation in views
CREATE VIEW "EHS_CE"."V_CE2_BB_HIER" ( "OBJ_ID", "CHILD_ID", "CHILD_SEQ_NO", "CHILD_IMPACT" ) AS SELECT ZOBJ_ID AS OBJ_ID, ZCHILD_ID AS CHILD_ID, ZCHILD_SEQ_NO AS CHILD_SEQ_NO, ZCHILD_IMPACT AS CHILD_IMPACT FROM EHS_CE.ZGL_CE2_BB_HIER WHERE MANDT = SESSION_CONTEXT('client');
Requires SET SESSION statement prior to execution
SET SESSION client = '100';
© 2012 SAP AG. All rights reserved. 21
Best Practices III
Avoid ROW Store! EXPLAIN PLAN
Example: SELECT GREATEST(1, MIN(status)) AS status FROM :tab; SELECT CASE WHEN status > 1 THEN status ELSE 1 END AS status FROM :tab;
Avoid UNION (ALL)
Use Build-In CE_UNION_ALL
One and only exception when to use build-ins
Requires proper data type handling
Example: stat1 = SELECT 1 AS status FROM :tab; -- Results in a float as data type for status stat2 = SELECT TO_INTEGER(1) AS status FROM :tab; -- Results in an integer for status Compilation of stat = CE_UNION_ALL(:stat1, :stat2); fails due to incompatibility of tables
© 2012 SAP AG. All rights reserved. 22
Best Practices IV
Encapsulate Imperative Statements in Separate Procedure
No Infinity-Values in SAP HANA DB Our solution: | att_value | is_infinity |
– Is_infinity: -1, 0, 1 for -, not , – If (is_infinity ≠ 0) then att_value = NULL
MIN and MAX operations work as expected
Otherwise: Absence of Best practices Try it! Examples:
– CASE vs. Branch & CE_UNION_ALL – GROUP BY on many columns vs. GROUP BY on key + additional JOIN
© 2012 SAP AG. All rights reserved. 23
Debugging – Trace Data
Debugging Debugging requires modifications Problem in customer system
Debugging example:
-- Intermediate result to be inspected:lt_composition = SELECT … FROM :lt_raw_composition;
lt_composition = TRACE (:lt_composition);
-- Consume intermediate resultsot_composition = SELECT … FROM :lt_composition;
© 2012 SAP AG. All rights reserved. 24
Debugging – Analyze Trace
Get Trace Result Tables
SELECT * FROM sys.sqlscript_trace;
Get Trace Result
SELECT * FROM SQLSCRIPT_TRACE_4F95E715495CC1CEE10000000A428043;
© 2012 SAP AG. All rights reserved. 25
Debugging – Impact on Compilation and Execution
Statements
lt_constraints_ext = SELECT request_id, cst_id, status, CASE WHEN (status = 1) THEN to_double(NULL) ELSE to_double(0.0) END AS maximum_dosage, CASE WHEN (status = 1) THEN to_integer(1) ELSE to_integer(0) END AS maximum_dosage_infinit FROM :lt_const_ext AS ce;
lt_constraints_ext_compl = SELECT concomp.request_id AS request_id, concomp.cst_id AS cst_id, conext.status AS status, conext.maximum_dosage AS maximum_dosage, conext.maximum_dosage_infinit AS maximum_dosage_infinit, FROM :lt_constraints AS concomp LEFT OUTER JOIN :lt_constraints_ext AS conext ON concomp.request_id = conext.request_id AND concomp.cst_id = conext.cst_id;
© 2012 SAP AG. All rights reserved. 26
Debugging – Impact on Compilation and Execution
are equivalent to
lt_constraints_ext_compl = SELECT concomp.request_id AS request_id, concomp.cst_id AS cst_id, conext.status AS status, conext.maximum_dosage AS maximum_dosage, conext.maximum_dosage_infinit AS maximum_dosage_infinit, FROM :lt_constraints AS concomp LEFT OUTER JOIN ( SELECT request_id, cst_id, status, CASE WHEN (status = 1) THEN to_double(NULL) ELSE to_double(0.0) END AS maximum_dosage, CASE WHEN (status = 1) THEN to_integer(1) ELSE to_integer(0) END AS maximum_dosage_infinit FROM :lt_const_ext AS ce ) AS conext ON concomp.request_id = conext.request_id AND concomp.cst_id = conext.cst_id;
© 2012 SAP AG. All rights reserved. 27
Debugging – Impact on Compilation and Execution
That‘s what the optimizer does
This results in a Bug
See Internal message 810472/2012
CASE statement is not working properly In optimized statement status may be NULL So the ELSE part is executed and 0.0 resp. 0 is returned instead of NULL
Workaround Consider NULL explicitely as third possible value
© 2012 SAP AG. All rights reserved. 28
Debugging – Impact on Compilation and Execution
TRACE statement breaks this optimization
i.e. two statements remain
lt_constraints_ext = SELECT request_id, cst_id, status, CASE WHEN (status = 1) THEN to_double(NULL) ELSE to_double(0.0) END AS maximum_dosage, CASE WHEN (status = 1) THEN to_integer(1) ELSE to_integer(0) END AS maximum_dosage_infinit FROM :lt_const_ext AS ce;
lt_constraints_ext = TRACE(: lt_constraints_ext);
lt_constraints_ext_compl = SELECT concomp.request_id AS request_id, concomp.cst_id AS cst_id, conext.status AS status, conext.maximum_dosage AS maximum_dosage, conext.maximum_dosage_infinit AS maximum_dosage_infinit, FROM :lt_constraints AS concomp LEFT OUTER JOIN :lt_constraints_ext AS conext ON concomp.request_id = conext.request_id AND concomp.cst_id = conext.cst_id;
© 2012 SAP AG. All rights reserved. 29
Unit Tests
General Concept
Create own schema for test data tables
Create own package(s) for unit test procedures
Create overall test procedure that calls every single unit test
Single Unit Tests for each procedure
Create tables for Input data Expected output
Unit test procedure Reads input data Calls procedure to be tested Compares calcualted output with stored output Returns differences
© 2012 SAP AG. All rights reserved. 30
User Setup
Coding refers to a default schema
Background: Transporting
Avoid qualified names in coding
To simplify transports, make sure everything is in one schema No need for dedicated replication user Replicate into app user’s schema
Have a dedicated Application User (Technical User)
Second Schema for Test Data
Use schema mapping for test execution
© 2012 SAP AG. All rights reserved. 31
Links
Next Generation DB Wiki
https://wiki.wdf.sap.corp/wiki/display/ngdb/Home
Service Market Place
https://websmp109.sap-ag.de/hana
Book
Joe Celko's Thinking in Sets: Auxiliary, Temporal, and Virtual Tables in SQL
Exercise
© 2012 SAP AG. All rights reserved. 33
Business Background
Products need to adhere to a huge number of constraints
Constraints are specified by laws, authorities, industry standards, organizations, customers, …
Constraints are hierarchally grouped with four levels
Regulators that contain constraints
Sub-profiles that may contain constraints and regulators
Super Sub-profiles that may contain constraints, regulators and sub-profiles
Profiles that may contain constraints, regulators, sub-profiles and super sub-profiles
Example
Baby food sold in Germany
Two regulators: Baby Food and Food in Germany and additional constraints
All constraints of both regulators need to be fulfilled
© 2012 SAP AG. All rights reserved. 34
Data Model
Compliance Objetcs
Table ZGL_CE2_COBJ
View V_CE2_COBJ
Hierarchy
Table ZGL_CE2_BB_HIER
View V_CE2_HIER
Request
Table ZGL_REQUEST
View V_REQUEST
(all objects are located in Schema HANA_BOOTCAMP
Object IDObject TypeMaintenance State
Compliance Object
Object IDChild Object IDChild Sequence NumberChild Impact
Compliance Object Hierarchy
is child*
is father
*
Request IDObject ID
Request* 1..*
© 2012 SAP AG. All rights reserved. 35
Exercise
Create Procedure to extract constaints from the hierarchy
Use own package
Create a procedure Input is one single request ID
– One value to simplify the procedure call Output is a table of all constraints with constraint type
– Master data maintenance error is indicated by NULL for constraint type
Integration with ABAP
© 2012 SAP AG. All rights reserved. 37
Access SAP HANA via Native DB Calls
Set up Secondary Connection
Then it’s just a DB call…
© 2012 SAP AG. All rights reserved. 38
Access SAP HANA via Native DB Calls
Set up Secondary Connection
Then it’s just a DB call…
© 2012 SAP AG. All rights reserved. 39
Access SAP HANA via Native DB Calls
Set up Secondary Connection
Then it’s just a DB call…
lo_dbconn = cl_sql_connection=>get_connection( con_name = 'DEMO_111' ).
CREATE OBJECT lo_sql EXPORTING con_ref = lo_dbconn.
lo_result = lo_sql->execute_query( 'SELECT * FROM MY_HANA_TABLE"' ).
© 2012 SAP AG. All rights reserved. 40
Invoking SQLScript Procedures from ABAP
Major issue: Table transfer
lo_sql->execute_query( 'SET SESSION ''client'' = ''' && sy-mandt && '''' ).
lo_sql->execute_ddl( 'TRUNCATE TABLE "IT_PROD_IDS"' ).LOOP AT it_prod_ids INTO lv_prod_id_input. lo_sql->execute_update( 'INSERT INTO "IT_PROD_IDS" VALUES(''' && lv_prod_id_input &&''' )' ).ENDLOOP.
lo_sql->execute_ddl( 'TRUNCATE TABLE "OT_COMPOSITION"' ).
lo_result = lo_sql->
execute_query( 'CALL "_SYS_BIC"."sap.teched/EXPLOSION"( "IT_PROD_IDS", "OT_COMPOSITION" ) WITH OVERVIEW ' ).
lo_result->close( ).
lo_result = lo_sql->execute_query( 'SELECT * FROM "OT_COMPOSITION"' ).
lo_sql->execute_query( 'SET SESSION ''client'' = ''' && sy-mandt && '''' ).
lo_sql->execute_ddl( 'TRUNCATE TABLE "IT_PROD_IDS"' ).LOOP AT it_prod_ids INTO lv_pid. lo_sql->execute_update( 'INSERT INTO "IT_PROD_IDS" VALUES('''&& lv_pid &&''')' ).ENDLOOP.
lo_sql->execute_ddl( 'TRUNCATE TABLE "OT_COMP"' ).
lo_result = lo_sql->execute_query(
'CALL "_SYS_BIC"."sap.teched/EXPLOSION"( "IT_PROD_IDS", "OT_COMP" ) WITH OVERVIEW').
lo_result->close( ).
lo_result = lo_sql->execute_query( 'SELECT * FROM "OT_COMP"' ).
Exercise
© 2012 SAP AG. All rights reserved. 43
Exercise
Customer wants to know which particular checks are executed
i.e. call existing SAP Hana procedure from ABAP
Configure a Data Base Connection
Create an ABAP Class that encapsulate the call of SAP HANA
One method to call the procedure created
Create an ABAP Report to test the method
Template
You may copy the report and class from local objects of user D030764
Conclusion
© 2012 SAP AG. All rights reserved. 45
Key Takeaways
CD is ready to deliver “next generation applications“ powered by SAP HANA First customer projects started SSRS revenue (coding must be IP relevant)
The key differences in a SQLScript project are The need for data Testing is very time-intensive
In a real project, you need to account for the evolving nature of SAP HANA DB Transports are difficult impacts testing Debugging is difficult esp. in customer landscape
Thank you
Contact information:
Christian BehrensPrincipal Solution Architect SAP Custom Development