![Page 1: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/1.jpg)
pr ligence Empowering Intelligence
Summary ManagementBy Advanced Queues
Paper # 419
Arup Nanda
Proligence, Inc
www.proligence.com
![Page 2: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/2.jpg)
pr ligence Empowering Intelligence
Real Time Datawarehouses
SourceDW
Source
Source
Summary
PullLog Push
![Page 3: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/3.jpg)
pr ligence Empowering Intelligence
Complex Query
• Query to find out the eligibility of a transaction based on data in other tables – complex joins
• Takes up too much time in the Query
![Page 4: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/4.jpg)
pr ligence Empowering Intelligence
Actual ExampleSELECT DISTINCT NVL(C.COL1, R.COL1) AS COL1,RQ.COL2, COUNT(C.COL3) AS COUNTERFROM TAB1 R, TAB2 C, TAB3 CP, TAB4 RQ WHERE R.COL4 = C.COL4 AND C.COL1 = RQ.COL1 AND ( CP.COL5 = :b0OR (CP.COL5 = :b1 AND CP.COL6 = :b2)) AND CP.COL3 = C.COL3 AND NOT EXISTS (SELECT 1 FROM TAB5 CCL,TAB6 CL, TAB7 CHO WHERE CCL.COL7 = CL.COL7AND CL.COL8 = CHO.COL8AND CCL.COL3 = C.COL3AND CHO.COL1 = 'Y') GROUP BY NVL(C.COL1, R.COL1), RQ.COL2 ORDER BY RQ.COL2
![Page 5: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/5.jpg)
pr ligence Empowering Intelligence
Options
• Materialized View– Fast Refresh Needed– Complex – Fast Refresh Not Allowed
• Materialized View of MVs– Too Many Levels – Difficult to Administer– Refresh Lags– Failure Prompts Calls to DBMS_MVIEW()
![Page 6: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/6.jpg)
pr ligence Empowering Intelligence
Options
• A Summary Table to Hold the Data• Triggers Set Up on Source Tables
T1
T2
T3
T4
SUMMARYTABLE
Trigger
Trigger
Trigger
Trigger
![Page 7: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/7.jpg)
pr ligence Empowering Intelligence
Example
• Tables– DEPT– EMP
• QuerySELECT DEPTNO, COUNT(*) EMP_COUNTSFROM EMPWHERE STATUS = 'ACTIVE'GROUP BY DEPTNO
![Page 8: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/8.jpg)
pr ligence Empowering Intelligence
Example Summary Table
• Table DEPT_COUNTS– DEPTNO– EMP_COUNTS
DEPTNO EMP_COUNTS1 1202 3403 230
![Page 9: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/9.jpg)
pr ligence Empowering Intelligence
LogicA Row Is Inserted
STATUS = ‘ACTIVE’?
Check the Existence of the DEPTNO in DEPT_COUNTS
Exists?
Upadate DEPT_COUNTS
Set EMP_COUNTS=EMP_COUNTS+1
Insert into DEPT_COUNTS
EMP_COUNTS=1DEPTNO=..
Y
NY
SELECT DEPTNO, COUNT(*) EMP_COUNTSFROM EMPWHERE STATUS = 'ACTIVE'GROUP BY DEPTNO
![Page 10: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/10.jpg)
pr ligence Empowering Intelligence
Flaw
Time 0
DEPTNO EMP_COUNTS1 1202 340
![Page 11: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/11.jpg)
pr ligence Empowering Intelligence
Flaw contd…
Time 1Session1 INSERTS into EMP (DEPTNO=3, EMPNO=5679)
DEPTNO EMP_COUNTS1 1202 340
![Page 12: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/12.jpg)
pr ligence Empowering Intelligence
Flaw contd…
Time 2Trigger Finds No Record for DEPTNO=3INSERTS into DEPT_COUNTS
DEPTNO EMP_COUNTS1 1202 3403 1 New
![Page 13: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/13.jpg)
pr ligence Empowering Intelligence
Flaw contd…
Time 3Session2 Inserts into EMP(DEPTNO=3, EMPNO=4567)
DEPTNO EMP_COUNTS1 1202 3403 1 Not Visible
![Page 14: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/14.jpg)
pr ligence Empowering Intelligence
Flaw contd…
Time 4Trigger tries to INSERT into DEPT_COUNTS(DEPTNO=3, EMP_COUNTS=1)
DEPTNO EMP_COUNTS1 1202 3403 1 Primary Key Violated
WAITS!
![Page 15: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/15.jpg)
pr ligence Empowering Intelligence
Flaw contd…
Time 5Session1 Commits!
DEPTNO EMP_COUNTS1 1202 3403 1 Row Is Present
![Page 16: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/16.jpg)
pr ligence Empowering Intelligence
Flaw contd…
Time 6Session2 Errors Out!
DEPTNO EMP_COUNTS1 1202 3403 1 Row Is Present
Error!
![Page 17: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/17.jpg)
pr ligence Empowering Intelligence
Problem
• Locking• Primary Key Violation• Solution
Session1
Session2
Trans1
Trans2 SummaryTable
ApplyingSession
![Page 18: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/18.jpg)
pr ligence Empowering Intelligence
Advanced Queues
User
Producer
User
Message Payload Consumer
enqueue dequeue
![Page 19: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/19.jpg)
pr ligence Empowering Intelligence
Complex
Producer1
Tags
Producer2
Producer3
Consumer1
Consumer2
![Page 20: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/20.jpg)
pr ligence Empowering Intelligence
Types of AQs
• Persistent• Non Persistent• Single Producer/Consumer• Multiple Producers/Consumers• Publisher-Subscriber Model
![Page 21: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/21.jpg)
pr ligence Empowering Intelligence
Queue Table (QT)Payload defined here
QueueException QueueDBMS_AQADMDBMS_AQ
AQ Components
![Page 22: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/22.jpg)
pr ligence Empowering Intelligence
Solution
Table
Trigger
Queue
QueueTable
EnQueue DeQueue
SummaryTable
Payload
![Page 23: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/23.jpg)
pr ligence Empowering Intelligence
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 24: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/24.jpg)
pr ligence Empowering Intelligence
Database Prep
• Turn on the queue timer processes– aq_tm_processes = 2– ALTER SYSTEM possible too– Process Identified as QMNn
• Grant privileges– GRANT EXECUTE ON DBMS_AQADM TO SCOTT;
– GRANT EXECUTE ON DBMS_AQ TO SCOTT;
![Page 25: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/25.jpg)
pr ligence Empowering Intelligence
Payload
• Oracle Object Type
create or replace type dept_counts_type as object
( action_type char(1), old_deptno number(2), new_deptno number(2))
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 26: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/26.jpg)
pr ligence Empowering Intelligence
Processing
• Procedure PROCESS_DEPT_COUNTS()• Input Parameters
– Action – I, D, U, T– Old DeptNo– New DeptNo
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 27: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/27.jpg)
pr ligence Empowering Intelligence
Queue Table
begin DBMS_AQADM.CREATE_QUEUE_TABLE ( queue_table => 'DEPT_COUNTS_QT', queue_payload_type=> 'DEPT_COUNTS_TYPE', multiple_consumers=> FALSE, storage_clause => 'TABLESPACE USR INITRANS 10 STORAGE (FREELISTS 10 FREELIST GROUPS 2)',
compatible => '8.1');end;
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 28: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/28.jpg)
pr ligence Empowering Intelligence
Queue
begin DBMS_AQADM.CREATE_QUEUE ( queue_name =>'DEPT_COUNTS_Q', queue_table=>'DEPT_COUNTS_QT', max_retries=>'5', retry_delay=>'0');
dbms_aqadm.start_queue( 'DEPT_COUNTS_Q',TRUE,TRUE);
end;
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 29: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/29.jpg)
pr ligence Empowering Intelligence
Enqueue Procedure
create or replace procedure enq_dept_counts_q (p_msg in dept_counts_type)as enq_opt dbms_aq.enqueue_options_t; msg_prop dbms_aq.message_properties_t; msg_id raw(16);begin sys.dbms_aq.enqueue ( 'DEPT_COUNTS_Q',enq_opt, msg_prop, p_msg, msg_id);end;
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 30: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/30.jpg)
pr ligence Empowering Intelligence
Dequeue Procedure
create or replace procedure deq_dept_counts_qas deq_opt dbms_aq.dequeue_options_t; msg_prop dbms_aq.message_properties_t; payload dept_counts_type; msgid raw(16);begin loop deq_opt.wait := dbms_aq.forever; deq_opt.navigation := dbms_aq.next_message; dbms_aq.dequeue( 'DEPT_COUNTS_Q', deq_opt, msg_prop, payload, msgid); process_dept_counts(payload.action_type, payload.old_deptno, payload.new_deptno); commit; end loop;end;
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 31: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/31.jpg)
pr ligence Empowering Intelligence
Trigger
create or replace trigger tr_ar_iud_emp after insert or delete or update on empfor each rowdeclare l_action char(1); l_old_deptno number(2); l_new_deptno number(2);begin enq_dept_counts_q( dept_counts_type( l_action, l_old_deptno, l_new_deptno));end;
Table
Trigger Queue
QueueTable
EnQueue
DeQueue
SummaryTable
Payload
![Page 32: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/32.jpg)
pr ligence Empowering Intelligence
Solution Revisited
Table
Trigger
Queue
QueueTable
EnQueue DeQueue
SummaryTable
Payload
![Page 33: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/33.jpg)
pr ligence Empowering Intelligence
Important Considerations
• Asynchronous• DecoupledTable
Trigger
Queue
QueueTable
EnQueue DeQueue
SummaryTable
Payload
Transaction 1 Transaction 2
![Page 34: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/34.jpg)
pr ligence Empowering Intelligence
Administration
• Number of Messages in Queue
select count(*) from AQ$DEPT_COUNTS_QT where queue = 'DEPT_COUNTS_Q'
![Page 35: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/35.jpg)
pr ligence Empowering Intelligence
Usage
• Datawarehouse• OLTP Complex Queries• Data from Heterogeneous Sources
– MQ Series• Maintaining Flattened Tables in OLTP
![Page 36: Prligence Empowering Intelligence Summary Management By Advanced Queues Paper # 419 Arup Nanda Proligence, Inc](https://reader036.vdocument.in/reader036/viewer/2022062511/5517757355034645368b4de6/html5/thumbnails/36.jpg)
pr ligence Empowering Intelligence
Thank you!
www.proligence.cowww.proligence.comm
Summary ManagementBy Advanced Queues
Paper # 419