bb world 2012 using database statistics to make capacity planning decisions (1)
TRANSCRIPT
Using Database Statistics to Make Capacity Planning
DecisionsGeoff Mower
Senior Software Performance Engineer, Blackboard
Steve FeldmanSenior Director Software Performance Engineering and Architecture, Blackboard
Who Are We?Geoff Mower- Senior Software Performance Engineer
5 years on the frontlines as a DBA
2 hour interview on a plane
Performance and Scalability for
Outcomes & Community
2 hour interview on a plane
Who Are We?Steve Feldman
• Blackboard since 2003• Performance Engineering
from the start• Platform Architecture in
20052005• Security Engineering in
2010“Love my job…love my team. If you email me, I will respond.”
@seven_seconds
http://goo.gl/Z4Rq5
What We Want You to Do?
Mine your data Ask follow-up questionsask questions Mine your data Ask follow-up questionsask questions
about your
data
What are our obstacles?
• Certain data points are transient– Logging will need to be created in certain situations
to fill in missing data points.• See slide #45 & #46 for an example• See slide #45 & #46 for an example
What You Need
• Permissions to run SQL scripts on your Blackboard Learn environment's database
• Basic SQL fundamentals• Analysis of results• Analysis of results• Database Tools
– Aquadata client & Server or Quest’s Toad
Skill and Knowledge SetsOpen DB Documentation
• Blackboard Learn™, Release 9.1 Open Database Document– http://bit.ly/loW9hi
Asking the Right Questions
Collecting the Right Data
Organizing and Analyzing the Data
Ask Follow-up Questions
Making Decisions
Types of Data
Data Types Table Example
Event ACTIVITY_ACCUMULATOR User navigation
Process COURSE_USERS snapshot
User COURSE_CONTENT User created User COURSE_CONTENT User created
entities
Static LEARN_admin.bb_instance installation
Transient SESSIONS Tracks who has
an active session
Areas of Capacity Planning
• How can the Blackboard Learn schema help you capacity plan for your business requirements in regards to?
Strategic Functional Decisions
Expense and Budget Management
Administrative Planning
Administrative Planning
Administrative Planning
To be able to restore individual courses, can we use the course archive tool as a backup function?
Administrative PlanningAsking the Right Questions
How will we know which courses need archiving?
Frequency of course archive task?
Will the archiving process impact user experience?Will the archiving process impact user experience?
Does the course archive tool have limitation?
Location of course archives?
Administrative PlanningCollecting the Right Data
How will we know which courses need archiving?– Course entity creation and modification
Creation TablesCOURSE_CONTENTS
GRADEBOOK_MAIN
Modification TablesCOURSE_CONTENTS
GRADEBOOK_MAINGRADEBOOK_MAIN
GRADEBOOK_GRADES
QTI_ASI_DATA
QTI_RESULT_DATA
FORUM_MAIN
MSG_MAIN
GRADEBOOK_MAIN
GRADEBOOK_GRADES
QTI_ASI_DATA
QTI_RESULT_DATA
FORUM_MAIN
MSG_MAIN
CALENDAR
STAFFINFORMATION
Administrative PlanningOrganizing and Analyzing the Data
• Data Points: Course/Organizations to be archived SQL/Oracle
COURSE_ID Number of New/Modified Course Entities
Hist_101 42
Eng_252 42
Sci_139 38
Eng_429 37
Art_202 35
Eco_323 35
Administrative PlanningMaking Decisions from the Data
• How many new or modified course/organization entities are required for a course/organization to be archived?
• How many courses/organizations should we archive each time the archive task is execution?
• What is the maximum time between archive tasks?
Administrative PlanningActing on Your Descion
SQL/OracleHist_101, c:\Course_Archives\2011-07-09\Eng_252, c:\Course_Archives\2011-07-09\Sci_139, c:\Course_Archives\2011-07-09\Eng_429, c:\Course_Archives\2011-07-09\……• Create a JOB that would run the SQL script to generate
a txt file containing the query results
• Create a schedule task that would run the batch_ImportExport.bat tool and direct it to the txt file created by the JOB in the previous step.
Expense and Budget Management
Asking the Right QuestionsExpense and Budget Management
• How much do we need to budget for in order to fund our database requirements?
Expense and Budget Management
• When are we going to hit our 95% disk spaced used threshold.
Expense and Budget Management
• Available disk space issue– Talk to Network Security Manager (owner of the SAN)
• We need more disk space
Expense and Budget ManagementAsking the Right Questions
• What did I miss?– When will we run out of space?– How long do we expect the requested disk space to
last us?
• Facts vs. guess work
Expense and Budget ManagementCollecting the Right Data
• Data Points: Database file size• Binary• Data Files (.mdf)• TempDB• Transaction Logs (.ldf)• Transaction Logs (.ldf)• Back-up/restore process
Expense and Budget ManagementOrganizing and Analyzing the Data
• Data Points: Database file size SQL/OracleDatabase Name Physical Name Size in MB
BBLEARN_stats F:\data\BBLEARN_stats_data.mdf 70338
BBLEARN F:\data\BBLEARN_data.mdf 45520
cms_files_inst F:\data\cms_files_inst_data.mdf 5704
BBLEARN G:\logs\BBLEARN_log.ldf 3083BBLEARN G:\logs\BBLEARN_log.ldf 3083
cms_files_inst G:\logs\cms_files_inst_log.ldf 1361
cms_files_courses F:\data\cms_files_courses_data.mdf 1072
cms_files_users F:\data\cms_files_users_data.mdf 926
msdb F:\data\MSSQL.1\MSSQL\DATA\MSDBData.mdf 792
cms_files_library F:\data\cms_files_library_data.mdf 720
cms_files_orgs F:\data\cms_files_orgs_data.mdf 384
msdb F:\data\MSSQL.1\MSSQL\DATA\MSDBLog.ldf 259
cms_files_users G:\logs\cms_files_users_log.ldf 214
cms_files_courses G:\logs\cms_files_courses_log.ldf 207
Expense and Budget ManagementOrganizing and Analyzing the Data
• Data Points: Table Row Count and Sizes SQL/Oracle
Table Name
Row
Count Reserved Data Index Size Unused
ACTIVITY_ACCUMULATOR 377266616 84097056 46905888 37183888 7280
COURSE_USERS 3837931 496024 258272 237496 256
USERS 303687 139480 96848 42440 192
COURSE_MAIN 121623 86024 47760 37984 280
NAVIGATION_ITEM 1876 2032 608 1128 296
SYSTEM_TRACKING 3550 1936 1672 160 104
USER_ROLES 4114 544 160 288 96
Expense and Budget ManagementOrganizing and Analyzing the Data
Data Points: Table growth rate (by year) of the Activity_Accumulator SQL/Oracle
100000000
120000000
0
20000000
40000000
60000000
80000000
2003 2004 2005 2006 2007 2008 2009 2010
Expense and Budget ManagementOrganizing and Analyzing the Data
Data Points: ACTIVITY_ACCUMULATOR growth rate (by month) SQL/Oracle
10000000
12000000
0
2000000
4000000
6000000
8000000
Expense and Budget Management
• Data Points: COURSE_USERS table growth rate (by month)
700000
800000
900000
0
100000
200000
300000
400000
500000
600000
August August August
Expense and Budget ManagementMaking Decisions from the Data
• Conclusion• Due to opening of school tasks we will see our
available disk space decrease below 5% during the month of August.
• In order to make it 3 years we will need a minimum • In order to make it 3 years we will need a minimum 120 GB.
Strategic Functional Decisions
Strategic Functional Decisions
• Can our database support all classroom testing if it was online?
Strategic Functional DecisionsAsking the Right Questions
• Study…• Online & offline test taking populations• The tests themselves• Test taking behavior • How testing impacts the database• How testing impacts the database• Locating when you will see resource contention
Strategic Functional DecisionsHistorical Use of Online Testing
What is the percent of students from the previous term that have completed a test online? SQL/Oracle
Online
Offline
Strategic Functional DecisionsCollecting the Right Data
Previous Use Full Adoption
Courses with Tests 11,274 55,504
Enrollments with Tests 225,480 1,110,080
Number of questions
submitted in a term
4,219,032 25,694,266
Peak Question Submission 60,346 297,094Peak Question Submission
by hour
60,346 297,094
Strategic Functional DecisionsCollecting the Right Data
• Determine CPU and I/O cost of testing (SQL server)– Track the Stored Procedures performing the works
SQL/Oracle» qti_result_data_cr» update_gradebook_grade» attempt_cr» attempt_cr» update_qti_res_bbmd_dtmod» gb2_is_log_enabled» gb2_attempt_logger
Strategic Functional DecisionsOrganizing and Analyzing the Data
• Determine CPU and I/O cost of testing (SQL server)– Track the Stored Procedures performing the works SQL/Oracle
Strategic Functional DecisionsOrganizing and Analyzing the Data
• Determine CPU and I/O cost of testing– Track the queries performing the work SQL/Oracle
Strategic Functional DecisionsOrganizing and Analyzing the Data
Determine IO cost of all cached stored procedures SQL/Oracle
500
600
700
Average IO
0
100
200
300
400
Strategic Functional DecisionsMaking Decisions from the Data
• A reasonable understanding…• How tests are currently used• How tests will be used• How much each question costs the database upon
submissionsubmission
• Lack information on…• Baseline of database’s IO and CPU resources• Non-testing database transactions
Strategic Functional DecisionsCollecting the Right Data
• Determine the breaking point of CPU and I/O of testing– Create a CPU logging script SQL/Oracle
• SQL server logging limitation– 4 hours of CPU tracking– Lacks IO counters
– Create a IO logging script SQL/Oracle• Snapshot in time
– Need to log the delta between the last logged IO metric and current query results
– Create a Query logging script SQL/Oracle• Compares test question submission to all other transactions
Strategic Functional DecisionsCollecting the Right Data
• Create a separate database from the Blackboard Learn’s databases (you can use the database server Blackboard Learn is running on)
• Create tables with in the new database that match • Create tables with in the new database that match the query results you wish to store
• Create a JOB that will run each query and insert the results into the corresponding table and database.
Capacity Planning
is an art if done
properly…
or a disaster if not
done at all.
[email protected]@blackboard.com
@@gvmowergvmower
@@seven_secondsseven_seconds
sevenseconds.wordpress.comsevenseconds.wordpress.com
[email protected]@blackboard.com
http://goo.gl/Z4Rq5
Please provide feedback for this session by [email protected].
The title of this session is:
Using Database Statistics to Make Capacity Planning Decisions
Appendices
Administrative PlanningWhat courses meet the course archive criteria
declare @Last_Archive datetime
declare @Datediff int
declare @Number_of_New_Entities int
set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ran
set @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archived
select course_id, CC.CC + GM.GM + GG.GG + QA.QA + A.A + ConM.CONFERENCE_MAIN + FM.FORUM_MAIN + MM.MSG_MAIN + C.C + S.S AS [Number of New/Modified Course Entities]
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > @Last_Archive
or cc.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > @Last_Archive
or date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gm.date_added) > @Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > @Last_Archive
OR gg.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, gg.date_added) > @Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > @Last_Archive
OR bbmd_date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > @Last_Archive
or a.date_modified > @Last_Archive)
and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediffand datediff(hour, cm.dtcreated, a.attempt_date) > @Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > @Last_Archive
OR CON.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > @Last_Archive
OR fr.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > @Last_Archive
OR mm.dtmodified > @Last_Archive)
and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, c.dtmodified) > @Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > @Last_Archive
and datediff(hour, cm.dtcreated, s.dtmodified) > @Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
Administrative PlanningWhat courses meet the course archive criteria
set serveroutput on size 100000
---Administrative PlanningWhat courses meet the course archive criteria
declare
v_Last_Archive date;
v_Datediff int;
v_Number_of_New_Entities int;
v_Directory varchar(256);
v_File_Name varchar(256);
Begin
v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran
v_Datediff := 1 ; --this is the day
--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
v_Number_of_New_Enti ties := 1;
--Enter the number of new course entities required for a course to be archived
select 'usr\local\blackboard\Course_Archi ves \' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '\' into v_Directory FROM Dual;
--Alter the string to build the directory you wish to store your archives
dbms_output.enable(9999999);
dbms_output.new_line();
FOR rec IN (
select course_id | | v_Directory as output
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > v_Last_Archive
or cc.dtmodified > v_Last_Archive)
and cm.dtcreated- cc.dtcreated> v_Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > v_Last_Archive
or date_modified > v_Last_Archive)
and cm.dtcreated- gm.date_added> v_Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > v_Last_Archive
OR gg.date_modified > v_Last_Archive)
and cm.dtcreated- gg.date_added> v_Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > v_Last_Archive
OR bbmd_date_modified > v_Last_Archive)
and cm.dtcreated- qa.bbmd_date_added> v_Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > v_Last_Archive
or a.date_modified > v_Last_Archive)
and cm.dtcreated- a.attempt_date > v_Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > v_Last_Archive
OR CON.dtmodified > v_Last_Archive)
and cm.dtcreated-con.dtcreated > v_Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > v_Last_Archive
OR fr.dtmodified > v_Last_Archive)
and cm.dtcreated- fr.dtcreated > v_Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > v_Last_Archive
OR mm.dtmodifi ed > v_Last_Archive)
and cm.dtcreated- mm.dtcreated > v_Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > v_Last_Archive
and cm.dtcreated- c.dtmodified > v_Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > v_Last_Archive
and cm.dtcreated- s.dtmodified > v_Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )
LOOP
dbms_output.put_line (rec.output);
END LOOP;
dbms_output.new_line();
end;
Administrative PlanningWhat courses meet the course archive criteria
declare @Last_Archive datetimedeclare @Datediff intdeclare @Number_of_New_Entities intdeclare @Directory varchar(256)declare @File_Name varchar(256)
set @Last_Archive = '2011-6-7' --Enter the last datetime the course archival process ranset @Datediff = 24 --Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.set @Number_of_New_Entities = 1 --Enter the number of new course entities required for a course to be archivedset @Directory = (SELECT ', c:\Course_Archives\' + CAST(CONVERT(date, GETDATE()) as varchar(10)) + '\') --Alter the string to build the directory you wish to store your archives
select course_id + @Directoryfrom course_main cmleft join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents ccleft join course_main cm on cm.pk1 = cc.crsmain_pk1where (cc.dtcreated > @Last_Archiveor cc.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, cc.dtcreated) > @Datediffgroup by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GMfrom gradebook_main gmleft join course_main cm on cm.pk1 = gm.crsmain_pk1where (date_added > @Last_Archiveor date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, gm.date_added) > @Datediffgroup by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GGfrom gradebook_main gmleft join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1left join course_main cm on gm.crsmain_pk1 = cm.pk1where (gg.date_added > @Last_ArchiveOR gg.date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, gg.date_added) > @Datediffgroup by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QAfrom qti_asi_data qaleft join course_main cm on cm.pk1 = qa.crsmain_pk1where (bbmd_date_added > @Last_ArchiveOR bbmd_date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, qa.bbmd_date_added) > @Datediffgroup by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) Afrom attempt aleft join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1left join course_main cm on cm.pk1 = gm.crsmain_pk1where (a.attempt_date > @Last_Archiveor a.date_modified > @Last_Archive)or a.date_modified > @Last_Archive)and datediff(hour, cm.dtcreated, a.attempt_date) > @Datediffgroup by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAINfrom conference_owner COleft join conference_main CON on CON.conference_owner_pk1 = CO.pk1left join course_main cm on cm.pk1 = co.owner_pk1where co.owner_table = 'COURSE_MAIN'and (CON.dtcreated > @Last_ArchiveOR CON.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, con.dtcreated) > @Datediffgroup by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAINfrom conference_owner COleft join conference_main CON on CON.conference_owner_pk1 = CO.pk1left join forum_main fr on fr.confmain_pk1 = CON.pk1left join course_main cm on cm.pk1 = co.owner_pk1where owner_table = 'COURSE_MAIN'and (fr.dtcreated > @Last_ArchiveOR fr.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, fr.dtcreated) > @Datediffgroup by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAINfrom conference_owner COleft join conference_main CON on CON.conference_owner_pk1 = CO.pk1left join forum_main fr on fr.confmain_pk1 = CON.pk1left join msg_main mm on mm.forummain_pk1 = fr.pk1left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1left join course_main cm on cm.pk1 = co.owner_pk1where owner_table = 'COURSE_MAIN'and (mm.dtcreated > @Last_ArchiveOR mm.dtmodified > @Last_Archive)and datediff(hour, cm.dtcreated, mm.dtcreated) > @Datediffgroup by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) Cfrom calendar cleft join course_main cm on cm.pk1 = c.crsmain_pk1where c.dtmodified > @Last_Archiveand datediff(hour, cm.dtcreated, c.dtmodified) > @Datediffgroup by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) Sfrom staffinformation Sleft join course_main cm on cm.pk1 = s.crsmain_pk1where s.dtmodified > @Last_Archiveand datediff(hour, cm.dtcreated, s.dtmodified) > @Datediffgroup by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where ISNULL(CC.CC,0) + ISNULL(GM.GM,0) + ISNULL(GG.GG,0) + ISNULL(QA.QA,0) + ISNULL(A.A,0) + ISNULL(ConM.CONFERENCE_MAIN,0) + ISNULL(FM.FORUM_MAIN,0) + ISNULL(MM.MSG_MAIN,0) + ISNULL(S.S,0) + ISNULL(C.C,0) >= @Number_of_New_Entities
Administrative PlanningWhat courses meet the course archive criteria
set serveroutput on size 100000
---Administrative PlanningWhat courses meet the course archive criteria
declare
v_Last_Archive date;
v_Datediff int;
v_Number_of_New_Entities int;
v_Directory varchar(256);
v_File_Name varchar(256);
Begin
v_Last_Archive := TO_DATE('7-JUN-2011'); --Enter the last datetime the course archival process ran
v_Datediff := 1 ; --this is the day
--Enter the time duration between course creation and course entities you would like to track. For example, to avoid archive courses that were created ----since the last archive process enter the time of your longest running snapshot or SIS process in terms of hours.
v_Number_of_New_Enti ties := 1;
--Enter the number of new course entities required for a course to be archived
select 'usr\local\blackboard\Course_Archi ves \' | | TO_CHAR(SYSDATE, 'DD-Mon-YYYY') | | '\' into v_Directory FROM Dual;
--Alter the string to build the directory you wish to store your archives
dbms_output.enable(9999999);
dbms_output.new_line();
FOR rec IN (
select course_id | | v_Directory as output
from course_main cm
left join (select crsmain_pk1, COUNT(cc.pk1) CC
from course_contents cc
left join course_main cm on cm.pk1 = cc.crsmain_pk1
where (cc.dtcreated > v_Last_Archive
or cc.dtmodified > v_Last_Archive)
and cm.dtcreated- cc.dtcreated> v_Datediff
group by crsmain_pk1) CC on CC.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gm.pk1) GM
from gradebook_main gm
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (date_added > v_Last_Archive
or date_modified > v_Last_Archive)
and cm.dtcreated- gm.date_added> v_Datediff
group by crsmain_pk1) GM on GM.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(gg.pk1) GG
from gradebook_main gm
left join gradebook_grade gg on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on gm.crsmain_pk1 = cm.pk1
where (gg.date_added > v_Last_Archive
OR gg.date_modified > v_Last_Archive)
and cm.dtcreated- gg.date_added> v_Datediff
group by crsmain_pk1) GG on GG.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(qa.pk1) QA
from qti_asi_data qa
left join course_main cm on cm.pk1 = qa.crsmain_pk1
where (bbmd_date_added > v_Last_Archive
OR bbmd_date_modified > v_Last_Archive)
and cm.dtcreated- qa.bbmd_date_added> v_Datediff
group by crsmain_pk1) QA on QA.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(a.pk1) A
from attempt a
left join gradebook_grade gg on a.pk1 = gg.last_attempt_pk1
left join gradebook_main gm on gm.pk1 = gg.gradebook_main_pk1
left join course_main cm on cm.pk1 = gm.crsmain_pk1
where (a.attempt_date > v_Last_Archive
or a.date_modified > v_Last_Archive)
and cm.dtcreated- a.attempt_date > v_Datediff
group by crsmain_pk1) A on A.crsmain_pk1 = cm.pk1
left join (select co.owner_pk1, COUNT(CON.pk1) CONFERENCE_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where co.owner_table = 'COURSE_MAIN'
and (CON.dtcreated > v_Last_Archive
OR CON.dtmodified > v_Last_Archive)
and cm.dtcreated-con.dtcreated > v_Datediff
group by owner_pk1) ConM on ConM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(fr.pk1) FORUM_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (fr.dtcreated > v_Last_Archive
OR fr.dtmodified > v_Last_Archive)
and cm.dtcreated- fr.dtcreated > v_Datediff
group by owner_pk1) FM on FM.OWNER_PK1 = cm.pk1
left join (select co.owner_pk1, COUNT(mm.pk1) MSG_MAIN
from conference_owner CO
left join conference_main CON on CON.conference_owner_pk1 = CO.pk1
left join forum_main fr on fr.confmain_pk1 = CON.pk1
left join msg_main mm on mm.forummain_pk1 = fr.pk1
left join msg_main mm_reply on mm_reply.msgmain_pk1 = mm.pk1
left join course_main cm on cm.pk1 = co.owner_pk1
where owner_table = 'COURSE_MAIN'
and (mm.dtcreated > v_Last_Archive
OR mm.dtmodifi ed > v_Last_Archive)
and cm.dtcreated- mm.dtcreated > v_Datediff
group by owner_pk1) MM on MM.OWNER_PK1 = cm.pk1
left join (select crsmain_pk1, COUNT(c.pk1) C
from calendar c
left join course_main cm on cm.pk1 = c.crsmain_pk1left join course_main cm on cm.pk1 = c.crsmain_pk1
where c.dtmodified > v_Last_Archive
and cm.dtcreated- c.dtmodified > v_Datediff
group by crsmain_pk1) C on C.crsmain_pk1 = cm.pk1
left join (select crsmain_pk1, COUNT(s.pk1) S
from staffinformation S
left join course_main cm on cm.pk1 = s.crsmain_pk1
where s.dtmodified > v_Last_Archive
and cm.dtcreated- s.dtmodified > v_Datediff
group by crsmain_pk1) S on S.crsmain_pk1 = cm.pk1
where nvl(CC.CC,0) + nvl(GM.GM,0) + nvl(GG.GG,0) + nvl(QA.QA,0) + nvl(A.A,0) + nvl(ConM.CONFERENCE_MAIN,0) + nvl(FM.FORUM_MAIN,0) + nvl(MM.MSG_MAIN,0) + nvl(S.S,0) + nvl(C.C,0) >= v_Number_of_New_Entities )
LOOP
dbms_output.put_line (rec.output);
END LOOP;
dbms_output.new_line();
end;
Expense and Budget ManagementDatabase File Size
SELECT DB_NAME(database_id) AS DatabaseName, Name AS Logical_Name, Physical_Name, (size*8)/1024 SizeMBFROM sys.master_filesorder by SizeMB desc
Expense and Budget ManagementDatabase File Size
select (select sum(bytes)/1024/1024 from dba_data_files)+(select sum(bytes)/1024/1024 from dba_temp_files) "Size in MB" from dual;
Expense and Budget ManagementTable Row Count and Sizes
select IDENTITY( int ) AS pk1, name into Table_List from sys.tables
CREATE TABLE Table_Results(name nvarchar(256),rows int,reserved nvarchar(256),data nvarchar(256),index_size nvarchar(256),unused nvarchar(256))
declare @pk1 intdeclare @table nvarchar(256)declare @end intset @pk1 = 0set @pk1 = 0set @end = (select top 1 PK1 from Table_List order by pk1 Desc)
WHILE @pk1 < @endbeginset @pk1 = @pk1 + 1set @table = (select name from Table_List where pk1 = @pk1)
insert into Table_Resultsexec sp_spaceused @table
end
select *from table_results
drop table table_listdrop table table_results
Expense and Budget ManagementTable Row Count and Sizes
Select OWNER, TABLE_NAME, TABLESPACE_NAME, NUM_ROWS, PCT_FREE , PCT_USED , (blocks*8/1024,2) as MB from dba_tablesorder by OWNER, TABLESPACE_NAME, TABLE_NAME;
Expense and Budget ManagementTable growth rate (by year)
/* depending on the table, the dtcreated could need to be changed to timestamp */
select YEAR(dtcreated), COUNT(*) from course_contents --Enter the table name heregroup by YEAR(dtcreated)
Expense and Budget ManagementTable growth rate (by year)
select TO_char(dtcreated,'YYYY') , COUNT(*) from course_contents --Enter the table name heregroup by TO_char(dtcreated,'YYYY') ;
Expense and Budget ManagementTable growth rate (by month)
/* depending on the table, the dtcreated could need to be changed to timestamp */
select YEAR(dtcreated), MONTH(dtcreated), COUNT(*)from course_main --Enter the table name heregroup by YEAR(dtcreated), MONTH(dtcreated)
Expense and Budget ManagementTable growth rate (by month)
select TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') , COUNT(*) from course_contents --Enter the table name heregroup by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM')order by TO_char(dtcreated,'YYYY'), TO_char(dtcreated,'MM') ;
Strategic Functional DecisionsWhat is the percent of enrollments from the previous term that have
completed a test online?
declare @start_date datetimedeclare @end_date datetimedeclare @TotalPopulation intdeclare @TestingPopulation int
set @start_date = '2010-9-1 00:00:00:000'set @end_date = '2011-9-1 00:00:00:000'
set @TotalPopulation = (select Count(distinct users_pk1)from course_users culeft join course_main cm on cm.pk1 = cu.crsmain_pk1where role = 'S'and dtcreated > @start_dateand dtcreated < @end_date)
set @TestingPopulation = (select Count(Distinct cu.users_pk1)from course_users culeft join gradebook_grade gg on gg.course_users_pk1 = cu.pk1left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1left join course_main cm on cm.pk1 = cu.crsmain_pk1where gm.qti_asi_data_pk1 > 0and gg.first_attempt_pk1 > 0and dtcreated > @start_dateand dtcreated < @end_date)
select @TestingPopulation as Online_Testing, @TotalPopulation - @TestingPopulation as Offline_Testing
Strategic Functional DecisionsWhat is the percent of enrollments from the previous term that have
completed a test online?
set serveroutput on size 100000
declare v_start_date date;
v_end_date date;
v_TotalPopulation int;
v_TestingPopulation int;
Begin
dbms_output.enable(9999999);
dbms_output.new_line();
v_start_date := to_date('1-JAN-2010');
v_end_date := to_date('1-JAN-2012');
select Count(distinct users_pk1) into v_TotalPopulation
from course_users cu
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where role = 'S'
and dtcreated > v_start_date
and dtcreated < v_end_date;
select Count(Distinct cu.users_pk1) into v_TestingPopulation
from course_users cu
left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1
left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1
left join course_main cm on cm.pk1 = cu.crsmain_pk1
where gm.qti_asi_data_pk1 > 0
and gg.first_attempt_pk1 > 0
and dtcreated > v_start_date
and dtcreated < v_end_date;
dbms_output.put_line (v_TestingPopulation || ',' || to_char(v_TotalPopulation - v_TestingPopulation));
--select v_TestingPopulation as Online_Testing, v_TotalPopulation - v_TestingPopulation as Offline_Testing;
end;
Strategic Functional DecisionsHow many tests have been completed per enrollment?
declare @enrollment_start datetimedeclare @enrollment_end datetime
set @enrollment_start = '2010-9-1 00:00:00:000'set @enrollment_end = '2011-9-1 00:00:00:000'
select NumberOfTests, COUNT(NumberOfTests) NumberOfEnrollmentsinto #BinTestsfrom (
select AllStudents.users_pk1, NumberOfTests = Case When TestingStudents.TheCount IS NULL THEN 0When TestingStudents.TheCount > 0 THEN TestingStudents.TheCountEND
FROM (select distinct users_pk1from course_userswhere role = 'S'and enrollment_date > @enrollment_startand enrollment_date < @enrollment_end) AllStudents
left join(select cu.users_pk1, COUNT(gg.pk1) as TheCountfrom course_users culeft join gradebook_gradegg on gg.course_users_pk1 = cu.pk1left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1where gm.qti_asi_data_pk1 > 0and gg.first_attempt_pk1 > 0and cu.enrollment_date > @enrollment_startand cu.enrollment_date < @enrollment_endgroup by cu.users_pk1) TestingStudents
on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBingroup by NumberOfTests
declare @start intdeclare @end intdeclare @enrollments int
set @start = 0set @end = (Select top 1 NumberOfTests from #BinTests order by NumberOfTests desc)
create table #Results(Number_Of_Tests int,Number_Of_Enrollments int)
While @start-1 < @endBegin
set @enrollments = (select NumberOfEnrollments from #BinTests where NumberOfTests = @start)
insert into #Results(Number_of_Tests, Number_Of_Enrollments)values(@start, @enrollments)
set @start = @start + 1
end
select Number_Of_Tests, Number_Of_Enrollments = CaseWhen Number_Of_Enrollments IS NULL THEN 0When Number_Of_Enrollments >= 0 THEN Number_Of_EnrollmentsEnd
from #Results
drop table #BinTestsdrop table #Results
Strategic Functional DecisionsHow many tests have been completed per enrollment?
set serveroutput on size 100000
declare
v_enrollment_start date;
v_enrollment_end date;
sql_stmt varchar(8000);
v_start int;
v_end int;
v_enrollments int;
Begin
dbms_output.enable(9999999);
dbms_output.new_line();
v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
--sql_stmt :=
EXECUTE IMMEDIATE ' drop table BinTests';
sql_stmt :=''
| | 'create table '
| | 'BinTests as '
| | 'select NumberOfTests , COUNT(NumberOfTests ) as NumberOfEnroll ments '
| | 'from ( '
| | ' select AllStudents.users_pk1, nvl(TestingStudents.TheCount,0) NumberOfTests'
| | ' FROM ( '
| | ' select distinct users_pk1 '
| | ' from course_users '
| | ' where role = ''S'' '
| | ' and enrollment_date > ' | | 'to_date('''| | v_enrollment_start | | ''')'
| | ' and enrollment_date < ' | | 'to_date('''| | v_enrollment_end | | ''')' | | ' ) AllStudents '
| | ' left join( '
| | ' select cu.users_pk1, COUNT(gg.pk1) as TheCount '
| | ' from course_users cu '
| | ' left join gradebook_grade gg on gg.course_users_pk1 = cu.pk1 '
| | ' left join gradebook_main gm on gg.gradebook_main_pk1 = gm.pk1 '
| | ' where gm.qti_asi_data_pk1 > 0 '
| | ' and gg.first_attempt_pk1 > 0 '
| | ' and cu.enrollment_date > ' | | 'to_date('''| | v_enrollment_start | | ''')'
| | ' and cu.enrollment_date < ' | | 'to_date('''| | v_enrollment_end | | ''')'
| | ' group by cu.users_pk1) TestingStudents '
| | ' on AllStudents.users_pk1 = TestingStudents.users_pk1) TestBin '
| | 'group by NumberOfTests' ;
dbms_output.put_line (sql_stmt );
EXECUTE IMMEDIATE sql_stmt ;
EXCEPTION
WHEN OTHERS THEN NULL;
End;
/
set serveroutput on size 100000
declare
v_enrollment_start date;
v_enrollment_end date;
sql_stmt varchar(8000);
v_start int;
v_end int;
v_enrollments int;
Begin
dbms_output.enable(9999999);
dbms_output.new_line();
v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
v_start := 0;
Select NumberOfTests into v_end from (select NumberOfTests from BinTests order by NumberOfTests desc) WHERE ROWNUM =1;
EXECUTE IMMEDIATE ' drop table Results';
EXECUTE IMMEDIATE ' create table Results ( Number_Of_Tests int, Number_Of_Enroll ments int)';
EXCEPTION
WHEN OTHERS THEN NULL;
END;
/
set serveroutput on size 100000
declare
v_enrollment_start date;
v_enrollment_end date;
sql_stmt varchar(8000);
v_start int;
v_end int;
v_enrollments int;v_enrollments int;
Begin
dbms_output.enable(9999999);
dbms_output.new_line();
v_enrollment_start := to_date('2010-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
v_enrollment_end := to_date('2011-9-1 00:00:00', 'yyyy- mm-dd HH24:MI:SS');
While v_start-1 < v_end
loop
select NumberOfEnroll ments into v_enrollments from BinTests where NumberOfTests = v_start;
insert into Results
(Number_of_Tests, Number_Of_Enroll ments)
values
( v_start, v_enrollments );
v_start := v_start + 1;
end loop;
dbms_output.new_line();
FOR rec IN (
select Number_Of_Tests, nvl(Number_Of_Enroll ments,0) Number_Of_Enroll ments from Results)
LOOP
dbms_output.put_line (rec.Number_Of_Tests | | ',' | | rec.Number_Of_Enrollments);
END LOOP;
dbms_output.new_line();
EXCEPTION
WHEN OTHERS THEN NULL;
--drop table BinTests
--drop table Results
END;
Strategic Functional DecisionsHow many test questions were submitted at our peak hour?
SELECT count(pk1) TheCount,cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' +cast(DATEPART(MONTH, bbmd_date_added) as varchar(2))
from qti_result_datagroup by DATEPART(YEAR, bbmd_date_added),
DATEPART(MONTH, bbmd_date_added)order by DATEPART(YEAR, bbmd_date_added),
DATEPART(MONTH, bbmd_date_added)
Strategic Functional DecisionsHow many test questions were submitted at our peak hour?
SELECT count(pk1) TheCount,TO_char(bbmd_date_added,'YYYY') || '-' || TO_char(bbmd_date_added,'MM')
from qti_result_datagroup by TO_char(bbmd_date_added,'YYYY'),
TO_char(bbmd_date_added,'MM')order by TO_char(bbmd_date_added,'YYYY'),
TO_char(bbmd_date_added,'MM');
Strategic Functional DecisionsHow many test questions were submitted at our peak hour?
SELECT top 25 count(pk1) TheCount,cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' +cast(DATEPART(MONTH, bbmd_date_added) as varchar(2)) + '-' +
cast(DATEPART(DAY, bbmd_date_added) as varchar(2)) + ' ' +cast(DATEPART(HOUR, bbmd_date_added) as varchar(2)) + ':00'
from qti_result_datagroup by DATEPART(YEAR, bbmd_date_added),
DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added), DATEPART(HOUR, bbmd_date_added)
order by count(pk1) desc
Strategic Functional DecisionsHow many test questions were submitted at our peak hour?
select * from (SELECT count(pk1) TheCount,
TO_char(bbmd_date_added,'YYYY') || '-' || TO_char(bbmd_date_added,'MM') || '-' || TO_char(bbmd_date_added,'DD') || '-' || TO_char(bbmd_date_added,'HH')from qti_result_datagroup by TO_char(bbmd_date_added,'YYYY'),
TO_char(bbmd_date_added,'MM'),TO_char(bbmd_date_added,'DD') ,TO_char(bbmd_date_added,'HH')order by count(pk1) desc)where rownum <26;
Strategic Functional DecisionsTest question submission by hour of the day?
declare @start_date datetimedeclare @end_date datetime
set @start_date = '2010-9-1 00:00:00:000'set @end_date = '2011-9-1 00:00:00:000'
SELECT count(pk1) TheCount,cast(DATEPART(hour, bbmd_date_added) as varchar(4)) TheHour
from qti_result_datawhere bbmd_date_added > @start_dateand bbmd_date_added < @end_dategroup by DATEPART(HOUR, bbmd_date_added)order by DATEPART(HOUR, bbmd_date_added)order by DATEPART(HOUR, bbmd_date_added)
Strategic Functional DecisionsTest question submission by hour of the day?
set serveroutput on size 100000declare v_start_date date;
v_end_date date;
Begin
v_start_date := to_date('2010-9-1 00:00:00', 'yyyy-mm-dd HH24:MI:SS');v_end_date := to_date('2011-9-1 00:00:00', 'yyyy-mm-dd HH24:MI:SS');dbms_output.enable(9999999);dbms_output.new_line();
FOR rec IN (FOR rec IN (SELECT count(pk1) TheCount,
TO_char(bbmd_date_added,'HH24') TheHourfrom qti_result_datawhere bbmd_date_added > v_start_dateand bbmd_date_added < v_end_dategroup by TO_char(bbmd_date_added,'HH24')order by TO_char(bbmd_date_added,'HH24'))
LOOPdbms_output.put_line (to_char(rec.TheCount));
END LOOP;dbms_output.new_line();
end;
Strategic Functional DecisionsTest question Submission by day of the week?
declare @start_date datetimedeclare @end_date datetime
set @start_date = '2010-9-1 00:00:00:000'set @end_date = '2011-9-1 00:00:00:000'
SELECT count(pk1) TheCount, TheDay = Casewhen cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 1 then
'Sunday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 2 then
'Monday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 3 then
'Tuesday''Tuesday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 4 then
'Wednesday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 5 then
'Thursday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 6 then
'Friday'when cast(DATEPART(DW, bbmd_date_added) as varchar(4)) = 7 then
'Saturday'end
from qti_result_datawhere bbmd_date_added > @start_dateand bbmd_date_added < @end_dategroup by DATEPART(DW, bbmd_date_added)order by DATEPART(DW, bbmd_date_added) asc
Strategic Functional DecisionsTest question Submission by day of the week?
--v_start_date := '2010-9-1 00:00:00:000'--v_end_date := '2011-9-1 00:00:00:000'
SELECT count(pk1) TheCount, to_char(to_date (bbmd_date_added, 'yyyy-mm-dd HH24:MI:SS'),'DAY') from qti_result_datawhere bbmd_date_added > to_date('2010-9-1', 'yyyy-mm-dd')and bbmd_date_added < to_date('2011-9-1', 'yyyy-mm-dd')group by to_char(to_date (bbmd_date_added , 'yyyy-mm-dd HH24:MI:SS'),'DAY') order by to_char(to_date (bbmd_date_added , 'yyyy-mm-dd HH24:MI:SS'),'DAY') asc
Strategic Functional DecisionsDetermine CPU and I/O cost of testing (SQL server)
SELECT p.name AS[SP_Name],ps.total_worker_time AS[Total_Worker_Time],ps.total_physical_reads AS[Total_Physical_Reads], ps.total_logical_reads,ps.total_logical_writes AS[Total_Logical_Writes],(ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS[TOTAL_IO],ps.execution_count,ISNULL(ps.execution_count/DATEDIFF(Second,ps.cached_time,GETDATE()),0)AS[Calls/Second],ps.total_elapsed_time,ps.last_worker_time,ps.last_worker_time,ps.total_elapsed_time/ps.execution_count AS[avg_elapsed_time],ps.last_elapsed_time,ps.last_execution_time,ps.cached_time, qs.text QueryFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()--and p.name = 'qti_result_data_cr' -- Enter the name of the stored procedureORDER BY ps.last_worker_time DESC;
Strategic Functional DecisionsDetermine CPU and I/O cost of testing (SQL server)
Typically, you would view the AWR data through Oracle Enterprise Manager screens or AWR reports. However, you can view the statistics with the following views:
V$ACTIVE_SESSION_HISTORY
This view displays active database session activity, sampled once every second. See "Active Session History (ASH)".
V$ metric views provide metric data to track the performance of the system
The metric views are organized into various groups, such as event, event class, system, session, service, file, and tablespace metrics. These groups are identified in the V$METRICGROUP view.
DBA_HIST views
The DBA_HIST views contain historical data stored in the database. This group of views includes:
DBA_HIST_ACTIVE_SESS_HISTORY displays the history of the contents of the in-memory active session history for recent system activity.DBA_HIST_BASELINE displays information about the baselines captured on the systemDBA_HIST_DATABASE_INSTANCE displays information about the database environmentDBA_HIST_SNAPSHOT displays information on snapshots in the systemDBA_HIST_SQL_PLAN displays the SQL execution plansDBA_HIST_WR_CONTROL displays the settings for controlling AWR
Strategic Functional DecisionsDetermine CPU and I/O cost of testing (SQL server)
SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) Query,qs.execution_count,qs.total_logical_reads, qs.last_logical_reads,qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO],qs.total_worker_time 'Total_CPU_Time',qs.last_worker_time 'Last_CPU_Time',qs.total_elapsed_time/1000000 total_elapsed_time_in_S,qs.total_elapsed_time/1000000 total_elapsed_time_in_S,qs.last_elapsed_time/1000000 last_elapsed_time_in_S,qs.last_execution_time,qs.creation_time,qp.query_planFROM sys.dm_exec_query_stats qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpWHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) LIKE '%qti%' -- Insert the query you are interested in findingORDER BY qs.last_execution_time DESC
Strategic Functional DecisionsDetermine CPU and I/O cost of testing (SQL server)
Typically, you would view the AWR data through Oracle Enterprise Manager screens or AWR reports. However, you can view the statistics with the following views:
V$ACTIVE_SESSION_HISTORY
This view displays active database session activity, sampled once every second. See "Active Session History (ASH)".
V$ metric views provide metric data to track the performance of the system
The metric views are organized into various groups, such as event, event class, system, session, service, file, and tablespace metrics. These groups are identified in the V$METRICGROUP view.tablespace metrics. These groups are identified in the V$METRICGROUP view.
DBA_HIST views
The DBA_HIST views contain historical data stored in the database. This group of views includes:
DBA_HIST_ACTIVE_SESS_HISTORY displays the history of the contents of the in-memory active session history for recent system activity.DBA_HIST_BASELINE displays information about the baselines captured on the systemDBA_HIST_DATABASE_INSTANCE displays information about the database environmentDBA_HIST_SNAPSHOT displays information on snapshots in the systemDBA_HIST_SQL_PLAN displays the SQL execution plansDBA_HIST_WR_CONTROL displays the settings for controlling AWR
Strategic Functional DecisionsDetermine I/O cost of question submission (SQL server)
SELECT p.name AS[SP_Name],(ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS AVG_IOFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()and p.name = 'qti_result_data_cr'ORDER BY (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count DESC;
Strategic Functional DecisionsDetermine I/O cost of question submission (SQL server)
please look at AWR reportsear qti_result_data_cr
Strategic Functional DecisionsDetermine IO cost of all cached stored procedures
SELECT p.name AS[SP_Name],(ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count AS AVG_IOFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()and ps.database_id =DB_ID()ORDER BY (ps.total_logical_reads +ps.total_logical_writes) /ps.execution_count DESC;
Strategic Functional DecisionsDetermine IO cost of all cached stored procedures
--look at AWR reportset pagesize 60;column executions format 999,999,999;column Mem_used format 999,999,999;
SELECT SUBSTR(owner,1,10) Owner,SUBSTR(type,1,12) Type,SUBSTR(name,1,20) Name,SUBSTR(name,1,20) Name,executions,sharable_mem Mem_used,SUBSTR(kept||' ',1,4) "Kept?"
FROM v$db_object_cacheWHERE type in ('PROCEDURE','PACKAGE BODY','PACKAGE')ORDER BY executions desc;
HERE IS THE OUTPUT OF MEMORY.SQL:SQL> @memory
Strategic Functional DecisionsDetermine total IO cost of all cached stored procedures
SELECT p.name AS[SP_Name],(ps.total_logical_reads +ps.total_logical_writes) AS TOTAL_IOFROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()ORDER BY (ps.total_logical_reads +ps.total_logical_writes) DESC;
Strategic Functional DecisionsDetermine total IO cost of all cached stored procedures
AWR report
Strategic Functional DecisionsQuery that compares the test taking activity cost vs. to other activity
select 'QTI_IO' as TYPE, SUM(TOTAL_LOGICAL_READS + TOTAL_LOGICAL_WRITES) IO
FROM (SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,
((CASE qs.statement_end_offset
WHEN -1 THEN DATALENGTH(qt.TEXT)
ELSE qs.statement_end_offset
END - qs.statement_start_offset)/2)+1) Query,
qs.execution_count,
qs.total_logical_reads, qs.last_logical_reads,
qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO],
qs.total_worker_time 'Total_CPU_Time',
qs.last_worker_time 'Last_CPU_Time',
qs.total_elapsed_time/1000000 total_elapsed_time_in_S,
qs.last_elapsed_time/1000000 last_elapsed_time_in_S,
qs.last_execution_time,
qs.creation_time,
qp.query_plan
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt
CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp
WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)
LIKE '%qti%' -- Insert the query you are interested in finding
and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)
NOT LIKE '%sys.dm%'
--and qs.last_execution_time + .005 > GETDATE()
)QTI_IO)QTI_IO
UNION
select 'NON_QTI_IO' as TYPE, SUM(TOTAL_LOGICAL_READS + TOTAL_LOGICAL_WRITES) IO
FROM (SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,
((CASE qs.statement_end_offset
WHEN -1 THEN DATALENGTH(qt.TEXT)
ELSE qs.statement_end_offset
END - qs.statement_start_offset)/2)+1) Query,
qs.execution_count,
qs.total_logical_reads, qs.last_logical_reads,
qs.total_logical_writes, qs.last_logical_writes, (qs.total_logical_reads +qs.total_logical_writes) /qs.execution_count AS[AVG_IO],
qs.total_worker_time 'Total_CPU_Time',
qs.last_worker_time 'Last_CPU_Time',
qs.total_elapsed_time/1000000 total_elapsed_time_in_S,
qs.last_elapsed_time/1000000 last_elapsed_time_in_S,
qs.last_execution_time,
qs.creation_time,
qp.query_plan
FROM sys.dm_exec_query_stats qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qt
CROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qp
WHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)
NOT LIKE '%qti%' -- Insert the query you are interested in finding
and SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1, ((CASE qs.statement_end_offset WHEN -1 THEN DATALENGTH(qt.TEXT) ELSE qs.statement_end_offset END - qs.statement_start_offset)/2)+1)
NOT LIKE '%sys.dm%'
--and qs.last_execution_time + .005 > GETDATE()
)NON_QTI_IO
Strategic Functional DecisionsQuery that compares the test taking activity cost vs. to other activity
AWR reportand v$sql, sqlarea
on qti_result_data_cr
Strategic Functional DecisionsQuery that joins the QTI tables (bbmd_data_added) with the cost of the
queries associated with the insert
declare @start_date datetimedeclare @end_date datetimedeclare @question_io int
set @start_date = '2010-9-1 00:00:00:000'set @end_date = '2011-9-1 00:00:00:000'set @question_io = (SELECT ps.last_logical_reads + ps.last_logical_writes Current_IO
FROM sys.procedures AS p, sys.dm_exec_procedure_stats AS pscross apply sys.dm_exec_sql_text (ps.sql_handle) AS qsWHERE p.[object_id] =ps.[object_id]and ps.database_id =DB_ID()and p.name = 'qti_result_data_cr')and p.name = 'qti_result_data_cr')
SELECT count(pk1) * @question_io Test_Question_IO, cast(DATEPART(YEAR, bbmd_date_added) as varchar(4)) + '-' +cast(DATEPART(MONTH, bbmd_date_added) as
varchar(2)) + '-' +cast(DATEPART(DAY, bbmd_date_added) as
varchar(2)) TheDatefrom qti_result_datawhere bbmd_date_added > @start_dateand bbmd_date_added < @end_dategroup by DATEPART(YEAR, bbmd_date_added),
DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added)
order by DATEPART(YEAR, bbmd_date_added),DATEPART(MONTH, bbmd_date_added), DATEPART(DAY, bbmd_date_added)
Strategic Functional DecisionsQuery that joins the QTI tables (bbmd_data_added) with the cost of the
queries associated with the insert
AWR reportand v$sql, sqlarea
on qti_result_data_cr
Strategic Functional DecisionsCreate a logging script
declare @ts_now bigintdeclare @start intdeclare @end intdeclare @Start_time datetimedeclare @End_time datetimedeclare @Bb_Activity intdeclare @QTI_RESULTS_Activity int
set @ts_now = (SELECT cpu_ticks/(cpu_ticks/ms_ticks)FROM sys.dm_os_sys_info);
SELECT TOP(256) SQLProcessUtilization AS [SQL_Server_Process_CPU_Utilization],SystemIdle AS [System_Idle_Process],100 - SystemIdle - SQLProcessUtilization AS [Other_Process_CPU_Utilization],DATEADD(ms, -1 * (@ts_now - [timestamp]), GETDATE()) AS [start_Time],DATEADD(minute, 1, DATEADD(ms, -1 * (@ts_now - [timestamp]), GETDATE())) AS [end_Time],IDENTITY( int ) AS TheCount
into #cpuFROM (SELECT record.value('(./Record/@id)[1]', 'int') AS record_id,
record.value('(./Record/SchedulerMonitorEvent/SystemHealth/SystemIdle)[1]', 'int') AS [SystemIdle],record.value('(./Record/SchedulerMonitorEvent/SystemHealth/ProcessUtilization)[1]', 'int') AS [SQLProcessUtilization], [timestamp]FROM (SELECT [timestamp], CONVERT(xml, record) AS [record]
FROM sys.dm_os_ring_buffersWHERE ring_buffer_type = N'RING_BUFFER_SCHEDULER_MONITOR'AND record LIKE N'%<SystemHealth>%') AS x) AS y
ORDER BY record_id DESC
set @start = 0set @end = (select MAX(TheCount) from #cpu)
CREATE TABLE [dbo].[#cpu_Activity](CREATE TABLE [dbo].[#cpu_Activity]([Activity_Accumulator_Activity] [int] NULL,[QTI_RESULTS_Activity] [int] NULL,[SQL_Server_Process_CPU_Utilization] [int] NULL,[System_Idle_Process] [int] NULL,[Other_Process_CPU_Utilization] [int] NULL,[start_Time] [datetime] NULL,[end_Time] [datetime] NULL,[TheCount] [int] NULL)
WHILE @start < @endBEGIN
SET @start = @start + 1
SET @Start_time = (select Start_Time from #cpu where TheCount = @start)SET @End_time = (select End_Time from #cpu where TheCount = @start)
set @Bb_Activity = (select COUNT(*) from activity_accumulator where timestamp > @Start_time and timestamp < @End_time)set @QTI_RESULTS_Activity = (select COUNT(*) from qti_result_data where bbmd_date_modified > @Start_time and bbmd_date_modified < @End_time)
Insert into #cpu_ActivitySELECT @Bb_Activity, @QTI_RESULTS_Activity, *from #cpuwhere thecount = @start
end
select *from #cpu_activity
drop table #cpu;drop table #cpu_activity;
Strategic Functional DecisionsCreate a logging script
AWR report, v$sqlselect ... rom activity_accmultor where event_type ='login';
Strategic Functional DecisionsCreate a logging script
CREATE TABLE [dbo].[#FILE_IO](
[name] [sysname] NULL,
[num_of_bytes_read] [bigint] NOT NULL,
[num_of_bytes_written] [bigint] NOT NULL,
[num_of_reads] [bigint] NOT NULL,
[num_of_writes] [bigint] NOT NULL,
[size_on_disk_bytes] [bigint] NOT NULL
) ON [PRIMARY]
declare @start int
declare @end int
set @start = 5
set @end = (select top 1 database_id
from sys.master_files
where database_id > 5
order by database_id desc)
while @start < @end
begin
insert into #FILE_IO
select name, num_of_bytes_read, num_of_bytes_written, num_of_reads, num_of_writes, size_on_disk_bytes
from sys.dm_io_virtual_file_stats(@start,NULL) s
left join sys.master_files f on s.database_id = f.database_id and s.file_id = f.file_id
set @start = @start + 1
end
select *
from #FILE_IO
drop table #FILE_IO
Strategic Functional DecisionsCreate a logging script
AWR report, v$sqlselect ... rom activity_accmultor where event_type ='login';
Strategic Functional DecisionsCreate a logging script
SELECT 'TEST QUESTION' Type, SUM(qs.execution_count) Execution_Count,SUM(qs.total_logical_reads) Total_Reads,SUM(qs.total_logical_writes) Total_Writes,SUM(qs.total_worker_time) Total_CPUFROM sys.dm_exec_query_stats qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpWHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) LIKE '%INSERT INTO QTI_RESULT_DATA%' --Enter a portion of the query text you are interested inand SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) NOT LIKE 'SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,%'UNIONSELECT 'OTHER TRANSACTIONS' Type,SUM(qs.execution_count) Execution_Count,SUM(qs.total_logical_reads) Total_Reads,SUM(qs.total_logical_writes) Total_Writes,SUM(qs.total_worker_time) Total_CPUFROM sys.dm_exec_query_stats qsCROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) qtCROSS APPLY sys.dm_exec_query_plan(qs.plan_handle) qpWHERE SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) NOT LIKE '%INSERT INTO QTI_RESULT_DATA%' --Enter a portion of the query text you are interested inand SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,((CASE qs.statement_end_offsetWHEN -1 THEN DATALENGTH(qt.TEXT)ELSE qs.statement_end_offsetEND - qs.statement_start_offset)/2)+1) NOT LIKE 'SELECT SUBSTRING(qt.TEXT, (qs.statement_start_offset/2)+1,%'
Strategic Functional DecisionsCreate a logging script
AWR report, and use v$sql, sqlarea to find INSERT INTO QTI_RESULT_DATA%'