sql tips and best practices

7/29/2019 SQL Tips and Best Practices

1/70

SQL Tips and Best Practices

for Meditechs DR

2011 MUSE International

Tuesday Education SessionMay 31st

Presenter: Jamie Gerardo


2/70

Operational Key Topics

Background Jobs

DR Errors

Auditing/Validation

Development Key Topics

Report Development Standards

Finding Data Writing Efficient Code

T-SQL Tips

Todays Agenda


3/70

Operational

Background jobs DR Errors

Audits / Validation

Server Maintenance Monitor space

Backing up and testing backup restore

Preventative server maintenance

Meditech Website Logging Tasks

Following recommendations andguidelines


4/70

MT DR Background Jobs

Client Server

* Graph from Meditech website

http://www.meditech.com/
http://www.meditech.com/http://www.meditech.com/


5/70

MT DR Background Jobs

Magic



6/70

MT DR Background Jobs 6.0



7/70

Monitoring/Maintaining

Background Jobs

You can create additional bkg jobs(depending on the ring release)

Do not reboot server withoutstopping jobs (completely)

Background job can be runningalthough a table is not updating

Review Meditech Recommendationsand website


8/70

Additional Background Jobs

Why create additional backgroundjobs? (Current job is taking too longto update the DR tables.)

1. You can split up larger applications

into multiple jobs.2. You can segment off particular

tables that are used for real timereporting.

3. You can split off applications thattake longer.

* Additional jobs depends on the ring release version


9/70

Rebooting the DR Server

Do not reboot the server without

stopping the background jobs!

What happens if you do?a. Table sequences will get off track

b. Data will be skippedc. Table ILs may need to be done to get

data over to the DR.

You can stop the background jobs by:

a. Calling Meditech or

b. Stopping the jobs through the DRapplication


10/70

DR Errors

DR Parameters Activity Index

Error Log

Monitor Error Summary It is not recommended to report every

error

Provide as much information as

possible Prioritize DR Errors Logged

See Meditech Website


11/70

DR ParametersActivity Days (Typically defaults to 7-

10 days): Determines how longactivity is kept around in an index to

be transferred to DR.

Error Log Days (Default is 3days): This should be at least 7-

10 days.

C/S 6.5 but all Meditech Versions havethis option in the DR Parameters (it

just looks different)

Can be set at the tablelevel by Meditech for

research/problem issues


12/70

DR Errors

Example to report:

DATE: 20110221 TIME: 1143

TABLE: VISITCLI AdmVisitClinicalQueries

Sequence: 4391

SOURCEID: OSC MIS DB: OSC

PROGRAM: DrXferBkg[T:ADMQUERY]

APPL DB: ADM.OSC DPM: ADM.PAT

TYPE: S Socket

ELEMENT: SQL

EXPECTED:ERROR VALUE:

ROW: 873550 MIO5UD Y 1

MESSAGE: Primary key VisitD is missing

Example not to report:

DATE: 20110414 TIME: 0001

TABLE: LACSAX1 LabSpecAlerts

Sequence:

SOURCEID: OSC MIS DB: OSC

PROGRAM: DrXferBkg[K]

APPL DB: SCH.SSM DPM:

LAB.C.SPEC

TYPE: PM Program missing

ELEMENT:EXPECTED:

ERROR VALUE:

ROW:

MESSAGE: SCH.SSM's LabSpecAlerts

expander is missing

o Only report DR Errors that reference tables you use or will use

in the future.o Only report the latest version of the same or similar error.

o Report Skipped activity as soon as possible, this data will

eventually be removed from the index.

This is typically not a table thatwill ever be used so dont report

the error

Key items when reviewing errors


13/70

DR Error Messages

Typical Messages you can ignore:

MESSAGE: Unable to open prefixes to ADM.OSH database [1]

MESSAGE: Missing subscript at position 0 for OE.STAT.transcription.stats

MESSAGE: Violation of PRIMARY KEY constraint 'mtpk_ep551386'. Cannot insert duplicate key

in object 'DMisUserC~. Check Server Error Log for more info SQL Non Fatal Error

MESSAGE: Line 1: Incorrect syntax near '{'.. Check Server Error Log for more info SQL NonFatal Error

MESSAGE: Unable to find segment from physical [No prefix for segment [Physical.] and nil in @.db]MESSAGE: No pointer for include children of AP.AHS.TmpOpIC table

MESSAGE: Syntax error converting datetime from character string.. Check Server Error Log

for more info SQL Non Fatal Error

MESSAGE: CON DR Server not responding to ECB command - ACK 98 SQL Fatal Error

DR Server not responding to ECB command - ACK 98

Messages you want to report:

MESSAGE: AdmVitalSigns Activity skipped, not in Pending statusMESSAGE: Primary key CheckID is missing

MESSAGE: Missing subscript at position 1 for MRI.DRC.insure.order

MESSAGE: Unknown error [SYS] - Segment A is down, unable to start DrXferBkg[T:ADMQUERY]

Always confirm with your

applications specialist. Theseare some general guidelines.


14/70

Logging DR Errors

Include ring release version in description along with tablename and indication of the issue.

Example Descriptions: 5.61 AdmVisitQueries Primary Key Missing VisitID

5.65 MRI.DRC.insure.order - Missing subscript Pos 1

5.61.1 OeOrderQuery - Expander is missing

Issue: DR #6133704 (Mar 2, 2009)

Status: Open

Priority: Routine

Priority Lists: #3 on DR Priority List

Description: 5.61.13 LabSpecimenTests - Column Discrepancies

Request Type: Problem

Customer Contact: James Durbin (617-555-1212)Issue Notification: [email protected] (All Edits)

Module Notifications: ACME,SMH ([email protected]) - MEDITECH

Edits Set a Priority List on yourtasks to get the quickest

resolution


15/70

Auditing / Validation

Blue Elm DR Auditor or other tool? Audit tables every 1-2 months

Re-Analyze (and within 2 days)

Research audits and provide MT recent

examples (within the activity Index) Application purge parameters

Manual validation

Matching NPR / DR Reports

Meditech Auditing Tool CS 5.65, Magic 5.64.17(?), 6.0

Is in its infancy with development andtesting.


16/70

Blue Elm DR Auditor

More DR Auditor Tips Start with auditing tables you use Audit like tables Only a few tables at a time Rebuild Schemas after a new Meditech ring release Recreate audits after an update

Delete old audits


17/70

DR Auditor A step further

You can query the audits tofind specific examples or themost recent examples. You

can also confirm that the datais still missing.

Meditech tables andcorresponding DRAuditor

table names:


18/70

Reporting discrepancies to Meditech

Meditech would prefer audits that are not older

than a week. Once the data has been purged

from the Activity index it makes it difficult to

troubleshoot this issue.

Typically log 1 table per Task

Task descriptions should include the Table

Name along with Missing Row or Column

Discrepancy

Ex: 5.65 AdmVisits Missing rows

Attach the HTML Report in your task or

provider recent examples from your sql query.


19/70

DR Maintenance

Overall Server Maintenance Monitor disk space

Database Backups

Database Maintenance

DBCCs on a monthly basis

Integrity checks (using wizard)

Analyze and repair index fragmentationon a monthly basis


20/70

Details of Maintenance

Monitor disk space Database space

How much space is the database using

Drive space Always have 25% of total disk space as

free (livedb E: drive)

Index defragmenting

Dont use the database maintenancewizard

Wizard will drop and recreate indexes,which you dont want to do.


21/70

Defragging Indexes

DETERMINING FRAGMENTATION AT THE

TABLE AND INDEX LEVEL Logical ScanFragmentation lower isbetter, anything above

30-35% would be a

candidate to defragment.

Avg. Page Densityhigher is better, ideally

90-95%. The fuller each

page is, the denser (less

fragmented) the data are.

DEFRAG INDEX IF NECESSARY

DBCC INDEXDEFRAG ( DBNAME, TABLENAME, INDEXNAME)


22/70

SQL Development

Finding Data

How can you find data in the DR?

Organization

Database, Stored Procedures and

Tables Efficiency

How to make your code run fast andmore efficiently

T-SQL TIPS

Helpful code tips


23/70

Finding Data

If you know the NPR structure then

finding data will be much easier In general you can think of a Detail

Segment as a table

Tools

1. Meditech Website

2. SysDrTables

3. Shift F9 (C/S) and Shift F8 forMagic

4. DR Application Menu


24/70

Meditech DR Home Page

Overview of the datarepository

* Table Structure

CustomerResponsibilities


25/70

Viewing the Table Structure

http://www.meditech.com/

Useful Tableinformation
http://www.meditech.com/http://www.meditech.com/


26/70

Meditech - Table Structure

1. Shows the equivalent NPR Parent/Child relationships

2. Interactive Primary Keys that displays other tables with foreign keys


27/70

SysDr Tables in livedb---------------------------------------------------------------------------

-- A general search by DR Field Name

-- '%Comment%' is a wildcard search for any field with Comment-- You can modify the name as needed for your search

---------------------------------------------------------------------------

SELECT T.Name, C.*

FROM livedb.dbo.SysDrColumns C

INNER JOIN livedb.dbo.SysDrTables T

ON C.TableID = T.TableID

WHERE C.Name like '%Comment%'

order by 1

-------------------------------------------

-- A search by specific NPR field

-------------------------------------------

SELECT T.Name, C.*

FROM livedb.dbo.SysDrColumns CINNER JOIN livedb.dbo.SysDrTables T


WHERE C.NprElement = 'BAR.PAT.account'

order by 1

-------------------------------------------------------

-- A search by Dpm and Detail Segment

-------------------------------------------------------

SELECT T.Name, C.*



WHERE C.NprDpm = 'BAR.PAT'

AND C.NprSegment = 'main'

order by 1

Examples of how toquery Sys DR tables


28/70

SysDr Result Example

SELECT T.Name, C.*



WHERE C.Name like '%Comment%'

order by 1

Shows table name, column, data type

along with the DPM, NprSegment and NprElement


29/70

Shift F9 (C/S) and Shift F8 for Magic


30/70

Table and Field Inquiry on the DR

Application MenuI rarely use these

tools on the DR Menu They can be helpful

but I find thepreviously covered

options to be the best.

Magic 5.61 Screen


31/70

Organization - Database

1. Dont save stored procedures

and/or tables in livedb.2. Create a database to keep

your stored procedures, views

and tables.

Make sure Recovery

Mode is set to simple

You can set up thedatabase files similar to

livedb

3. You will want to include the

database you create in your

backup plan.


32/70

What is a stored procedure?

A stored procedure is a saved set of

code on the sql server that allows youto run:

EXEC spBarAccountsByAccountType

Rather than

SELECT

BVFD.AccountType,

BV.PostalCode,

BV.AccountNumber

FROM livedb.dbo.BarVisits BV

INNER JOIN livedb.dbo.BarVisitFinancialData BVFD

ON BV.SourceID = BVFD.SourceID

AND BV.BillingID = BVFD.BillingID


33/70

Organize your Stored Procedures

Name your stored procedures so that you

can easily locate them. Ex: spErDepartVolumesByHour

Ex: spAdmRevisitsTable

Re-name stored procs no longer in use.

Ex: x_spErDepartVolumesByHour

Use Header information to describe astored procedures use.

Only save useable code as a storedprocedure.

Save test code and research code as a textfile or label appropriately.


34/70

Stored Procedures

To save a stored procedure

you CREATE PROC. Thissaves the stored procedureon the server/databaseyouve selected

Once created youcan Modify and

View Dependencies


35/70

Stored Procedure Example Header


36/70

Saving code as a text file

Items to save as a text file1. Research queries

2. Testing queries DefaultLocation this can bechanged


37/70

Creating Efficiencies

Indexing

Estimated Execution Plan

Joining on Primary Keys Creating Datamarts (tables)

Using temp tables vs. sub queries

Functions Using With (NoLock)


38/70

Table Indexing

Clustered Every Meditech table has a clustered index, which

is the physical order of the table by primarykey(s). Never modify or delete

There is only 1 per table

Non-Clustered A non-clustered index creates a separate

'internal' table that stores only the selected keyvalues of the table in order. Each 'record' in thisindex contains the key value from one record in

the table along with a pointer to either the datarecord itself or to a value in the clustered index.

A max of 249 non-clustered indexes per table You will want to be selective with indexes youbuild


39/70

Typical Indexes used

BarChargeTransactions ixServiceDateTime

ixTransactionProcedureID

ixProcedureChargeDept

BarVisits ixVisitID

ixAdmitDateTime

ixServiceDateTime

AdmVisits ixServiceDateTime

ixStatus

BarCollectionTransactions ixReceiptDateTime

ixInsuranceID

AdmittingData ixAdmitDateTime

AbstractData ixVisitID

DMisUserStatisticsDetail ixAccountNumber (Field4)

ixUnitNumber (Field3)


40/70

Example of Execution Plan

Display Estimated Execution Plan


41/70

Creating an Index


42/70

Joining on Primary Keys

Joining on the primary keys willmake your report run faster.

Omitting the primary key will not

only slow down your query butmany times your output will not becorrect.

Each application has a unique

identifier (primary key) that willallow you to join to otherapplications.


43/70

Primary Key Example

SELECT AV.VisitID,

AV.LocationID,

AV.[Name],

AD.AdmitDateTime

FROM livedb.dbo.AdmVisits AV WITH (NOLOCK)

INNER JOIN livedb.dbo.AdmittingData AD WITH (NOLOCK)

ON AV.SourceID = AD.SourceID

AND AV.VisitID = AD.VisitID

WHERE Status='ADM IN

Even if there is only one

SourceID, you will want to usethe Clustered Index for faster

processing.


44/70

SQL Design Query Editor

You can use this tool but you still need tojoin on the primary keys. This tool will not

automatically do that for you.


45/70

Primary Key Example 2

SELECT AV.VisitID,

AV.LocationID,

AV.[Name],

AD.AdmitDateTime,

BV.PrimaryInsuranceID,

BVFD.Balance

FROM livedb.dbo.AdmVisits AV

INNER JOIN livedb.dbo.AdmittingData AD

ON AV.SourceID = AD.SourceID

AND AV.VisitID = AD.VisitID

LEFT JOIN livedb.dbo.BarVisits BV

ON AV.SourceID = BV.SourceID

AND AV.VisitID = BV.VisitID

LEFT JOIN livedb.dbo.BarVisitFinancialData BVFD

ON BV.SourceID = BVFD.SourceID

AND BV.BillingID = BVFD.BillingID

where Status='ADM IN'

VisitID is in a number of tablesbut youll only want to use it tojoin to a parent type table -BarVisits, AbstractData, Lab

Specimens, SchAppointments

Use the applications primarykey (unique identifier) within

the application tables.

Adm - VisitID Bar BillingID Abs AbstractID Oe OrderID Sch - AppointmentID

C ti D t t / T bl


46/70

Creating Datamarts / Tables

When do you create a new table? Processing time for a report may

dictate the need for a table.

Combining data from variousapplications for faster queries.

Combining data and creatingcustom fields for reporting needs.

Building Report Models


47/70

Datamart Example

Issue:

You have a very large query for nursing with lots ofdata from various assessments and with pharmacydata. This query takes over 10 minutes to run and willtake longer if additional filters are added.

Solution:

1. Create a table with all the data you need for yourreports

2. Set up a SQL Server Agent Job to populate the

data on a nightly basis3. Create a set of store procedures to use for your

report output.


48/70

Datamart Example - Continued

NursingStoredProcedure

This table gets updated daily. ASQL Server Job is set up to

populate on a nightly basis with

new data.

Additional Stored procedures

are written to use for reportingpurposesNursingLocationSummary
http://localhost/var/www/apps/conversion/tmp/scratch_10/Muse2011/DataMartNursingExample.rtfhttp://localhost/var/www/apps/conversion/tmp/scratch_10/Muse2011/NurLocationSummary.rtfhttp://localhost/var/www/apps/conversion/tmp/scratch_10/Muse2011/NurLocationSummary.rtfhttp://localhost/var/www/apps/conversion/tmp/scratch_10/Muse2011/DataMartNursingExample.rtf


49/70

Creating Tables

You can write sql code to insert data into a table by first creatingthe table then inserting the data.

1. CREATE TABLE PatientBirthDates (

Name varchar(35) not null,

AccountNumber varchar(15) not null,

DOB datetime null)

2. INSERT PatientBirthDates

SELECT Name, AccountNumber, BirthDateTime

FROM AdmVisits

You can insert directly into a table as below. The data types andfields will populate based on the data returned.

Select * INTO tbNewTable from DMisUsers

Either way of creating the table is fine. Just note that you may need alter datatypes and write code to add Primary keys. Typically modifications will need tobe made when the table is created by an INTO. Youll also need to TruncateTable where necessary. ** Will demonstrate

After the first table insert typically you will:

TRUNCATE TABLE PatientBirthDatesINSERT PatientBirthDates

The INTO only works the first time.


50/70

User Defined Function

What is a User Defined Function?Functions are subroutines used to encapsulate frequentlyperformed logic. Any code that must perform the logic

incorporated in a function can call the function rather than

having to repeat all of the function logic.

Built-in functions operate as defined in the Transact-SQL Reference andcannot be modified. The functions can be referenced only in Transact-SQL

statements using the syntax defined in the Transact-SQL Reference. Examples AVG, SUM, COUNT,DATEADD, DATEDIFF,NAME, ETC..

User-defined functions allow you to define your own Transact-SQL

functions using the CREATE FUNCTION statement. For more information

about these built-in functions This is what well be doing in class.


51/70

FUNCTIONS

Useful Acmeware functions

fxAge

fxProperCase

fxConvertGramsToLbs

fxMeditechTimeIDToDateTime

fxIsNumeric

F ti f A


52/70

Function - fx.Age


53/70

Function - fxProperCase

SELECT

[Name],

dbo.fxProperCase(Name) AS ProperName,

ProviderGroupName,dbo.fxProperCase(ProviderGroupName)AS

ProperGroupName

FROM livedb.dbo.DMisProvider

This takes any value

and converts it toupper and lower

case. Works great forcreating

consistencies in yourreports.

Selecting the data:


54/70

Function - fxConvertGramsToLbs

Code could possiblyneed to be tweakedbased on the Grams

data type.

F ti


55/70

Function - fxMeditechTimeIDToDateTime

There arevarious fieldsthroughout

Meditech that

are in seconds.This function will

calculate thedate for you.


56/70

Function - fxIsNumeric

There will be times where you

need to ensure that a fieldstrictly has numeric values.Using the System IsNumeric

does not always work.


57/70

SQL Tips

Dates

Dropping Temp Tables

Documenting in code

Creating a #Dates andpopulating

Using WITH (NOLOCK)


58/70

TSQL Tips - Dates

DateDiff Calculates the difference between two

dates

DateAdd

Adds a period of time to a date (orsubtracts)

Years, Months, Days, Hours, Minutes orSeconds

SQL Date Default5/26/11 defaults to 5/26/11 00:00:00


59/70

DateDiff DateDiff calculations havecountless uses in your code

development.

* Be aware of the time valuesand rounding specifically withDays

D t Diff C l l ti f El d Ti


60/70

DateDiff Calculations for Elapsed Time

Be aware of using standardsql functions, they may notwork the way you expect.

D t Add C l l ti


61/70

DateAdd CalculationsFirst Day of Current Month:

SELECT DATEADD(MM, DATEDIFF(MM,0,GETDATE()), 0)

Explanation:1. 0 = 19000101

2. The DATEDIFF calculates the number of months since 19000101

3. The DATEADD adds the same number of months back to 19000101 to give you the

beginning of the current month

First Day of Last Month:SELECT DATEADD(MM, DATEDIFF(MM,0,DATEADD(MM,-1,GETDATE())),0)

Explanation:

1. DATEADD(MM,-1,GETDATE()) - Subtracts 1 month from current date

2. DATEDIFF(MM,0,DATEADD(MM,-1,GETDATE())) - calculates the number of months since

19000101

3. The DATEADD adds the calculated number of months back to 19000101 to give you the

beginning of the previous month

Last Day of Current Month:SELECT DATEADD(SS,-1,DATEADD(MM,DATEDIFF(MM,0,GETDATE())+1,0))

Explanation:

1. DATEDIFF(MM,0,GETDATE())+1 - calculates the number of months from the current date since

19000101 and adds 1

2. DATEADD(MM,DATEDIFF(MM,0,GETDATE())+1,0) - adds the above number of months to

19000101 (this will give you the first day of next month)

3. The last DATEADD substracts 1 second to give you the last day of the current month

(ie. 9/30/09 23:59:59 )

D t Add C l l ti


62/70

DateAdd Calculations

Last Day of Last Month:

SELECT DATEADD(SS,-1,DATEADD(MM,DATEDIFF(MM,0,GETDATE()),0))

Explanation:1. DATEADD(MM,DATEDIFF(MM,0,GETDATE()),0) - same code as getting the first day of

the current month

2. DATEADD substracts 1 second to give you the last day of previous month

Last Day of Last Year:

SELECT DATEADD(SS,-1,DATEADD(YY,DATEDIFF(YY,0,GETDATE()),0))Explanation:

1. 0 = 19000101

2. The DATEDIFF calculates the number of years since 19000101

3. The DATEADD adds the same number of years back to 19000101 to give you the

beginning of the current year

4. The next DATEADD substracts 1 second to reflect the day before just before midnight.

First Day of Current Year:SELECT DATEADD(YY,DATEDIFF(YY,0,GETDATE()),0)

Explanation:1. 0 = 19000101

2. The DATEDIFF calculates the number of years since 19000101

3. The DATEADD adds the same number of years back to 19000101 to give you the beginning of the

current year

4. This is the same as the month calculations but instead of mm for month you use the yy for year


63/70

Examples using DateDiff & DateAdd

Understanding how the datafunctions work will help youright the appropriate codefor your particular needs.


64/70

SQL Date Defaults

Because SQL defaults to atime of 00:00:00. We codefor that with a DateAdd.

Keep this in mind when creating data range parameters sothat you include the full last day of the search

C d f D i T T bl


65/70

Code for Dropping Temp Tables

When using temp tables enter this before eachtemp table and it will save you a lot of time and

hassle with continuously dropping the table.

IF OBJECT_ID('tempdb.dbo.#TableName')IS NOT NULL

DROP TABLE #TableName

SELECT

Fields

INTO #TableName

FROM MyTables

You can also enter the codeat the end of your stored

procedure or query to makesure the temp table has been

dropped.


66/70

Document in your code!!

Documenting through out your code isa huge help with understandinglaterjust what you were thinking atthe time.

Document on any bit of code that is theleast bit out of the ordinary. Not onlywhat by why.

Notate in each step of your code whatyou are doing.


67/70

Documenting Example:


68/70

Creating a #Dates Temp Table

The #Dates table works

great for Summaries thatneed to include every dayregardless of data values.

U i WITH (NOLOCK)


69/70

Using WITH (NOLOCK)When data in a database is

read or modified, the database

engine uses special types ofcontrols, called locks, to

maintain integrity in the

database. Locks basically work

by making sure database

records involved in a

transaction cannot be modified

by other transactions until the

first transaction has committed,

ensuring database consistency.

The benefit of using WITH

(NOLOCK) is that it allows you

to keep the database engine

from issuing locks against the

tables in your queries; thisincreases concurrency and

performance because the

database engine does not have

to maintain the shared locks

involved.


70/70

Thank You!

6/3 10:00 Replacement Session - 6.0

Custom Reports with Data Repository and

SSRS 2008, Ian Proffer presenting

6/3 11:00 - 372 Basic Meaningful UseReporting From Data Repository, Glen

DAbate presenting

6/3 1:30 - 334 Inspiring Reporting

Options in the World of 6.0, Jamie Gerardo

presenting

sql tips and best practices

Documents