www.dba4aday.com

Troubleshooting 11i issues

Adam Janbolatwww.dba4aday.com

adam@dba4aday.com

www.dba4aday.com

DBA JOKE

• No one really knows what the Database Administrator does, and no one is smart enough to know if the DBA is doing it or not. But every shop must have one DBA, because no place can afford two of them.

www.dba4aday.com

Different issues with each user

• A user needs a trace file to upload to support• A user is waiting on a payroll process that should not take

more than 10 min but it has been running for three hours• A user heard someone talk about locking issues and now

they have locking issues!• A user is getting error ORA-1652: unable to extend temp

segment when running his/her code from sqlplus• Your manager says the application is running slow; can

you improve performance• A developer wants you to tune her code because it is a dba

thing to do

www.dba4aday.com

Overview

• Tracing application forms

• Tracing database processes

• Tracing concurrent requests

• Locking issues

• Temp space issues

www.dba4aday.com

Diagnostic Tools

• Enterprise Manager

• unix/sql scripts

www.dba4aday.com

Tracing Forms

• Verify diagnostics is turned on in the Help menu.

• You can turn it on for any user by setting this profile option to No:

Hide Diagnostics menu entry

MIS:

DEMO

MIS:

DEMO

www.dba4aday.com

Turn diagnostics on for user BROWN

www.dba4aday.com

Set the profile option to No

www.dba4aday.com

Tracing forms

www.dba4aday.com

Tracing forms

www.dba4aday.com

Tracing forms

• The form will produce a trace file in the udump directory.

• The trace file will be ora_173xxx.trc

• Use tkprof on the trace file: tkprof ora_173.trc ora_173.out explain=apps/password sort=prsela

• Which users have trace enabled:

MIS:

DEMO

MIS:

DEMO

www.dba4aday.com

Who has diagnostics privs

Run this query to find who has diagnostics privs:

select a.user_name

from fnd_user a,

fnd_profile_option_values b

where a.user_id=b.level_value and

b.profile_option_id=3214 and

b.profile_option_value='N' and

b.level_id= 10004

www.dba4aday.com

Tracing database processes

• Identify the process

• Trace the process

• Trace file will end up in the udump directory

• Use tkprof on the trace file

www.dba4aday.com

Tracing concurrent request

• Find out the request_id

• Find out if the process has child processes

• Use this query to find which process it is: select oracle_process_id from fnd_concurrent_requests where request_id=941206

• Oracle_process_id corresponds to the spid in OEM.

www.dba4aday.com

www.dba4aday.com

Debugging performance issues

• Use top to find which session is consuming lots of cpu time.

• Identify the process

www.dba4aday.com

Process 2034132

www.dba4aday.com

PID translates to SPID in TOAD and OEM; find out what the process is doing

MIS:

DEMO

MIS:

DEMO

www.dba4aday.com

Tune the code

• Run an explain plan

• Try passing a hint

• Try creating an index

• Consider partitioning

• Verify there are no blocking locks

www.dba4aday.com

Locks created by open forms

• How do I know which forms are open and who is using them: Code is attached

MIS:

DEMO

MIS:

DEMO

www.dba4aday.com

Monitor the health of the db

• Check the alert log daily (automate)• Check for space issues on unix and db• Verify there are no trace files generated• Monitor for processes that consume a lot of temp

space• Monitor sga utilitzation• Check application component logs on regular

basis• Use statspack to monitor performance

www.dba4aday.com

Monitor for processes that consume a lot of temp space

• ORA-1652: unable to extend temp segment by xx in tablespace TEMP

• Look for the number of free_extents:

select * from v$sort_segment

• Monitor what process is using lots of temp space. Here is a way to do that:

MIS:

DEMO

MIS:

DEMO

www.dba4aday.com

Temp space scripts

• Create a table in the tools tablespace:Sql> create table mon_temp_tab (COL_SID, VARCHAR2(10)COL_MODULE VARCHAR2(200),COL_NAME VARCHAR2(40),COL_EXTENTS NUMBER,COL_BLOCKS NUMBER,TIME_STMP DATE,GROUP_ID NUMBER)

• Create a sequence: create sequence mon_temp_seq start with 1;

• Create a procedure that does the monitoring:

www.dba4aday.com

• create or replace procedure mon_temp_space • as• v_free_extents number;• v_max_free_extents number;• v_seq number;• cursor c1 is• select s.sid "SID",• s.module "MODULE",• s.username "USERNAME", • u.extents "EXTENTS" , • u.blocks "BLOCKS"• from v$session s , v$sort_usage u• where s.saddr=u.session_addr• and u.contents='TEMPORARY'• and s.status='ACTIVE'• order by extents desc;•  • begin• select mon_temp_seq.nextval into v_seq from dual;

www.dba4aday.com

•  

• select free_extents into v_free_extents from v$sort_segment;

• if v_free_extents <=1000

• then

• for i in c1

• loop

• insert into mon_temp_tab (col_sid, col_module,col_name,col_extents,col_blocks,time_stmp,group_id)

• values (i.sid,i.module, i.username,i.extents,i.blocks, sysdate,v_seq);

• commit;

• end loop;

• else

• null;

• end if;

•  

• end;

• /

www.dba4aday.com

Create a dbms_job to run every hour

declare

 job_num integer;

begin

dbms_job.submit(job_num,'mon_temp_space;',

sysdate + 1/24,

'sysdate + 1/24');

 end;

www.dba4aday.com

Monitor the mon_temp_tab

www.dba4aday.com

Monitor the health of your applications

• Verify that your are purging concurrent request logs often

• Verify that your are analyzing your schemas often• Verify you are purging workflow runtime data

often• Monitor apache logs for errors and warnings • Verify that debugging is disabled except for

specific modules you are monitoring:

www.dba4aday.com

Use this query to see which executables have debugging enabled

• select * from fnd_concurrent_programs where enable_trace=‘Y’

• Turn off tracing

www.dba4aday.com

Enable_trace_example

www.dba4aday.com

Questions?