troubleshooting · • undertakes initial troubleshooting before ringing roche • otherwise it is...

Troubleshooting 101

Troubleshooting: A logical approach:

• Important that your lab:

• Records all problems in real time

• Records actions taken

• Records Instrument alarms

• Performs all maintenance as required

• Undertakes initial troubleshooting before ringing Roche

• Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures before we can go any further thus slowing down the process.

Troubleshooting: A logical approach:

• Can you establish if the problem is most likely Instrument related or Reagent/Calibrator/QC etc related?

• Always look at the most likely causers first

• Calibrations are not a problem solving tool: you calibrate after you have solved the problem

Troubleshooting: A logical approach:• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results?

• What else?

Troubleshooting: A logical approach:

• What have you done so far?

• Your own troubleshooting:

• Checked maintenance is done and up to date (including consumables expiries).

• Checked instrument diary, service reports etc for previous similar events.

• Checked for any “outside” factors (eg water quality/flow, UPS status etc).

• Performed relevant checks (eg ISE Check).

• Recorded everything you have done?

• Still a problem?... who you gonna call????

• Make sure you have ready:

• Analyser name and serial number

• Assay Lot Number(s) (if applicable)

• Calibrator Lot number(s) (if applicable)

• All applicable data (eg Error Codes, Printouts of Checks, Calibrations etc)

Roche Techline

Techline (24 hours)

outside NZ +64 9 276 8155

Cobas IT Helpline

0800 COBAS IT

Techline email (Mon- Fri)(for non-urgent matters)

nz.rdnztechline@roche.com

Customer services for product supply (Mon-Fri)

0800 65 26 34

Techline (24 hours)

0800 36 36 22

When you call, what information do we require?

Please be prepared to provide the following information (if we are on another call, you will encounter a recorded message asking for the same information):

Name

Laboratory

Analyser model and serial number

Contact phone number

A brief but clear description of the problem (including error code if applicable)

To ensure you get a prompt response please contact Techline, as Product Specialists and Service Engineers may be out of the office

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Elton from Yellow Brick Rd Medical Centre here; “my urates are running high…”

Example

Example

• No patient results released

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results?

• C8000 (c702)

• No• No

• Affecting both

• All of a sudden, this morning after start-up, QC out of range

• Just the daily maintenance

• Don’t think so

• Not sure

• Running as expected

• Running as expected

• Running as expected• Ummm ... Hang on … TRIG and Cholesterol also high

• Overnight, engineers performed maintenance on water system (decontamination and cleaning)

Example

• Use H2O2 to decontaminate…..

• UA, Cholesterol, Triglyceride are all examples of Trinder assays; H2O2 is part of the reaction:

• Not all H2O2 was flushed out after the decontamination process

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Brucie from Fast Eddy’s Lab Services here; I can’t calibrate my folates”

Problem 1

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Drucilla here from Wacko Lab; We have a problem with CRP … PCCC2 is perfect but PCCC1 is too high”

Problem 2

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Eugene here from Wallamalloo Lab; We have a problem with Phosphate and Urea test not running…”

Problem 3

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Juliet here from Shakespeare Community Lab; We are getting ISE Noise alarms”

Problem 4

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Bruno here from My Lab; “My Enzymes QC is running at + 2SD…”

Problem 5

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Doris from Black Plum Path Lab here; I can’t get B12s to run…”

Problem 6

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Brucie from Fast Eddy’s Lab Services here; I can’t calibrate my folates”

Problem 1

• No patient results released

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results?

• c6000 (e601)

• No• No, but earlier in week had 311-0003 alarm

• Only have one analyser

• Today when we went to calibrate

• Just the daily maintenance

• Don’t think so

• Not sure

• Yes, some OK some not

• Running as expected

• Running as expected• All problem assays are using MC 2

Problem 1

• e601 (c6000)

• Folate, FSH and PSA won’t calibrate

• All other assays calibrate fine

• MC1: Vitamin B12, HIV Combi, Beta-hCG, Cortisol, Ferritin, FT3-III

• MC2: Folate, FSH, PSA, TSH, FT4 II, Prolactin

• Error code 311-0003

Problem 1

Problem 1

Problem 1

• Cause:

One PW Magnet had a cotton swab stuck in it; reaction cups not sitting properly

Problem 1

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Drucilla here from Wacko Lab; We have a problem with CRP … PCC2 is perfect but PCC1 is too high”

Problem 2

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • Haven’t done any patient checks

• c6000 (c501)

• No• No

• Other analyser OK

• Last Friday

• Just the daily maintenance

• Don’t think so

• Not sure

• No

• No

• No• Just CRP and just this analyser

Problem 2

• c501

• Since last Friday

• New calibrator, before that we were perfect.

• Did not update Prozone limits as per CN-SWA-2019-018

• All other controls are perfect (chemistry and immunology)

• PCCC1 lot 250254, PCCC2 lot 191125

• Usual scientist is on leave, I (Section Leader), currently doing her job.

• Reagent was taken off one line and put on the other (was fine on the other)!

Problem 2

Problem 2

Problem 2

Problem 2

Problem 2

Problem 2Problem 2

• Cause:

Reagent not being mixed when loaded

Problem 2

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Eugene here from Wallamalloo Lab; We have a problem with Phosphate and Urea test not running…”

Problem 3

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • No QC or patient results

• c8000 (c702)

• No• No (no Data Manager Alarms either)

• Other analyser OK

• Just noticed

• Just the daily maintenance; did deactivate urine QC for MAlb

• No

• No

• No

• No

• No• Ur PO4 & Urea (Se PO4 & Urea OK, Ur/Se Crea & Mg OK as well)

Problem 3

• Cobas 8000 (c702)

• URELU and PHO2U not running but CRE2U and MGU-2 are working

• Serum Urea and Phosphate working OK

• Patients and QC both not running

• Yes, I checked and saline is on the system,

• No QC Violations that might explain masking

• Same QC running with other urine assays (CRE2U, MGU-2) on the same analyser

Problem 3

Problem 3

• Cause:

URELU and PHO2U on reagent carousel A

CRE2U and MGU-2 on reagent carousel B

Saline was onboard but only on reagent carousel B

Serum Urea, PO4, Crea, Magnesium don’t require saline (unless running on decrease)

Problem 3

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Juliet here from Shakespeare Community Lab; We are getting ISE Noise alarms”

Problem 4

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • Seen in patient results but always repeat without error

• c6000

• ISE.N• No

• Only 1 analyser

• Probably getting worse over the past few months

• No

• No

• No

• No

• No

• No• Only ISEs and only randomly

Problem 4

• Cobas 6000

• Intermittent issue with ISE Noise alarms.

• It seems to happen once or twice in a day every few days. Possibly coming out of Rack Supply Complete

• The samples always repeat ok and we don’t have any QC issues

• The electrodes are only just 4 months old.

• Green rack is done each afternoon

• Have you done an ISE Check (n=30)

• No bubbles in syringes

• Pinch valve tubing OK

• ISE Drain not crystallised

Problem 4

Problem 4

Problem 4

• O Ring missing from Perspex block

Problem 4

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Bruno here from My Lab; “My Enzymes QC is running at + 2SD…”

Problem 5

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • Can’t tell if patients as well but EQA also high

• c311

• No• No

• Only 1 analyser

• Probably getting worse over the past few months

• No

• No

• No

• No

• Yes

• No• Enzyme assays

Problem 5

• ALTL Lot: 409761 exp: 08/20

• ASTL Lot: 411761 exp: 08/20

• AMY-P Lot: 415043 exp: 02/20

• CFas Lot 305596 exp 02/2020

• PCCC1 Lot 250228 exp 06/2020

• PCCC2 Lot 25025405 exp 10/2020

• Cfas and PCCCs made up and aliquots are frozen

Problem 5

Problem 5

Problem 5

• New Cfas lot 305596 now in use but never uploaded from Cobas Link (still using values for lot 250133)

Problem 5

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Doris from Black Plum Path Lab here; I can’t get B12s to run…”

Problem 6

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • No results available

• c6000

• No• No

• Only 1 analyser

• No, all of a sudden after loading reagents

• After 2 Weekly LFC

• No

• Don’t think so

• Just B12, others running OK

• Nos

• No• No

Problem 6

• Just loaded a new kit; new lot number lot 123456

• How can I tell if it is calibrated?

• How can I tell if it is masked?

Problem 6

Problem 6

Problem 6

Problem 6

• Cause:

No Pre-treatment reagent loaded

Problem 6

Doing now what patients need next