troubleshooting · • undertakes initial troubleshooting before ringing roche • otherwise it is...

55
Troubleshooting 101

Upload: others

Post on 23-Mar-2020

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Troubleshooting 101

Page 2: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Troubleshooting: A logical approach:

• Important that your lab:

• Records all problems in real time

• Records actions taken

• Records Instrument alarms

• Performs all maintenance as required

• Undertakes initial troubleshooting before ringing Roche

• Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures before we can go any further thus slowing down the process.

Page 3: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Troubleshooting: A logical approach:

• Can you establish if the problem is most likely Instrument related or Reagent/Calibrator/QC etc related?

• Always look at the most likely causers first

• Calibrations are not a problem solving tool: you calibrate after you have solved the problem

Page 4: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Troubleshooting: A logical approach:• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results?

• What else?

Page 5: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Troubleshooting: A logical approach:

• What have you done so far?

• Your own troubleshooting:

• Checked maintenance is done and up to date (including consumables expiries).

• Checked instrument diary, service reports etc for previous similar events.

• Checked for any “outside” factors (eg water quality/flow, UPS status etc).

• Performed relevant checks (eg ISE Check).

• Recorded everything you have done?

• Still a problem?... who you gonna call????

• Make sure you have ready:

• Analyser name and serial number

• Assay Lot Number(s) (if applicable)

• Calibrator Lot number(s) (if applicable)

• All applicable data (eg Error Codes, Printouts of Checks, Calibrations etc)

Page 6: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Roche Techline

Techline (24 hours)

outside NZ +64 9 276 8155

Cobas IT Helpline

0800 COBAS IT

Techline email (Mon- Fri)(for non-urgent matters)

[email protected]

Customer services for product supply (Mon-Fri)

0800 65 26 34

Techline (24 hours)

0800 36 36 22

When you call, what information do we require?

Please be prepared to provide the following information (if we are on another call, you will encounter a recorded message asking for the same information):

Name

Laboratory

Analyser model and serial number

Contact phone number

A brief but clear description of the problem (including error code if applicable)

To ensure you get a prompt response please contact Techline, as Product Specialists and Service Engineers may be out of the office

Page 7: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Elton from Yellow Brick Rd Medical Centre here; “my urates are running high…”

Example

Page 8: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Example

• No patient results released

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results?

• C8000 (c702)

• No• No

• Affecting both

• All of a sudden, this morning after start-up, QC out of range

• Just the daily maintenance

• Don’t think so

• Not sure

• Running as expected

• Running as expected

• Running as expected• Ummm ... Hang on … TRIG and Cholesterol also high

Page 9: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Overnight, engineers performed maintenance on water system (decontamination and cleaning)

Example

• Use H2O2 to decontaminate…..

• UA, Cholesterol, Triglyceride are all examples of Trinder assays; H2O2 is part of the reaction:

• Not all H2O2 was flushed out after the decontamination process

Page 10: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Brucie from Fast Eddy’s Lab Services here; I can’t calibrate my folates”

Problem 1

Page 11: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Drucilla here from Wacko Lab; We have a problem with CRP … PCCC2 is perfect but PCCC1 is too high”

Problem 2

Page 12: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Eugene here from Wallamalloo Lab; We have a problem with Phosphate and Urea test not running…”

Problem 3

Page 13: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Juliet here from Shakespeare Community Lab; We are getting ISE Noise alarms”

Problem 4

Page 14: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Bruno here from My Lab; “My Enzymes QC is running at + 2SD…”

Problem 5

Page 15: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Doris from Black Plum Path Lab here; I can’t get B12s to run…”

Problem 6

Page 16: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Brucie from Fast Eddy’s Lab Services here; I can’t calibrate my folates”

Problem 1

Page 17: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• No patient results released

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results?

• c6000 (e601)

• No• No, but earlier in week had 311-0003 alarm

• Only have one analyser

• Today when we went to calibrate

• Just the daily maintenance

• Don’t think so

• Not sure

• Yes, some OK some not

• Running as expected

• Running as expected• All problem assays are using MC 2

Problem 1

Page 18: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• e601 (c6000)

• Folate, FSH and PSA won’t calibrate

• All other assays calibrate fine

• MC1: Vitamin B12, HIV Combi, Beta-hCG, Cortisol, Ferritin, FT3-III

• MC2: Folate, FSH, PSA, TSH, FT4 II, Prolactin

• Error code 311-0003

Problem 1

Page 19: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 1

Page 20: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 1

Page 21: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Cause:

One PW Magnet had a cotton swab stuck in it; reaction cups not sitting properly

Problem 1

Page 22: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Drucilla here from Wacko Lab; We have a problem with CRP … PCC2 is perfect but PCC1 is too high”

Problem 2

Page 23: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • Haven’t done any patient checks

• c6000 (c501)

• No• No

• Other analyser OK

• Last Friday

• Just the daily maintenance

• Don’t think so

• Not sure

• No

• No

• No• Just CRP and just this analyser

Problem 2

Page 24: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• c501

• Since last Friday

• New calibrator, before that we were perfect.

• Did not update Prozone limits as per CN-SWA-2019-018

• All other controls are perfect (chemistry and immunology)

• PCCC1 lot 250254, PCCC2 lot 191125

• Usual scientist is on leave, I (Section Leader), currently doing her job.

• Reagent was taken off one line and put on the other (was fine on the other)!

Problem 2

Page 25: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 2

Page 26: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 2

Page 27: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 2

Page 28: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 2

Page 29: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 2Problem 2

Page 30: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Cause:

Reagent not being mixed when loaded

Problem 2

Page 31: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Eugene here from Wallamalloo Lab; We have a problem with Phosphate and Urea test not running…”

Problem 3

Page 32: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • No QC or patient results

• c8000 (c702)

• No• No (no Data Manager Alarms either)

• Other analyser OK

• Just noticed

• Just the daily maintenance; did deactivate urine QC for MAlb

• No

• No

• No

• No

• No• Ur PO4 & Urea (Se PO4 & Urea OK, Ur/Se Crea & Mg OK as well)

Problem 3

Page 33: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Cobas 8000 (c702)

• URELU and PHO2U not running but CRE2U and MGU-2 are working

• Serum Urea and Phosphate working OK

• Patients and QC both not running

• Yes, I checked and saline is on the system,

• No QC Violations that might explain masking

• Same QC running with other urine assays (CRE2U, MGU-2) on the same analyser

Problem 3

Page 34: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 3

Page 35: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Cause:

URELU and PHO2U on reagent carousel A

CRE2U and MGU-2 on reagent carousel B

Saline was onboard but only on reagent carousel B

Serum Urea, PO4, Crea, Magnesium don’t require saline (unless running on decrease)

Problem 3

Page 36: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Juliet here from Shakespeare Community Lab; We are getting ISE Noise alarms”

Problem 4

Page 37: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • Seen in patient results but always repeat without error

• c6000

• ISE.N• No

• Only 1 analyser

• Probably getting worse over the past few months

• No

• No

• No

• No

• No

• No• Only ISEs and only randomly

Problem 4

Page 38: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Cobas 6000

• Intermittent issue with ISE Noise alarms.

• It seems to happen once or twice in a day every few days. Possibly coming out of Rack Supply Complete

• The samples always repeat ok and we don’t have any QC issues

• The electrodes are only just 4 months old.

• Green rack is done each afternoon

• Have you done an ISE Check (n=30)

• No bubbles in syringes

• Pinch valve tubing OK

• ISE Drain not crystallised

Problem 4

Page 39: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 4

Page 40: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 4

Page 41: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• O Ring missing from Perspex block

Problem 4

Page 42: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Bruno here from My Lab; “My Enzymes QC is running at + 2SD…”

Problem 5

Page 43: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • Can’t tell if patients as well but EQA also high

• c311

• No• No

• Only 1 analyser

• Probably getting worse over the past few months

• No

• No

• No

• No

• Yes

• No• Enzyme assays

Problem 5

Page 44: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• ALTL Lot: 409761 exp: 08/20

• ASTL Lot: 411761 exp: 08/20

• AMY-P Lot: 415043 exp: 02/20

• CFas Lot 305596 exp 02/2020

• PCCC1 Lot 250228 exp 06/2020

• PCCC2 Lot 25025405 exp 10/2020

• Cfas and PCCCs made up and aliquots are frozen

Problem 5

Page 45: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 5

Page 46: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 5

Page 47: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• New Cfas lot 305596 now in use but never uploaded from Cobas Link (still using values for lot 250133)

Problem 5

Page 48: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Ring, ring…..

• “Hello Techline, how can I help you?”

• “Hi, Doris from Black Plum Path Lab here; I can’t get B12s to run…”

Problem 6

Page 49: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• What instrument?

• Are there any alarms generated?

• Data alarms?

• Instrument alarms? (record the codes)

• If you have more than one analyser, is it affecting both analysers or just one?

• “Yes”: likely to be something common (eg water, reagents)

• When did the problem first occur

• Gradual deterioration or “Happened just like that”?

• Was anything else happening at about the same time?

• (eg) Maintenance?

• (eg) Water cartridges just been changed?

• (eg) Shift engineer testing power circuits?

• Are all assays affected?

• (eg) Just immunoassays?

• (eg) Just enzymes?

• (eg) Just drug assays?

• (eg) Just a particular type of assay reaction?

• Patients and QC results? • No results available

• c6000

• No• No

• Only 1 analyser

• No, all of a sudden after loading reagents

• After 2 Weekly LFC

• No

• Don’t think so

• Just B12, others running OK

• Nos

• No• No

Problem 6

Page 50: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Just loaded a new kit; new lot number lot 123456

• How can I tell if it is calibrated?

• How can I tell if it is masked?

Problem 6

Page 51: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 6

Page 52: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 6

Page 53: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Problem 6

Page 54: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

• Cause:

No Pre-treatment reagent loaded

Problem 6

Page 55: Troubleshooting · • Undertakes initial troubleshooting before ringing Roche • Otherwise it is likely we will be asking you all these questions or getting you to do certain procedures

Doing now what patients need next