Automated Chip QC

Michael Elashoff

Chip QC

• Transition from mostly manual/visual chip QC to mostly automated chip QC

• Database of passing and failing chips to serve as the training set (5K passing, 2K failing)

Chip QC: Defect Classes• In order of occurrence:

– Dimness– High Background– Unevenness– Spots– Haze Band– Scratches– Brightness– Crop Circle– Cracked– Snow– Grid Misalignment

• Training set of 7K chips (Human, Rat, Mouse)

0 500 1000 1500 2000Intensity

Cou

nt

0 500 1000 1500 2000Expression

Cou

ntDimness/Brightness

Passing Chips

Bright/Dim Chips

A chip Low Scan

Dimness/Brightness

Passing Chips

Bright/Dim Chips

0 500 1000 1500 2000Intensity

Cou

nt

0 500 1000 1500 2000Expression

Cou

ntA chip Low Scan

Dimness/Brightness• Each chip type has a different typical

brightness range• Typical brightness range depends on

scanner setting– tuned-up versus tuned-down– scanners must be calibrated to achieve consistency

Spots, Scratches, etc.

Spots, Scratches, etc.

Implementation of Li-Wong• With training set of 5K passing chips, apply

Li-Wong algorithm

• For each probe set, algorithm yields:– “outlier” status for each probe-pair– probe weights for non-outlier probe-pairs

),0(~

...1,...12

N

NjNi

MMPM

ij

pairsprobechips

ijjiijij

j

Implementation of Li-Wong

• For QC, new chips are screened individually• For each probe set:

– Ignore “model outlier” probes– Using training ‘s, compute– Compute residuals for each probe pair– Flag residuals that are large

j

Implementation of Li-Wong• Compare distributions of outlier count for

passing and failing chips in training set• Determine upper bound of acceptable outlier

count:

Hgu95a 3100Hgu95b 4200Hgu95c 4100Hgu95d 4700Hgu95e 4300Rgu34a 3300Rgu34b 3000Rgu34c 3100

Grid Alignment

Grid Alignment

0

40

80

120

160

Outlier Count

Limitations of Li-Wong

• Must estimate 1.8 million probe weights for human/rat chip sets

• Works poorly for rare genes• Probe weights may vary

– Tissue Type– RNA Processing– Chip Lot– Training Set

Haze Band

Vertical 10th Percentile Profile

0

100

200

300

400

500

6001 15 29 43 57 71 85 99 113

127

141

155

169

183

197

211

225

239

253

Bin Number

Inte

nsi

tyHaze Band

Crop Circles

Crop Circles

Horizontal 95th Percentile Profile

0

5000

10000

15000

20000

25000

30000

35000

400001 16 31 46 61 76 91 106

121

136

151

166

181

196

211

226

241

256

Bin Number

Inte

nsi

ty

Using Spike-Ins

10

100

1000

10000

0.1 1 10 100

Concentration

Ex

pre

ss

ion

10636 10638 10639 10640 10641 10643 10644 1064510646 10648 10649 10650 10651 10652 10653 1065410655 10656 10657 10658 10659 10660 10661 1066210663 10664 10665 10666 10667 10668 10669 1067010671 10673 10674 10675 10677 10678 10679 10680

Spike-in R2 must be >96.5%

QC Metrics• Mean of Non-control Oligo Intensity• Mean OligoB2 Intensity• Spike-in R2

• Li-Wong Outlier Count• Several measures of LiWong Outlier

“clustering”• Vertical profiles• Horizontal profiles• Thresholds differ for each chip type

QC Metrics

QC Metrics: Performance

AutoQCPass

AutoQCFail

Total

ManualPass

449 131 580

ManualFail

2 149 151

Total 451 280 731

Two week validation run

False Negative Rate = 0.4%

These will not be manually QC’d anymore

False Positive Rate = 46.8%

These are still manually QC’d

Conclusions

• Automated QC has: – reduced the number of chips in visual QC– made the process more objective

• Automated QC has not:– eliminated the need for visual QC– incorporated the impact on real world data

quality/analysis

Thanks

• Peter Lauren• Chris Alvares• John Klein• Michelle Nation• Jeff Wiser