summary of pod study results - cnde iastate · floyd w. spencer distinguished member of technical...
TRANSCRIPT
1
Summary of POD Study Results
Floyd W. Spencer
Distinguished Member of Technical StaffSandia National Laboratories
ASNT 16th Annual Research SymposiumOrlando, FloridaMarch 26-30, 2007
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
2
AIRLINE MAINTENANCE FACILITY EXPERIMENT
Realism of Inspection (specimens & location)
Accessibility
Arrangeable layouts
3
2nd set of Specimens
4
POD summary by Facility examples
5
Facility E - Probability of De tection Curves(184 flawed sites in 782 de tection opportunities)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 50 100 150 200
crack size (mils)
E1 (132,2)E2 (126,3)E2R (125,6)E3 (135,2)E4 (133,53)
6
Facility G - Probability of Detection Curves(184 flawed sites in 782 de tection opportunities)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.9
1
0 20 40 60 80 100 120 140 160 180 200
crack size (mils)
0.8
G1 (91,5)G1R (94,4)G2 (126,6)G3 (89,10)G4 (124,59)
7
Facility Differences•Facility differences significant
– gauged by inspection-to-inspection variances
•Inspection-to-inspection variation significant– gauged by repeatability within
inspector
8
Proportion of Detects - All Inspections
flaw size (mils)
prop
ortio
n of
det
ectio
ns
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 20 40 60 80 100 120 140 160 180 200 220 240 260 280 300 320 340 360 380 400
Individual flaws
no thresh fit
C=.023 fit
2 points at (812, .98)
9
( ))ln()1()( acaPOD ⋅+Φ⋅−= βα
∑ ∑∑ ⋅⋅+⋅+i
jiijii III ααα
∑ ∑∑ ⋅⋅+⋅+i
jiijii III βββ
Model extension Probit with fitted asymptote
Generalized parameters to include factor effects under study
10
crack angle(Iang)
surface condition(Isurf)
lap splice position(Ipos)
number of cracks at rivet (Inum)
procedure followed(Iproc)
density of cracked rivet sites (Idens)
level 0
11 to 22 degs.
painted low 2 template low (~10%)
level 1
horizontal bare high 1 sliding high (~40%)
Factors modeled in the POD
11
Values are calculated as averages across other factors. All values are in thousandths of an inch.
levelangle surface
/facility
pos flaw # procedure/facility
density
50th percentile 0 72 73 69 63 71 66
50th percentile 1 63 62 66 72 64 69
90th percentile 0 109 115 105 95 109 104
90th percentile 1 96 90 100 110 96 101
Factor Effects in mils
12
Conclusions from Experiment•Individual inspector differences are a major factor affecting inspections
•Environmental factors can influence an inspection
•Inspection misses result that are independent of crack length
13
Automated Ultrasonic Inspection Result Summary
General Goals were to establish•Capability•Assurance of adequate inspection transfer to customer
14
Example of image to be interpreted
15
Lab phase 12 356 16 18 7 9 4 4
Field phase 8 238 15 16 6 7 4 3
Test
secti
ons
Insp
ectio
n Si
tes
< 0.
050
inch
0.05
to 0
.10
inch
0.10
to 0
.15
inch
0.15
to 0
.20
inch
> 0
.20
inch
not m
easu
red
Flaw distributions
16
Experimental Variables and levels for Laboratory experiment
VariableLow level (-1) High level (1) Nominal level (0)
1. time base delay nominal - 0.005 nominal + 0.005 determined in calibration
used 0.35-ch 1 0.355-ch 2
0.36-ch 1 0.365-ch 2 0.355-ch 1 0.36-ch 2
2. depth velocity table value for probe angle - 1 °
table value for probe angle + 1 °
tabled value determined from probe angle
used 85400 in/sec 88500 in/sec 87000 in/sec
3. receiver gain nominal - 0.6 dB nominal + 0.6 dB as determined at time of calibration
used 35.6 dB 36.8 dB 36.2 dB
4. scanner skew 0.25 inch left 0.25 inch right centered
5. probe pressure pressure off nominal arbitrary
used 0 - 1 lbs. indicated 16 lbs indicated 16 lbs indicated on dial
17
Laboratory Results
•Variations in 2 different signal aspects (area and amplitude) quantified with respect to expected variation allowed by procedures
18
Table 3. Effects of Experimental factors on selected signals – as % of nominal
Experimental Factors
Response time delay depth velocity
gain skew probe pressure
time delay * depth vel.
Inner Transducer
Average signal strength
in interaction 7.8 1.2 2.9
Area of flaw signal 11.2 17.0
Average signal strength of flaw
areain interaction 7.1 15.4
Outer Transducer
Average signal strength 4.4 9.1 2.9 6.3
Area of flaw signal in interaction 19.8
Average signal strength of flaw
area4.7 9.9 4.6
19
Field Inspection Results•Gain and time base delays observed in the field data exceeded the range studied in the laboratory phase
• Misidentification of hole from the image not uncommon–Able to be determined from saving
inspectors images and comparing to calls made
20
Average PoDs
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.05 0.1 0.15 0.2crack length (inch)
prob
abili
ty
4par
4par align
2par
2par align
21
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
LJLJLJLJLJUIUIUIUIUIUITITITI
PoD Curves fit to individual inspections
22
Common Image Set
•Nominal run of Laboratory phase•10 inspectors made calls
– on 356 images
23
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.90
1.00
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14crack length (inch)
prob
abili
tyRange of POD fits to calls on common images
0.80
24
Conclusions•Target flaw size (0.050 inch) estimated to have average detection rate of 0.70
•Set-up variation in the field higher than expected
•Decision process a large contributor to observed variation–Lead to development of automated
decision process (neural net)
25
References for full reports• Spencer, F. and Schurman D., “Reliability
Assessment at Airline Inspection Facilities, Volume III: Results of an Eddy Current Inspection Reliability Experiment,” DOT/FAA/CT-92/12,III, May 1995.
• Mullis, R. and MacInnis, T., “C-141 Spanwise Splice Advanced NDI Method,” The First Joint DoD/FAA/NASA Conference on Aging Aircraft" in Ogden, Utah, July 10, 1997. (NDTnet – November 1997, Vol.2 No. 11)
• Andrew, G., MacInnis, T., and Mullis, T. “Second-layer ultrasonic inspection of C-141 splice joints,”Nondestructive Evaluation of Aging Aircraft, Airports, and Aerospace Hardware, eds Rempt, R and Broz, A, SPIE Proceedings Vol. 2954, 1996.