marshall 2006a obj vs subj for selenia pmb

Upload: adis-radoncic

Post on 14-Apr-2018

220 views

Category:

Documents


0 download

TRANSCRIPT

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    1/23

    INSTITUTE OF PHYSICS PUBLISHING PHYSICS IN MEDICINE AND BIOLOGY

    Phys. Med. Biol. 51 (2006) 24412463 doi:10.1088/0031-9155/51/10/006

    A comparison between objective and subjective imagequality measurements for a full field digitalmammography system

    N W Marshall

    Clinical Physics Group, Barts and the London NHS Trust, St Bartholomews Hospital,London EC1A 7BE, UK

    Received 6 September 2005, in final form 30 March 2006

    Published 27 April 2006

    Online at stacks.iop.org/PMB/51/2441

    Abstract

    This paper presents pre-sampling modulation transfer function (MTF),

    normalized noise power spectrum (NNPS) and detective quantum efficiency

    (DQE) results for an amorphous selenium (a-Se) full field digital

    mammography system. MTF was calculated from the image of an angled

    0.5 mm thick Cu edge, acquired without additional beam filtration. NNPS

    data were acquired at detector air-kerma levels ranging from 9.1 Gy to

    331 Gy, using a standard mammography x-ray spectrum of 28 kV, Mo/Mo

    target/filter combination and 4 cm of PMMA additional filtration. Prior to

    NNPS estimation, the image statistics were assessed using a variance image.

    This method was able to easily identify a detector artefact and should proveuseful in routine quality assurance (QA) measurements. Detector DQE,

    calculated from the NNPS and MTF data, dropped to 0.3 for low detector

    air-kerma settings but reached an approximately constant value of 0.6 above

    50 Gy at the detector. Subjective image quality data were also obtained at

    these detector air-kerma settings using the CDMAM contrast-detail (c-d) test

    object. The c-d data reflected the trend seen in DQE, with threshold contrast

    increasing at low detector air-kerma values. The c-d data were then compared

    against predictions made using two established models, the Rose model and a

    standard signal detection theory model. Using DQE(0), the Rose model gave

    results within approximately 15% on average for all the detector air-kerma

    values studied and for detail diameters down to 0.2 mm. Similar agreement

    was also found between the measured c-d data and the signal detection theory

    results, which were calculated using an ideal human visual response function

    and a system magnification of unity. The use of full spatial frequency DQE

    improved the agreement between the calculated and observer results for detail

    sizes below 0.13 mm.

    0031-9155/06/102441+23$30.00 2006 IOP Publishing Ltd Printed in the UK 2441

    http://dx.doi.org/10.1088/0031-9155/51/10/006http://stacks.iop.org/PMB/51/2441http://stacks.iop.org/PMB/51/2441http://dx.doi.org/10.1088/0031-9155/51/10/006
  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    2/23

    2442 N W Marshall

    Introduction

    The past few years have seen the introduction to the UK of full field digital mammography

    (FFDM) systems, initially for the purpose of breast assessment rather than breast screening.

    FFDMtechnology falls into twodistinctcategoriesthe computedradiography (CR)approach

    involving the use of photostimulable phosphor plates held in a cassette and the so-called

    integrated detector methods. Integrated detector systems use an x-ray converter bonded to

    a thin film transistor (TFT) array which performs the image readout. One of the major

    advantages of integrated digital detectors is that the image is available directly on the hospital

    information networkCR systems currently rely on manual handling of the imaging cassette

    by the radiographer between the cassette holder and the CR reader. Hence, integrated digital

    systems should allow greater patient throughput and enable the radiographer to stay with the

    patient in the x-ray room throughout the duration of the examination.

    Assessment of conventional film/screen mammography systems in terms of

    commissioning and quality assurance (QA) tests is covered extensively in IPEM Report 89

    (2005). One of the challenges of the new full field digital technology lies in extending existing

    test protocols to fully assess the capabilities of these new detectors. One such protocol thathas recently been made available is the EUREF Digital Addendum (2003), which gives tests

    suitable for the both the commissioning and routine QA assessment of both CR and direct

    detection FFDM systems. This protocol suggests that objective measurements such as the

    noise powerspectrum (NPS) andmodulation transfer function (MTF) mayalso be measured as

    part of the commissioning and routine QA measurements performed on these units. Objective

    measures are a well-established and powerful means of assessing image quality (Metz et al

    1995, ICRU 1996), especially given the relatively easy access to the required image data now

    offered by theDICOM format. Within the past 15 years or so, these methods have been applied

    to various digital imaging modalities (Hillen et al 1987, Workman and Cowen 1993, Dobbins

    et al 1995, Flynn and Samei 1999, Stierstorfer and Spahn 1999, Williams et al 1999, Evans

    et al 2002, Samei and Flynn 2003). As demonstrated by Samei and Flynn (2003), accurate

    measurement of MTF, NPS and DQE is a complex procedure with many important parameters

    that affect the final result.

    For imaging modalities such as conventional screen/film (S/F) mammography or

    fluoroscopy, it is far harder to apply these objective methods for reasons of image data

    access, where images are held by a film or in the form of a TV voltage. Hence, in the field

    of conventional screen/film (S/F) mammography, subjective or semi-quantitative test objects

    are commonly used to assess image quality. In the UK, this might include the TOR(MAM)

    (Cowen et al 1992) test objects, along with the CDMAM 3.4 (Artinis Medical Systems BV,

    Netherlands) contrast-detail (c-d) test object. All of these test objects provide a controlled

    input signal when imaged under standard conditions and require the observer to perform a

    visual assessment of the image. It is likely that some or all of these methods will be carried

    over in to the assessment of image quality for FFDM units.

    The aim of this work was to assess a Lorad Selenia (Lorad Corp., USA) FFDM imaging

    system using both subjective and objective image quality measurements over a range of air-kerma values at the detector. Subjective image quality was evaluated using c-d data acquired

    with CDMAM test object. While there are known problems with the c-d method (ICRU 1996),

    current acceptable andachievable imagequality standardsin theEUREFdigital mammography

    protocol are couched in terms of a c-d curve. Objective image quality was measured using

    NPS and pre-sampling MTF. Results were then compared in the light of two models: the Rose

    model (Rose 1948) and a version of the well-known signal detection theory model as applied

    to medical imaging systems (Wagner and Brown 1985) supplied by Aufrichtig (1999).

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    3/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2443

    Materials and methods

    Tube and generator

    Tube and generator function was assessed under current protocols (IPEM 2005). Accuracy ofthe tube potential was within the expected range of1 kV of nominal across the range from

    24 kV to 34 kV. X-ray output at 50 cm was 232 Gy mAs1 for a nominal tube potential

    of 28 kV. System half value layer (HVL), measured at 30 kV nominal with the compression

    plate out of the beam and Mo filter selected was 0.31 mm Al giving an estimated x-ray tube

    filtration of 0.54 mm Al equivalent.

    Detector

    The Lorad Selenia uses a direct conversion flat panel detector in which x-rays are directly

    converted to charge by the amorphous selenium (a-Se) (Yorkeret al 2002). An in-depth review

    of this direct detection technology is given by Rowlands (2000). In this type of detector, a

    large potential difference (typically 2000 V) is held across the a-Se layer and this ensures

    that charge generated in the photoconductor by x-ray interaction is drawn directly towardsthe pixel readout system with very little lateral spread. The a-Se x-ray converter layer in the

    Lorad Selenia is 200 m thick (CEP Report 05084 2006). The holes generated during x-ray

    interaction are collected by pixel electrodes and stored on a capacitor located at each pixel.

    Readout of the pixel charge is achieved using an amorphous silicon active matrix thin film

    transistor (TFT) array to switch out the charge, which is then converted to a voltage by an

    amplifier attached to each data line. The 24 29 cm detector is mapped on to a 3584

    4096 pixel array, giving a 70 m pixel size and a sampling limit of 7.1 mm1.

    Variance

    TheNPSdescribesthe spectral decomposition of thenoise variance in an image as a function of

    spatial frequency. When estimating NPS, various assumptions are made regarding the nature

    of the stochastic processes that result in the final image variance. For digital imaging systems,

    it is often assumed that the random process generating the variance is wide sense cyclo-

    stationary (WSCS) (Cunningham 2000), meaning that the expected value and autocorrelation

    of the noise realized in the image are both stationary i.e. they are independent of position in

    the image and that these statistics do not change between two positions separated by an integer

    multiple of the sample spacing (Cunningham 2003, Maidment et al 2003). The importance

    of these assumptions is discussed elsewhere (ICRU 1996, Cunningham 2000, Williams et al

    1999).

    Most NPS algorithms sample image data from a variety of positions across the image

    (Cunningham 2000) and therefore the notion of stationarity is crucial. If we wish to uncover

    the spectral decomposition of the image variance using a method that relies on periodicity,

    such as Fourier analysis, then it is important the variance remains constant over the region

    from which the data are sampled. One of the principal reasons a digital imaging system mayfail this criterion is that of detector artefacts, be it in the form of dead pixels, dead lines or

    some other artefact. Before proceeding to full NPS analysis, it has been recommended that an

    image of the variance is initially formed (Maidment et al 2003) and assessed for stationarity.

    In this study, variance images were produced from a single image using a 10 10 pixel

    region of interest (corresponding to 0.7 0.7 mm). The variance of the pixels within this

    ROI was calculated and this value was assigned to all the pixels in the ROI. The ROI was then

    moved across the image in steps of 10 pixels, so that the entire image was covered. Finally,

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    4/23

    2444 N W Marshall

    the variance value for each ROI was assigned a grey scale value; low variance equal to black

    and high variance equal to white. This image then allows a quick visual inspection of the

    uniformity of the variance in the image and whether there are significant artefacts present.

    Signal transfer property (STP)

    The first step in objective image quality analysis is measurement of the system signal transfer

    property (STP), or detector response as it is defined in the EUREF protocol. This will tell us

    whether the system has a linear relationship between the incident x-ray fluence striking the

    detector and the output parameter (usually a pixel value). For objective image methods to be

    valid the relationship between system input and output must be linear or at least linearizable,

    for the case of screen/film and computed radiography systems. The STP was measured

    following the EUREF protocol by placing 4 cm PMMA at the x-ray tube head with the

    x-ray grid removed and exposing using standard conditions of 28 kV and Mo/Mo target filter

    combination. Initially, a calibrated dose meter was placed 8 cm above the detector at the

    centre of the x-ray field and x-ray output was measured for a range of mAs product values.

    The detector was protected using a lead beam block during this procedure. This enabled theaccurate estimation of air-kerma as a function of mAs at the detector input plane. The lead

    block was then removed and mAs product varied from 4 to 140 mAs and the pixel value (PV)

    from flat field images (i.e. linear images, corrected for detector artefacts) wasmeasured using

    a region of interest (ROI) placed at the centre of the image. The STP was derived by plotting

    measured PV against air kerma at the detector.

    Modulation transfer function (MTF)

    Several methods for measuring MTF have been described in the literature (Judy 1976, Sones

    and Barnes 1984, Hillen et al 1987, Fujita et al 1992, Samei et al 1998) and these have been

    applied to a range of x-ray imaging systems. In this study, a version of the angled edge was

    used to measure pre-sampling MTF i.e. the MTF prior to sampling by the pixel matrix (Giger

    et al 1984). It is well known that FFDM systems are not shift invariant (Dobbins et al 1995,

    Cunningham 2000, Albert and Maidment 2000), as they are based around discrete digital

    detector. Hence, the signal (image) produced by the edge used in the MTF measurement

    method will vary depending upon its position relative to the pixel matrix. The angled-edge

    method is used to produce a very finely sampled edge spread (Samei et al 1998, Albert and

    Maidment 2002) that can be used to estimate the pre-sampling MTF i.e. the MTF prior to

    sampling by the pixel matrix. The details of the angled edge method are described fully

    elsewhere (Samei et al 1998) but some points relevant to the implementation of this method

    are given briefly here.

    Using this method, the pre-sampling MTF can be estimated in two directions: MTF

    parallel to the chest wall edge is defined as the data line direction for this detector and the

    direction away from the chest wall towards the nipple edge is the gate line direction. The MTF

    was measured at the detector centre; MTF was not studied as a function of position across thedetector. First, a 0.5 mm thick copper sheet was placed on the tabletop at the centre of detector

    field and angled slightly with respect to the pixel matrix. An image of the edge was acquired at

    28 kV and 20 mAs and saved to CD-R in DICOM format. As will be shown later, the system

    has a linear signal transfer property and therefore it was not necessary to linearize the image

    before calculating pre-sampling MTF. Next, the angle of the edge was found by differentiating

    along the direction of interest. A simple least-squares method was used to fit a first-order

    polynomial y = a + bxto the pixel values lying along the edge. The angle of the edge was

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    5/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2445

    then calculated as tan1(b) where b is the gradient of the line. Knowing the angle of the edge

    allows us to re-sample the image using a finer sample distance than the pixel pitch to create

    the over-sample. The pixel values from a square region spanning the edge were then assigned

    to the over-sample array, with a position along the array given by the distance from the edge.

    This distance was calculated using the equation of the line (Rogge et al 2003).The pixel datain the over-sample array were then re-binned according to a sub-pixel binning factor (Samei

    et al 1998). In this work we use = 15, which represents a compromise between resolution

    andnoise on the MTF result. Finally, the over-sample wasnormalized for the number of pixels

    assigned to each bin.

    Before differentiating theover-samplededge spread function, somefiltering canbe applied

    in order to reduce noise (see the paper by Maidment and Albert (2003) for a discussion on

    conditioning of MTF data). Great care should be taken when trying to reduce noise in

    the pre-sampling MTF estimateinappropriate choice of filter type could easily bias the pre-

    sampling MTF. In this work, amedian filter is applied to the over-sample before differentiating

    as this a simple and relatively benign filter. After median filtering, the over-sampled edge

    spread function was differentiated, Fourier transformed and normalized to obtain the

    pre-sampling MTF.

    Noise power spectrum (NPS)

    As part of the aim of this paper is to examine the feasibility of measuring objective image

    quality parameters as prescribed in the EUREF protocol, a simple NPS method suitable for

    routine QA measurement was sought. Some of the important theoretical aspects of noise power

    measurements are detailed in the work of Cunningham (2000) and Dobbins (2000). Williams

    et al (1999) give a thorough account of the practicalities of noise power estimation for digital

    mammography systems. Noise power spectra in digital mammography are estimated using

    flat field images, usually produced by imaging a block of PMMA under standard conditions.

    In this study, we used a tube potential of 28 kV, a Mo/Mo target filter combination and the

    4 cm PMMA blocks supplied with the CDMAM test object. These blocks were suspended at

    the x-ray tube port, generating a uniform image with reduced levels of scatter. Images were

    acquired with the anti-scatter grid out of the x-ray beam (similar to the geometry employed

    by Evans et al (2002)).

    In this study we compared objective and subjective image quality parameters across a

    range of input air-kerma values at the image receptor. First, the typical operating point for the

    automatic exposure control (AEC) was taken to be the mAs delivered under AEC control with

    4 cm PMMA and the CDMAM c-d test object placed on the table and with the anti-scatter grid

    in place. AEC sensing region was set to position 2 i.e. approximately 3 cm from chest wall

    edge of the detector. For 28 kV, Mo/Mo factors, the AEC delivered 105 mAs. The standard

    clinical mAs product for 4 cm PMMA and the CDMAM c-d test object was taken to be

    110 mAs (i.e. the nearest mAs station to 105 mAs). This gave a mean PV measured at the

    centre of the image of 507. Flat field images for 4 cm PMMA at an equivalent detector input

    air kerma were acquired by matching the PV for this image; in this case 35 mAs gave a PVof 497. Flat field NPS images were then acquired at values of 4, 9, 18, 35, 70 and 140 mAs,

    corresponding to detector air-kerma values of 9.1, 21.0, 42.3, 82.5 165 and 331 Gy.

    The NPS was calculated using a two-dimensional algorithm from a flat field image as

    follows. First, a region (e.g. 1024 1024 or 2048 2048 pixel) was extracted from the

    central area of the 3584 4096 image. Non-overlapping regions of interest or records of size

    128 128 pixels were then taken from this area (Evans et al 2002). Williams et al (1999)

    acquired their records from a region of approximately 5 cm 5 cm, however we have found

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    6/23

    2446 N W Marshall

    that using a central region of 7.2 cm (of 1024 1024 pixels) or 14.4 cm (2048 2048 pixels)

    gave similar results.

    TheNPS data in this study arecalculated from un-subtracted imagesand thereforecontain

    contributions from stochastic, varying non-stochastic (i.e. non-stochastic but varying from one

    image to another, such as slight ghosting artefacts etc) and fixed pattern noise sources presentin the system (Williams et al 1999). These can be present, despite the flat fielding process that

    is routinely applied to this system. Evans et al (2002) excluded fixed pattern noise sources

    from their analysis by using subtracted images. In our analysis, each record was corrected for

    the presence of background trends by fitting a polynomial and subtracting the fitted surface

    S(x, y) from the record. For a 128 128 record size, there was some difference between first-

    and second-order polynomials at low spatial frequencies (

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    7/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2447

    Table 1. Pixel value and air kerma at detector as a function of mAs for CDMAM images andPMMA images used for NPS analysis.

    CDMAM 4 cm PMMA

    Pixel Air kerma at Pixel Air kerma at

    mAs value detector (Gy) mAs value detector (Gy)

    14 246 11.9 4 231 9.1

    28 269 18.5 9 278 21.0

    55 350 41.0 18 353 42.3

    110 507 84.8 35 497 82.5

    220 828 174 70 796 165

    400 1430 342 140 1388 331

    As discussed by several authors (Hillen et al 1987, Stiersorfer and Spahn 1999, Williams et al

    1999), most current x-ray detectors integrate the energy of the incident x-ray fluence, rather

    than count the individual x-ray photons. This implies that for DQE, we must compare thedetector against an ideal energy integrating device, rather than an ideal photon counter. Hence,

    for an input spectral distribution (E ), q0 in equation (4) is given by

    q0 =

    (E)E dE

    2(E)E2 dE

    . (5)

    We are thereforeusing thevarianceof the incident energyfluence to quantify the input quantum

    noise (Stierstorpher and Spahn 1999). Using the data given by Cranley et al (1997) gives

    a value ofq0 = 5300 photons mm2 Gy1 for the 28 kV spectrum from a Mo target with

    a nominal 30 m Mo filter used in this study. This is close to the value of 5500 photons

    mm2 Gy1 adopted by Evans et al (2002). There is a carbon fibre cover plate of unspecified

    thickness protecting the detector and grid assembly. The effect of this cover plate on the

    number of transmitted photons was not included on the DQE figures presented in this study.

    Contrast-detail data

    To study the c-d performance of the system as a function of dose/image, two sets of CDMAM

    images were acquired at values of 14, 28, 55, 110, 220 and 400 mAs. The detector input air

    kerma for these mAs settings were 12, 18, 41, 85, 174 and 342 Gy. These figures were

    estimated from the detector STP measured previously for 4 cm PMMA. The detector STP

    gradient changes slightly with mean energy of the x-ray beam, however a beam load of 4 cm

    PMMA is sufficiently close to 4 cm PMMA with the CDMAM test object not to introduce

    significant errors into this estimation. Table 1 compares air kerma at the detector for 4 cm

    PMMA (i.e. for the NPS results) against detector air kerma for the CDMAM images.

    The contrast for a gold disc of thickness tAu was calculated as follows. First, the number

    of photons mm2 transmitted through the disc at a given energy E (disc (E )) was calculated:

    disc(E) =

    E max0

    inc(E) exp((Au(E)tAu + PMMA(E)tPMMA + Al(E)tAl)) dE. (6)

    The number of photons mm2 in the background spectrum (no gold disc), bkg(E ), is

    bkg(E) =

    E max0

    inc(E) exp((PMMA(E)tPMMA + Al(E)tAl)) dE (7)

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    8/23

    2448 N W Marshall

    where Au(E ), PMMA(E ) and Al(E ) are the linear attenuation coefficients of gold, PMMA

    and aluminium at energy E, respectively. inc(E ) is the Mo x-ray spectrum incident on to the

    CDMAM test object at 28 kV and includes attenuation by 0.69 mm Be and 0.03 mm Mo tube

    filtration. The contrast of the gold disc (Cp) was then calculated in terms of air kerma using

    Cp =

    E max0

    bkg(E)K(E) dE E max

    0disc(E)K(E) dEE max

    0bkg(E)K(E) dE

    (8)

    where K(E ) is the photon to kerma conversion factor (Cranley et al 1997). The contrast is

    dimensionless and therefore has no physical units.

    As CDMAM is placed between 4 cm of PMMA, the radiation contrasts generated (CP)

    will be degraded by scattered radiation:

    C =CP

    (1 + S/P)(9)

    where S/P is the ratio of scattered to primary radiation (Dance and Day 1984). A typicalfactor for 4 cm breast thickness calculated for a linear grid is approximately 0.45. The Selenia

    system uses a cellular (HTC) grid (Lorad Corp., USA) however, in the absence of data for this

    grid type, a nominal factor of 0.4 was used in this work.

    The CDMAM test object is formed of a grid of 1 cm 1 cm squares, with each square

    containing two identical circular discs, one at the centre and the other in a randomly selected

    corner (the eccentric disc). Included in the CDMAM manual is a scoring and correction

    scheme that utilizes the correct identification of the position of eccentric disc in each square.

    This method, requiring at least three observers, produces reproducible results but is time

    consuming to implement in a QA programme. CDMAM, as used in the QA programme at

    our physics centre, is scored as a conventional c-d test object i.e. the observer counts to the

    last disc visible at the centre of the square for each detail diameter. The contrast of this disc

    is then taken to be the threshold contrast CT

    for that detail diameter. This is the method used

    when scoring fluoroscopy (Hay et al 1985) and digital subtraction angiography (Cowen et al

    1987) c-d test objects. We are therefore comparing subjective and objective measurements

    acquired under typical QA conditions.

    The c-d images were scored by two experienced observers on the radiologist softcopy

    reporting station. This utilizes two Barco (USA) five megapixel cathode ray tube (CRT)

    monitors (2500 2000 pixels with a display bit depth of 10 bits). The luminance response of

    these CRT monitors is checked using a luminancemeter as part of the routine quality assurance

    checks made on the system. The MediCal Pro (Barco, USA) QA package is used for these

    checks with the luminance response calibrated against the DICOM luminance response. The

    CRT monitors are situated in the radiologist reporting room and therefore a low level of

    ambient light was set during the c-d scoring. Care was also taken to keep reflections on the

    CRT screens to a minimum.

    When an image is brought up for display on the softcopy station, the whole image isdisplayed at reduced resolution (i.e. not at 1 to 1 pixel resolution) so that the 3584 4096

    image can be displayed on the monitor. The software supplied includes a magnification

    function that allows a section of the image to be viewed at full resolution and this was

    used during the scoring. Before scoring the images, window width and level were set to a

    satisfactory level by the observers. C-d images tend to have a narrow dynamic range and

    therefore fairly narrow window widths (high contrast setting) can be used when scoring this

    type of image.

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    9/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2449

    Contrast-detail models

    The experimentally derived c-d data were compared against two observer models. The c-d

    results were modelled using the version of signal detection given by Aufrichtig (1999) for the

    task of detecting a static, circular disc shaped object imaged against a homogeneous noisybackground. The SNR for this detection task can be written as

    SNR =C|S(u, v) MTF(u, v)O(u,v)|2 du dv

    |S(u, v) MTF2(u, v)O2(u,v)|2(q0 DQE(u, v))1 du dv1/2 (10)

    where S(u, v) is thespectrumof thetarget objectin frequency space, MTF(u, v)andDQE(u, v)

    are the imaging system MTF and DQE, respectively and O(u, v) represents the spatial

    frequency response function of the observers visual system. For detection to occur according

    to this model, theSNRwill have exceededsome thresholdcriterion fordetection, ksay, held by

    the observer. At this point, the contrast for the signal detection model will be at the threshold

    defined as CTS

    CTS = k

    |S(u, v) MTF(u, v)O(u/m, v/m)|2

    du dv|S(u,v) MTF2(u, v)O2(u/m, v/m)|2(q0 DQE(u, v))1 du dv

    1/21

    . (11)

    The spatial frequency response of the observer was taken from Kelly (1979). We also studied

    the effect of setting O(u, v) = 1 (i.e. with no spatial filtering by the observer).

    As the images are presented to the observer with some magnification by the soft copy

    workstation (SCW), we could model the effect of display magnification factor using the factor

    m in equation (11). A typical viewing distance of 25 cm was set for the calculation ofCTS.

    The software magnification function was used when scoring the test objects so that sections

    of the image were displayed at a 1 to 1 pixel resolution. When this was engaged, 8 cm in the

    image was displayed as 18.4 cm on the screen giving a magnification m= 2.3. With the image

    displayed at the default size (i.e. without the magnification function), the SCW magnification

    was 1.38.

    The model of Rose (1948) was used to calculate the threshold contrast CTR for a circularobject of diameter using the using the relationship:

    CTR =2k

    1/2

    1

    q0 DQE(0)

    1/2(12)

    where q0 is the number of photons mm2 incident on the x-ray detector and DQE(0) is the

    detector DQE at zero spatial frequency. Assumptions required for the Rose approach to be

    valid include (1) the photons are assumed to be uncorrelated (2) the signal at the detector

    is flat-topped and has sharp edges (there is no imaging system blur) (Burgess 1999). The

    relationship between the signal detection theory given in equation (10) and the theory of Rose

    is discussed at length by Burgess (1999).

    In equations (11) and (12), k is the observers threshold signal-to-noise ratio (SNR).

    Various values ofkhave been reported in the literature (Rose 1948, Schnitzler1973, Moran1982, Ishida et al 1984). k depends on the detection threshold adopted by the observers

    during the period of the experiment. Although trained observers can hold this threshold

    reasonably constant over a period of time, the actual value adopted by each observer is not

    known or controlled in a contrast-detail measurement of this type. In previous studies we have

    found that a value ofk= 3.8 (Ishida et al 1984) gives calculated results that are close to the

    measured results (Marshall et al 2001). A strict observer would demand that the threshold

    disc is extremely well visualized (e.g. a clear, continuous edge and well-defined position) and

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    10/23

    2450 N W Marshall

    (a) (b)

    Figure 1. (a) Variance image produced by imaging a 4 cm thick PMMA block at 28 kV, Mo/Mo,95 mAs using a 10 10 pixel ROI. (b) The same grey scale data plotted as an SNR image.

    have a correspondingly high value ofk(around 4 or 5). A more lenient observer would be

    operating nearer 50% detectability and have a lower value ofk(closer to 2 or 3) (Burgess

    1999). Changing kshifts the calculated or expected threshold contrasts for the system with

    respect to the y-axis. This is a shortcoming of the contrast-detail method used in the current

    study with regard to the prediction of absolute threshold contrasts (ICRU 1996). However,

    this type of experiment can still provide valuable qualitative information on the performanceof the system as various parameters are changed (e.g., detector input dose).

    Results and discussion

    Variance

    All FFDM systems have a software map containing the locations of defective pixels. On

    calibration of the detector by the engineer, some form of correction algorithm is applied

    whereby a pixel value (PV) interpolated from nearest pixel neighbours is given to the defective

    pixel. This is a form of averaging, reducing variance locally and should show up as a region

    of reduced variance in the variance image. Defective pixels that have not been mapped by

    the engineer, on the other hand, may show up as regions of increased variance in the variance

    image.Figure 1(a) shows a variance image for the Selenia system generated at from one image

    acquired at 95 mAs (28 kV, Mo/Mo, 4 cm PMMA). Variance is uniform across the central

    and chest wall regions of the image, increasing as we move further away from the chest wall

    edge. As a result of the heel effect, the number of x-ray photons mm2 is greater at the chest

    wall than at the nipple edge. X-ray photon noise follows a Poisson distribution and therefore

    x-ray variance should increase at the chest wall. This is not seen and is possibly a result of the

    flat field and detector gain corrections affecting the noise variance. The variance is lower at

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    11/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2451

    the image detector edges (top and bottom in this figure) and this may be due to reduced noise

    from the lower TFT pixel capacitance at the detector edges.

    Inaddition to thevariance image,wecanalsoderive a visualassessmentof SNRuniformity

    for a single image using this method (figure 1(b)). The SNR in this instance was calculated

    from the mean pixel value for the ROI divided by the standard deviation of the pixels withinthe ROI. This is of interest as the EUREF Digital Addendum gives a limit of 15% for SNR

    uniformity calculated from five regions in the image. The image of the SNR gives a fuller

    realization of the SNR variation across a flat field image and shows that SNR increases at the

    chest wall. While we can expect excellent pixel value uniformity for a grey scale image as a

    result of the flat fielding process, the SNR will tend to be less uniform (especially in the chest

    wall to nipple direction) because of the heel effect. This should be taken into account when

    devising QA protocols for these systems.

    Finally, it should be noted that this system developed a fault that was seen by the

    radiographer during the flat fielding QA process (note that this study was performed after

    the faulty detector was replaced). A spongy artefact was observed at the chest wall edge of

    the detector on the grey scale flat field image. This was investigated by the field engineer

    and ascribed to delamination of the a-Se layer and the TFT array. Ultimately, this fault wastraced by the manufacturer to a known set of detectors via the detector serial numberall

    detectors with this fault have since been replaced. The detector fault is clearly demonstrated

    by the variance image methodsee figure 2. In fact, the variance image indicates a second

    artefact region that had not been noted by the radiographer. The variance image also uncovers

    a line of dead pixels at the bottom left of the image. Extracts from the greyscale image are

    presented to show the bubble-like appearance of the greyscale x-ray image in the fault region.

    The variance image is now included as part of the routine physics QA measurements for the

    FFDM systems at this site.

    Signal transfer property

    Pixel value was plotted against air kerma at the detector to give the STP shown in figure 3.

    Coefficient of variation for theairkermaat thedetector in this measurement was approximately

    0.2%. While there is a linear response between PV and air kerma at the receptor, there is

    clearly an offset to the STP. The offset is added at calibration by the engineers and is used to

    avoid negative pixel values in the image.

    Modulation transfer function

    Pre-sampling MTFs calculated using median filter lengths of 5 and 7 pixels showed a slight

    reduction at high spatial frequencies for the 7 pixel filter and therefore a median filter of

    just 5 pixels was applied when calculating MTF in this study. Figure 4(a) shows the pre-

    sampling MTF in the data line and gate line directions, along with the pixel aperture MTF for

    a pixel size of 0.07 mm. Results for both directions are similar, indicating that pre-sampling

    MTF is fairly isotropic. All three curves touch the x-axis at the same value of approximately14 mm1this is an indication that the actual pixel size is close to the nominal pixel size of

    0.07 mm (a first zero of 14.3 mm1 is expected for a 0.07 mm pixel). The limit caused by the

    aperture response can be higher if the dimension of the sensitive region of the pixel (detector

    element) is smaller than the nominal value (Rowlands 2000). The measured pre-sampling

    MTF is some way below the limit set by the pixel sampling aperture. Yorkeret al (2002) and

    Zhao et al (2003) show that re-absorption of K-shell fluorescence x-rays at a position remote

    from the initial interaction is a source of blurring, leading to a reduction in MTF compared to

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    12/23

    2452 N W Marshall

    line of dead pixels

    greyscale extract

    greyscale extract

    Figure 2. Variance image of a 4 cm PMMA block acquired at 28 kV, Mo/Mo and 60 mAs clearlydemonstrating two regions of severe artefact. The extracted regions show greyscale images of theartefact. The variance image also highlights a line of dead pixels.

    the ideal aperture. Zhao et al (2003) also suggest that charge trapping in the blocking layer

    just above the pixel electrodes may also be a source of MTF degradation.Plotted in figure 4(b) are data taken from Yorkeret al (2002) for an earlier version of this

    detector. The measured pre-sampling MTF in this study is somewhat lower at all frequencies

    compared to the result of Yorkeret al (2002). Factors that could account for such a difference

    include changes in design parameters, such as the voltage across the a-Se layer. This figure

    also presents the pre-sampling MTF given by Zhao et al (2003) for a 200 m thick a-Se

    detector with an 85 m pixel size. For comparison with a typical screen-film detector, the

    MTF of a Min-R 2000 mammography screen-film system (Bunch 1999) is plotted. The MTF

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    13/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2453

    y = 3.58x + 203

    0

    200

    400

    600

    800

    1000

    1200

    1400

    1600

    0 50 100 150 200 250 300 350

    air kerma at detector (Gy)

    PixelValue

    grid out 28 kV, Mo Mo 4 cm

    PMMA at tube

    Figure 3. Signal transfer property for the a-Se detector.

    0.0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1.0

    0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

    spatial frequency (mm-1)

    MTF

    data linegate lineaperture

    0.0

    0.1

    0.2

    0.3

    0.4

    0.5

    0.6

    0.7

    0.8

    0.9

    1.0

    0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

    spatial f requency (mm-1)

    MTF

    data line (this study)

    MinR-2000

    Yorker et al

    Zhao et al data line

    (a) (b)

    Figure 4. (a)MTF measuredin thedataline(parallel tothe chestwall edge)andgatelinedirections.Also plotted is the response for a sinc function of 0.07 mm (the pixel size for this detector).(b) MTF result in data line direction compared against a-Se digital mammography detector data

    from Yorkeret al (2002) (14.3 mm

    1 sampling limit) and Zhao et al (2003) (11.8 mm

    1 samplinglimit). Also shown is the MTF for a Min-R 2000 mammography film/screen system taken fromBunch (1999).

    of the much thicker a-Se layer (200 m versus approx 80 m for Min-R phosphor screen)

    is surprisingly close to screen-film MTF and is due to electric field applied to the a-Se layer

    stopping lateral charge spread. The screen-film MTF is superior to that of the a-Se layer at

    spatial frequencies greater than 12 cycles mm1. The 70 m pixel size in the Selenia system

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    14/23

    2454 N W Marshall

    1.0E-07

    1.0E-06

    1.0E-05

    1.0E-04

    0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0

    spatial frequency (mm-1)

    NNPS(u)(mm

    2)

    9.1 Gy 21.0 Gy 42.3 Gy

    82.5 Gy 165.3 Gy 330.9 Gy

    gate line

    Figure 5. Normalized NPS measured in the data line (symbols) and gate line directions (solidlines) for a range of detector air-kerma settings.

    means that only frequencies up to the sampling limit of 7.1 cycles mm1 can be displayed in

    the digital images.

    Noise power spectrum

    Figure 5 shows NPS in the data line and gate line directions at detector air-kerma values of

    9.1, 21.0, 42.3, 82.5 165 and 331 Gy. A detailed exploration of the noise transfer properties

    of integrated x-ray detectors (such as that given by Siewardsen et al (1997), Evans et al (2002)

    and Zhao et al (2003)) is beyond the scope of the current workjust a few salient points will

    be discussed.

    The NPS has similar magnitude in both the gate line and data line directions, indicating

    good isotropy (as might be expected from the MTF results). In the data line direction, NPS

    at the sampling limit has fallen to approximately 60% of the NPS(0) value, while in the gate

    line direction, NPS at the sampling limit is roughly 40% of the NPS(0) value. These results

    are close to the values given by Zhao et al (2003) for an a-Se detector.

    As this is a digital detector, the measured noise power spectrum will suffer from aliasingbecause the noise data have been sampled at discrete intervals by the pixel matrix (Gigeret al

    1984). Williams et al (1999) discuss aliasing of the NPS for digital mammography systems;

    detectors that produce images with significant spatial frequency content above the sampling

    limit prior to sampling will suffer from aliasing. Figure 4(a) shows that the pre-sampling MTF

    of the a-Se layer has fallen to only 50% at 7.1 mm1 and hence some aliasing of the NPS

    for this detector is expected. Ultimately, aliasing leads to an increase in noise power at high

    spatial frequencies.

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    15/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2455

    0.00

    0.10

    0.20

    0.30

    0.40

    0.50

    0.60

    0.70

    0.80

    0.90

    1.00

    0.0 1.0 2.0 3.0 4.0 5.0 6.0 7.0spatial frequency (mm-1)

    DQE(u)

    9.1 Gy 21.0 Gy 42.3 Gy

    82.5 Gy 165.3 Gy 330.9 Gy

    Figure 6. DQE in the data line direction measured for a range of detector air-kerma values. X-rayspectrum was 28 kV, Mo/Mo target filter and 4 cm PMMA added filtration.

    Detective quantum efficiency

    Figure 6 present DQE(u) measured for the data line direction at detector air-kerma values of

    9.1, 21.0, 42.3, 82.5 165 and 331 Gy. The curves have a similar shape, regardless of the

    air kerma at which they were measured, dropping by a factor of approximately 3 as spatial

    frequency changes from 0 mm1 to 7.1 mm1. It is clear that DQE is dependent on air

    kerma to the detector. DQE increases for all spatial frequencies as air kerma at the detector

    increases, until reaching a plateau above approximately 50

    Gy. At low spatial frequency,DQE increases from approximately 0.3 at 9.1 Gy to approximately 0.6 for detector air-kerma

    values of 50 Gy and above. The reduction in DQE at low detector air kerma is due to the

    presence of additive noise from the input stages of charge amplifier read out circuitry (Yorker

    et al 2002). Detector air kerma will be low for regions of the breast that have low x-ray

    transmission (probably corresponding to areas of highly glandular or dense tissue). In these

    regions, it is likely that DQE will be reduced and this may impact on the ability of the system

    to detect clinical details in these areas.

    Absolute magnitude of the DQE for air-kerma values above 50 Gy is roughly 0.6 at

    low spatial frequenciesin good agreement with data published by Yorker et al (2002) and

    Zhao et al (2003) for similar detectors. DQE for this a-Se detector is therefore considerably

    greater than the figure of 0.35 (at low frequencies) for a well-established mammography

    screen/film (S/F) system such as Kodak Min-R 2000 (Bunch 1999).The advantage of the

    S/F system is that DQE extends above the 7 mm1 limit imposed by the sampling matrixfor the Selenia system (albeit at the low value of 0.1). As it is thought that detectability

    for a given task is related to the DQE (ICRU 1996), we should expect improved detection

    performance compared to S/F detectors (certainly for well-defined signal known exactly,

    background known exactly (SKE/BKE) tasks). It should also be noted that DQE is calculated

    using the pre-sampling MTF in combination with the digital (sampled) NPS (equation (4)).

    As noted above, aliasing results in an increase in the high frequency NPS and hence DQE

    calculated in this manner may be an underestimate of the true DQE at high frequencies.

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    16/23

    2456 N W Marshall

    0.1

    1.0

    10.0

    0.01 0.10 1.00 10.00

    detail diameter (mm)

    thresholdcontras

    t(%)

    9.1 Gy

    theory

    42.3 Gy

    330.9 Gy

    0.1

    1.0

    10.0

    100.0

    0.01 0.10 1.00 10.00

    detail diameter (mm)

    thresholdcontrast(%)

    11.9 Gy

    theory

    41.0 Gy

    342 Gy

    Figure 7. Observer c-d results at three detector air-kerma settings (symbols) and c-d curvescalculated according to the Rose theory.

    Table 2. DQE(0) found by extrapolating DQE(u) back to the y-axis.

    Air kerma at detector (Gy) DQE(0)

    9.1 0.29

    21.0 0.43

    42.3 0.54

    82.5 0.61

    165 0.62

    331 0.59

    Contrast-detail results and the Rose model

    The observer c-d results are shown in figure 7; for reasons of clarity, only data for detector

    air-kerma values of 11.9, 41.0 and 342 Gy are plotted out of the six sets of data available.

    Note that for the 342 Gy results, all the discs present in the row for diameters of 1.0 mm

    and above were seen and these threshold contrast points have also been excluded from the

    plot. The errors bars indicate an uncertainty of 15% (Cohen et al 1984), estimated for two

    observers reading two images once

    Also plotted are the expected c-d curves calculated from Rose theory (equation (12)) at

    air-kerma values of 11.9, 41.0 and 342 Gy. Two assumptions were made regarding DQE

    for this calculation. The first is a practical pointwe are using DQE results obtained at

    slightly different detector air-kerma values (9.1, 42.3 and 331 Gy) for the calculation. DQE

    is changing quite quickly for the low air-kerma results (11.9 Gy) and this 2 Gy differencewill introduce some error. Above 50 Gy at the detector, DQE reaches a plateau and changes

    more slowly with air kerma and virtually no error is introduced by this assumption. Second,

    in order to find DQE(0), a first-order linear extrapolation to DQE(u) was used (table 2). This

    is because the measured value of DQE is often affected by low frequency artefacts present in

    the image (Dobbins 2000).

    In order to find a value fork, this equation was fitted to the observer threshold contrast

    results for the centre set of air-kerma results (42.3 Gy), for detail sizes ranging from 2.0 mm

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    17/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2457

    0.1

    1.0

    10.0

    100.0

    1.0 10.0 100.0 1000.0

    air kerma at detector (Gy)

    Ct

    (%)

    0.80 mm

    limiting DQE=0.6

    CTR

    (a)

    0.1

    1.0

    10.0

    100.0

    1.0 10.0 100.0 1000.0

    air kerma at detector (Gy)

    Ct

    (%)

    0.20 mm

    limiting D QE=0.6

    CTR

    (b)

    0.1

    1.0

    10.0

    100.0

    1.0 10.0 100.0 1000.0

    air kerma at detector (Gy)

    Ct

    (%)

    0.13 mm

    limiting DQE=0.6

    CTR

    (c)

    Figure 8. Comparison of observer threshold contrast results (squares) and Rose threshold contrastcalculated using DQE(0) (circles) for (a) 0.80 mm, (b) 0.20 mm and (c) 0.13 mm diameter discs.Dotted line shows result for limiting DQE of 0.60.

    down to 0.13 mm. This gave a value ofk= 2.56 and therefore an approximate value of 2.5

    was used forkfor all calculations ofCTR and CTS in this paper.

    The model gives results within approximately 15% for the c-d curve with changing

    detector air kermathe spread between the low and high air-kerma results simply reflects the

    changes in quantum noise in the image. The CDMAM test object is sensitive enough (in terms

    of percentage contrast step between consecutive discs) to follow these changes in quantum

    mottle.

    Calculated and measured results were also within approximately 15% for disc diametersdown to 0.2 mm, however there is considerable discrepancy for diameters less than 0.20 mm.

    This is examined more closely in figure 8, where the threshold contrast data for a particular

    detail diameter have been plotted as a function of detector air kerma. In order to smooth the

    measured c-d data, a second-order least-squares fit wasmade to all the c-d curves. Data plotted

    in figure 8 arecontraststakenfrom thefitted curves rather than the raw thresholdcontrast result.

    The dotted curve in figure 8(a) is the Rose threshold contrast (CTR) calculated for

    DQE = 0.6 (we are using this as the approximate limiting value of the DQE) and 0.8 mm

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    18/23

    2458 N W Marshall

    0.1

    1.0

    10.0

    100.0

    0.01 0.10 1.00 10.00

    diameter (mm)

    thresholdcontrast(%)

    11.9 Gy

    41.0 Gy

    342 Gy

    theory

    Figure 9. Observer c-d results at three detector air-kerma settings (symbols) and c-d curvescalculated using signal detection model (equation (11)) with observer visual response (O(u,v)) =1 and system magnification m = 1.

    detail diameter. The measured threshold contrast is well modelled by the theory of Rose

    using DQE(0) with an average difference between measurement and theory of 10%. At the

    two lowest air-kerma values studied, the DQE drops due to the presence of detector amplifier

    noise and this clearly results in an increase in threshold contrast. As air kerma increases,

    measured and calculated threshold contrast approach the limiting value determined by DQE

    of 0.6. Figure 8(b) shows that similar results are found for the 0.2 mm diameter discs.

    At the smallest disc diameters (0.13 mm and 0.10 mm), however, the difference between

    the measured result and the contrast calculated using DQE(0) is approximately 40% (see

    figure 8(c)). One possible reason for this is that the observer visual response (not included

    in equation (6)) is starting to limit threshold contrast. There will be some influence of the

    visual response (especially for small detail sizes) on the measured c-d results but the use of

    carefully controlled viewing conditions and the software magnification function on the SCW

    should keep this to minimum. Another reason might be that detection of such small details

    uses a higher spatial frequency region of the DQE (ICRU 1996).

    Threshold contrast follows the expected Rose relationship, even for high air-kerma values

    at the detector. A previous study (Marshall et al 2001) has discussed the influence of fixed

    pattern noise in the form of phosphor structure mottle (Kume et al 1986) on native digital

    fluorography images. This can limit the threshold contrast resolution of an x-ray system at

    high detector air-kerma values. It is clear that is not the case for the a-Se detector in this

    system; as expected the a-Se x-ray converter does not suffer from the phosphor granularitynoise that can limit contrast resolution. Low frequency DQE remains close to 0.6, even at

    high detector air-kerma values.

    Contrast-detail results and signal detection theory model

    Threshold contrast (CTS) was calculated using equation (11) for detector air-kerma values of

    9.1, 42.3 and 331 Gy. Although this equation uses the 2D forms of MTF, NPS and DQE, the

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    19/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2459

    0.1

    1.0

    10.0

    100.0

    1.0 10.0 100.0 1000.0

    air kerma at detector (Gy)

    thresholdcontrast(%)

    0.80 mm

    Rose C% limiting DQE=0.6

    signal detection theory

    0.1

    1.0

    10.0

    100.0

    1.0 10.0 100.0 1000.0

    air kerma at detector (Gy)

    thresholdcontrast(%)

    0.20 mm

    Ro se C% limiting DQE=0.6

    signal detection theory

    0.1

    1.0

    10.0

    100.0

    1.0 10.0 100.0 1000.0

    air kerma at detector (Gy)

    thresholdcontrast(%)

    0.13 mm

    Ro se C% limiting DQE=0.6

    signal detection theory

    (a) (b)

    (c)

    Figure 10. Comparison of observer threshold contrast data (squares) and signal detection model(triangles) for (a) 0.80 mm, (b) 0.20 mm and (c) 0.13 mm diameter discs. Dotted line shows resultfor limiting DQE of 0.60. Signal detection results were calculated with observer visual response(O(u,v)) = 1 and system magnification m = 1.

    results demonstrated reasonably good isotropy and therefore just the results from the data line

    direction were used to simplify the calculations. When calculating threshold contrast we set

    system magnification m = 1 and O(u) = 1, assuming initially that observers could optimize

    their viewing conditions and that the effect of the display/perception would be negligible.

    A similar approach was adopted by Workman and Cowen (1993) for computed radiographysystems. Figure 9 plots the observed c-d data along with CTS calculated from equation (11)

    with an observer spatial frequency response O(u) = 1, m = 1 and k= 2.5. Again, only

    data for detector air-kerma values of 9.1, 41.0 and 331 Gy are plotted. Two points can be

    made here. First, the calculated curves follow the measured data closely as air kerma at the

    detector is changed. Second, the full spatial frequency range DQE is used in the calculations

    and therefore somewhat better agreement is seen at small disc sizes compared to the use of

    DQE(0) in the Rose theory calculations. For example, the difference between measured and

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    20/23

    2460 N W Marshall

    0.1

    1.0

    10.0

    100.0

    0.01 0.10 1.00 10.00

    diameter (mm)

    thresholdcontrast(%)

    11.9 Gy

    41.0 Gy

    342 Gy

    theory

    Figure 11. Comparison of observer c-d data (symbols) and signal detection model calculatedwith system magnification m= 1.38, typical observer visual response function O(u,v) taken fromKelly (1979) and a viewing distance of 25 cm.

    calculated data for the 0.13 mm disc at 342 Gy for the detection theory calculation is 26%

    compared to a difference of 44% for the Rose theory result.

    Figure 10 examinessignal detectionmodel results as a function of airkerma at thedetector.

    Figures 10(a) and (b) show the measured threshold contrast data along with calculated contrast

    CTS plotted as a function of air kerma at the detector for 0.8 mm and 0.2 mm discs. For these

    diameters, the difference between measured and calculated threshold contrasts was 8% on

    average. Figure 10(c) plots the results for 0.13 mm diameter discs where the difference

    between measured and calculated results is 20%. There are several possible reasons for this

    discrepancy. For example, the MTF and NPS data were acquired under a low scatter geometry

    while scattered radiation was present at the c-d acquisition stage. It is possible that the

    observers are using a different detection threshold for the smallest discs. Another possibility

    is that the observers visual system spatial frequency response is starting to influence the

    results. We should also consider the possibility that internal visual system noise may be

    affecting observer performance (Ishida et al 1984, Abbey and Bochud 2000) for detail sizes

    less than 0.2 mm.

    Finally, CTS was calculated using a spatial frequency response curve for a typical human

    observer (Kelly 1979), with a viewing distance of 25 cm and a magnification factorm of 1.38

    (magnification for an image displayed at the default size)seefigure 11. For these conditions,

    the observer spatial frequency response used increased from 0.14 at 0.1 mm1 to a value of

    1.0 at 0.67 mm1 and then fell to 1.70 104 at 5 mm1. This gives the characteristic bowoften seen in c-d curves, especially for larger detail diameters. Agreement is now closer

    at the smallest disc diameters, however, it should be noted that the observed c-d data were

    acquired using the software magnification function andhence m should be 2.3. Setting m= 2.3

    in equation (11) gives values for CTS that are too low at small diameters. There was no

    significant difference (within experimental error) in the observed threshold contrast data

    when measured with and without the software magnification tool on this system. Further

    modification of equation (11) appears necessary in order to correctly model the effect of the

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    21/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2461

    observer visual system and system display magnification on these c-d curves. These results

    highlight thedifficulty in using objective measurementssuch as NPSto model/predict absolute

    values, even for the simple c-d task. The objective measurements are an evaluation of detector

    performance alone, while the observer threshold contrast results include effects of observer

    detectability, observer visual response, display magnification and processing, ambient viewingconditions including display reflection and scattered radiation/grid performance

    Conclusions

    This study used objective and subjective measures to evaluate the image quality of an a-Se

    digital mammography system. The objective analysis began with the generation of variance

    images from flat field images. These images were useful in obtaining a visual impression

    of the variance across the entire detector before proceeding to perform NPS analysis on the

    images. The variance image also clearly identified a severe detector artefact and a line of

    dead pixels not included in the system dead pixel map. Results from variance images were

    then used to generate SNR images which showed the varying x-ray SNR across image due

    to the heel effect, as would be expected for a typical quantum noise dominated x-ray system.

    This variation in x-ray SNR should be taken into account when deriving performance limits

    in testing protocols.

    Both pre-sampling MTF and axially averaged NPS were reasonably isotropic. At the

    sampling limit (7.1 mm1), the NPS fell to approximately 60% and 40% of the NPS(0) value,

    for the data line and gate line directions, respectively. DQE calculated from these parameters

    was approximately 0.6 at low spatial frequencies. DQE was also found to change with air

    kerma at the image receptor, increasing from 0.3 (at low spatial frequencies) at 9.1 Gy to

    approximately 0.6 for air-kerma settings of 50 Gy and above.

    Subjective image quality results in the form of c-d measurements were then obtained

    at the same detector air-kerma settings used for the objective analysis. The measured c-d

    results reflected the reduction in DQE found at the low detector air-kerma setting; c-d data

    also followed the changes in detector air kerma/image even for the relatively high air kerma of342 Gy/image. Two models (the Rose model and a version of signal detection theory) were

    used to examine the measured c-d data. The Rose model (using DQE(0)) gave results within

    approximately 15% for all the detector air-kerma values studied and for detail diameters down

    to 0.2 mm. Similar agreement (approximately 15%) was also found between the measured

    c-d data and signal detection theory results, which were calculated with an ideal human visual

    response function and a system magnification of unity. Using the full spatial frequency DQE

    improved the agreement between the calculated and observer results for detail sizes below

    0.13 mm.

    This studyhas demonstrated closecorrespondence, in functional terms, between objective

    detector imagequality parameterscalculateddirectly from imagedataand subjective (contrast-

    detail) image quality results. As QA physicists gain experience with objective image quality

    analysis over a wider range of digital mammography systems, standards can be formed using

    these objective parameters. Eventually, we may hope to perform quality assurance of the x-ray

    detector using measured image quality data rather than a carefully controlled, but ultimately

    limited subjective task such as the contrast-detail method.

    Acknowledgment

    I would like to thank Dr Julie Horrocks for her help in preparing this work.

  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    22/23

    2462 N W Marshall

    References

    Albert M and Maidment A D 2000 Linear response theory for detectors consisting of discrete arrays Med. Phys.

    27 241734

    Aufrichtig R 1999 Comparison of low contrast detectability between a digital amorphous silicon and a screenfilmbased imaging system for thoracic radiography Med. Phys. 26 134958

    Burgess A E 1999 The Rose model, revisited J. Opt. Soc. Am. A 16 63346

    Bunch P C 1999 Advances in high-speed mammographic image quality Proc. SPIE 3659 12030

    CEP Report 05084 2006 Full Field Digital Mammography Systems Hologic Selenia: A Technical Report(London:

    HMSO)

    Cohen G, McDaniel D L and Wagner L K 1984 Analysis of variations in contrast-detail experiments Med. Phys.

    11 46973

    Cowen A R, Brettle D S, Coleman N J and Parkin G J 1992 A preliminary investigation of the imaging performance

    of photostimulable phosphor computed radiography using a new design of mammographic quality control test

    object Br. J. Radiol. 65 52835

    Cowen A R, Haywood J M, Workman A and Clarke O F 1987 A set of X-ray test objects for image quality control in

    digital subtraction fluorography: I. Design considerations Br. J. Radiol 60 10019

    Cranley K, Gilmore B J, Fogarty G W A and Desponds L 1997 Catalogue of diagnostic x-ray spectra and other data

    IPEM Reportvol 78 (York: Institute of Physics and Engineering in Medicine)

    Cunningham I A 2000 Applied linear systems theory Physics and Psychophysics Vol 1 of Handbook of MedicalImaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 79159

    Cunningham I A and Shaw R 1999 Signal-to-noise optimization of medical imaging systems J. Opt. Soc. Am. A 16

    113

    Dainty J C and Shaw R 1974 Image SciencePrinciples, Analysis and Evaluation of Photographic-Type Imaging

    Processes (London: Academic)

    Dance D R and Day G J 1984 The computation of scatter in mammography by Monte Carlo methods Phys. Med.

    Biol. 29 23747

    Dobbins J T III 2000 Image quality metrics for digital systems Physics and Psychophysics Vol 11 ofHandbook of

    Medical Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 161222

    Dobbins J T III, Ergun D L, Rutz L, Hinshaw D A, Blume H and Clark D C 1995 DQE(f) of four generations of

    computed radiography acquisition devices Med. Phys. 22 158193

    EUREF Digital Addendum 2003 Addendum on Digital Mammography to Chapter 3 of the European Guidelines for

    Quality Assurance in Mammography Screening (Version 1.0, November 2003) (Nijmegen, the Netherlands:

    European Reference Organisation for Quality Assured Breast Screening and Diagnostic Services (EUREF))

    Evans D S, Workman A and Payne M 2002 A comparison of the imaging properties of CCD-based devices used forsmall field digital mammography Phys. Med. Biol. 47 11735

    Flynn M J and Samei E 1999 Experimental comparison of noise and resolution for 2 k and 4 k storage phosphor

    radiography systems Med. Phys. 26 161223

    Fujita H, Tsai D-Y, Itch T, Doi K, Morishita J, Ueda K and Ohtsuka A 1992 A simple method for determining the

    modulation transfer function in digital radiography IEEE Trans. Med. Imaging 11 3439

    Giger M L, Doi K and Metz C E 1984 Investigation of basic imaging properties in digital radiography: 2. Noise

    Wiener spectrum Med. Phys. 11 797805

    Hanson K M 1983 Variations in task and the ideal observer Proc. Soc. Photo-Opt. Instrum. Eng. 419 6067

    Hay G A, Clarke O F, Coleman N J and Cowen A R 1985 A set of X-ray test objects for quality control in television

    fluoroscopy Br. J. Radiol. 58 33544

    Hillen W, Schiebel U and Zaengel T 1987 Imaging performance of a digital storage phosphor system Med. Phys.

    14 74451

    ICRU 1996 Medical imagingthe assessment of image quality ICRU Report 54 (Bethesda, MD: International

    Commission on Radiation Units and Measurements (ICRU))

    IPEM 2005 Commissioning and routine testing of mammographic X-ray systems IPEM Report 89 ed A C Moore,

    D R Dance, D S Evans, C P Lawinski, E M Pitcher, A Rust and K C Young (York: IPEM)

    Ishida M, Doi K, Loo L-N, Metz C E and Lehr J L 1984 Digital image processing: effect on detectability of simulated

    low-contrast radiographic patterns Radiology 150 56975

    Judy P F 1976 The line spread function and modulation transfer function of a computed tomographic scannerMed.

    Phys. 3 2336

    Kelly D H 1979 Motion and vision II stabilized spatio-temporal threshold surface J. Opt. Soc. Am. 69 13409

    Kume Y, Doi K, Ohara K and Giger M L 1986 Investigation of basic imaging properties in digital radiography: 10.

    Structure mottle of II-TV digital imaging systems Med. Phys. 13 84349

    http://dx.doi.org/10.1118/1.1286592http://dx.doi.org/10.1118/1.1286592http://dx.doi.org/10.1118/1.598630http://dx.doi.org/10.1118/1.598630http://dx.doi.org/10.1118/1.598630http://dx.doi.org/full_texthttp://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.595539http://dx.doi.org/10.1118/1.595539http://dx.doi.org/10.1088/0031-9155/29/3/003http://dx.doi.org/10.1088/0031-9155/29/3/003http://dx.doi.org/10.1088/0031-9155/29/3/003http://dx.doi.org/10.1118/1.597627http://dx.doi.org/10.1118/1.597627http://dx.doi.org/10.1118/1.597627http://dx.doi.org/10.1088/0031-9155/47/1/309http://dx.doi.org/10.1088/0031-9155/47/1/309http://dx.doi.org/10.1118/1.598656http://dx.doi.org/10.1118/1.598656http://dx.doi.org/10.1118/1.598656http://dx.doi.org/10.1109/42.126908http://dx.doi.org/10.1109/42.126908http://dx.doi.org/10.1118/1.595583http://dx.doi.org/10.1118/1.595583http://dx.doi.org/10.1118/1.596127http://dx.doi.org/10.1118/1.596127http://dx.doi.org/10.1118/1.594283http://dx.doi.org/10.1118/1.594283http://dx.doi.org/10.1118/1.594283http://dx.doi.org/10.1118/1.595808http://dx.doi.org/10.1118/1.595808http://dx.doi.org/10.1118/1.595808http://dx.doi.org/10.1118/1.595808http://dx.doi.org/10.1118/1.594283http://dx.doi.org/10.1118/1.596127http://dx.doi.org/10.1118/1.595583http://dx.doi.org/10.1109/42.126908http://dx.doi.org/10.1118/1.598656http://dx.doi.org/10.1088/0031-9155/47/1/309http://dx.doi.org/10.1118/1.597627http://dx.doi.org/10.1088/0031-9155/29/3/003http://dx.doi.org/10.1118/1.595539http://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.598630http://dx.doi.org/10.1118/1.1286592
  • 7/29/2019 Marshall 2006a OBJ vs SUBJ for Selenia PMB

    23/23

    A comparison between objective and subjective image quality measurements for a FFDM system 2463

    Maidment A D and Albert M 2003 Conditioning data for calculation of the modulation transfer function Med.

    Phys. 30 24853

    Maidment A D A, Albert M, Bunch P C, Cunningham I A, Dobbins J T, Gagne R M, Nishikawa R M, Wagner R F

    and Van Metter R L 2003 Standardization of NPS measurement: Interim Report of AAPM TG16 Proc.

    SPIE 5030 52332

    Marshall N W, Kotre C J, Robson K J and Lecomber A R 2001 Receptor dose in digital fluorography: a comparison

    between theory and practice Phys. Med. Biol. 46 128396

    Metz C E, Wagner R F, Doi K, Brown D G, Nishikawa R M and Myers K J 1995 Toward consensus on quantitative

    assessment of medical imaging systems Med. Phys. 22 105761

    Moran P R 1982 A physical statistics theory for detectability of target signals in noisy images: I. Mathematical

    background, empirical review and development of theory Med. Phys. 9 40113

    Rogge F, Bosmans H, Willems P and Marchal G 2003 Use of MTF calculation in global and local resolution

    assessment in digital mammography Proc. SPIE 5030 896907

    Rose A 1948 The sensitivity performance of the human eye on an absolute scale J. Opt. Soc. Am. 38 196208

    Rowlands J A and Yorkston J 2000 Flat panel detectors for digital radiography Physics and Psychophysics Vol 1 of

    Handbook of Medical Imaging ed J Beutel, H L Kundel and R L van Metter (Bellingham: SPIE) pp 223328

    Samei E and Flynn M J 2003 An experimental comparison of detector performance for direct and indirect digital

    radiography systems Med. Phys. 30 60822

    Samei E, Flynn M J and Reimann D A 1998 A method for measuring the presampled MTF of digital radiographic

    systems using an edge test device Med. Phys. 25 10213

    Siewerdsen J H and Antonuk L E 1998 DQE and system optimization for indirect-detection flat-panel imagers in

    diagnostic radiology Proc. SPIE 3336 54655

    Siewerdsen J H, Antonuk L E, el-Mohri Y, Yorkston J, Huang W, Boudry J M and Cunningham I A 1997 Empirical

    and theoretical investigation of the noise performance of indirect detection, active matrix flat-panel imagers

    (AMFPIs) for diagnostic radiology Med. Phys. 24 7189

    Schnitzler A D 1973 Image-detector model and parameters of the human visual system J. Opt. Soc. Am 63 135768

    Sones R A and Barnes G T 1984 A method to measure the MTF of digital x-ray systems Med. Phys. 11 16671

    Stierstorfer K and Spahn M 1999 Self-normalizing method to measure the detective quantum efficiency of a wide

    range of x-ray detectors Med. Phys. 26 13129

    Wagner R F and Brown D G 1985 Unified SNR analysis of medical imaging systems Phys. Med. Biol. 30 489518

    Williams M B, Mangiafico P A and Simoni P U 1999 Noise power spectra of images from digital mammography

    detectors Med. Phys. 26 127993

    Workman A andCowen A R 1993Signal,noise andSNRtransfer propertiesof computedradiographyPhys. Med. Biol.

    38 1789808

    Yorker J G, Jeromin L S, Lee D L Y, Palecki E F, Golden K P and Jing Z 2002 Characterization of a full field digitalmammography detector based on direct x-ray conversion in selenium Proc. SPIE 4682 219

    Zhao W, Ji W G, Debrie A and Rowlands J A 2003 Imaging performance of amorphous selenium based flat-panel

    detectors for digital mammography: characterization of a small area prototype detectorMed. Phys. 30 25463

    http://dx.doi.org/10.1118/1.1534111http://dx.doi.org/10.1118/1.1534111http://dx.doi.org/10.1118/1.1534111http://dx.doi.org/full_texthttp://dx.doi.org/full_texthttp://dx.doi.org/10.1088/0031-9155/46/4/325http://dx.doi.org/10.1088/0031-9155/46/4/325http://dx.doi.org/10.1088/0031-9155/46/4/325http://dx.doi.org/10.1118/1.597511http://dx.doi.org/10.1118/1.597511http://dx.doi.org/10.1118/1.597511http://dx.doi.org/10.1118/1.595061http://dx.doi.org/10.1118/1.595061http://dx.doi.org/full_texthttp://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.1561285http://dx.doi.org/10.1118/1.1561285http://dx.doi.org/10.1118/1.1561285http://dx.doi.org/10.1118/1.598165http://dx.doi.org/10.1118/1.598165http://dx.doi.org/10.1118/1.598165http://dx.doi.org/full_texthttp://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.597919http://dx.doi.org/10.1118/1.597919http://dx.doi.org/10.1118/1.595493http://dx.doi.org/10.1118/1.595493http://dx.doi.org/10.1118/1.595493http://dx.doi.org/10.1118/1.598626http://dx.doi.org/10.1118/1.598626http://dx.doi.org/10.1118/1.598626http://dx.doi.org/10.1088/0031-9155/30/6/001http://dx.doi.org/10.1088/0031-9155/30/6/001http://dx.doi.org/10.1118/1.598623http://dx.doi.org/10.1118/1.598623http://dx.doi.org/10.1118/1.598623http://dx.doi.org/10.1088/0031-9155/38/12/007http://dx.doi.org/10.1088/0031-9155/38/12/007http://dx.doi.org/full_texthttp://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.1538233http://dx.doi.org/10.1118/1.1538233http://dx.doi.org/10.1118/1.1538233http://dx.doi.org/full_texthttp://dx.doi.org/10.1088/0031-9155/38/12/007http://dx.doi.org/10.1118/1.598623http://dx.doi.org/10.1088/0031-9155/30/6/001http://dx.doi.org/10.1118/1.598626http://dx.doi.org/10.1118/1.595493http://dx.doi.org/10.1118/1.597919http://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.598165http://dx.doi.org/10.1118/1.1561285http://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.595061http://dx.doi.org/10.1118/1.597511http://dx.doi.org/10.1088/0031-9155/46/4/325http://dx.doi.org/full_texthttp://dx.doi.org/10.1118/1.1534111