Assessing the accuracy of caries diagnosis via radiograph
Film versus print
LINDA L. OTIS, D.D.S. and
ROBERT G. SHERMAN, D.M.D.
 |
ABSTRACT
|
|---|
Background. The authors conducted this study to determine if proximal caries diagnoses made using bitewing radiographic images printed on photographic paper were comparable with diagnoses made using traditional radiographic film images.
Methods. The authors digitized 15 posterior bitewing radiographs that contained 74 carious and 127 sound unrestored proximal surfaces and printed them on photographic paper. Fourteen dentists evaluated the radiographs and two printed image formats (4 x 3 centimeters and 8 x 6 cm) for evidence of caries. The diagnostic accuracy and interobserver agreement for caries diagnoses obtained in the two printed image formats were compared with those for radiographic film images.
Results. Overall, the diagnostic accuracy of printed images did not differ significantly from radiographic film images for dentinal caries. However, for caries limited to the enamel surface, a decrease in sensitivity was noted in six of the 14 observers for the smaller print images, while no significant differences in the diagnoses of enamel caries were observed among any of the observers in the enlarged print format.
Conclusion. This study provides evidence that printed images can be used to diagnose dental caries reliably.
Clinical Implications. The results of this study indicate that the diagnostic information obtained by viewing printed images is equivalent to that obtained by viewing standard radiographs. Size of the printed image also may be important in caries diagnosis and care must be taken to print bitewing radiographic images at a size that optimizes interpretation. Other factors that must be considered are the type of printer, printer resolution, paper quality and type of ink used. With careful consideration of printing parameters, clinicians can be assured of diagnostic quality in printed images.
Key Words: Caries diagnoses; copying processes; ink jet printer; paper; receiving operating characteristic; 
statistic
With the increased use of digital imaging in dentistry, the dilemma of how dentists communicate information that is stored in a computer between practices now is a reality. Published investigations to date have focused on evaluating the diagnostic accuracy of digital images as they appear on computer monitors, while little is known about the effects of printed media on dental diagnoses. As digital imaging becomes prominent in dental practice, it is likely that both initial diagnosis and archiving of patient records will occur in digital format. Despite a transition to digital format, the necessity for hard copy images will remain. Printed images are a convenient format for transmitting diagnostic information to referral specialist practices and to provide documentation to third-party payers. Clinical studies that examine the diagnostic quality of printed images can provide important information to practitioners with regard to optimizing print formats for specific clinical tasks. Magnification and image size have been shown to be important factors that may influence caries diagnosis in bitewing radiographic images.1–4 Consequently, we investigated the diagnostic impact of two sizes of printed images as compared with standard radiographs for the detection of proximal caries.
Bitewing radiographic images printed on photographic paper in an easy-to-read size can be used to diagnose proximal caries.
|
MATERIALS AND METHODS
|
|---|
Images.
Fifteen bitewing radiographs were selected from patient records to represent a broad range of caries activity from incipient to extensive lesions. The inclusion criteria we used to select radiographs included optimal density and contrast, no overlapping contacts and no more than one missing tooth in the bitewing radiographic image. We did not include occlusal surfaces in this study, and we excluded 21 proximal surfaces because they had been restored previously. We evaluated 201 nonrestored proximal surfaces in 15 bitewing radiographic images. We converted the radiographs into digital format using a matrix charge couple device line transparency scanner (xenon cold cathode fluorescent lamp), employing 8 bits per pixel to produce images with an optical density range of 3.6.
The authors converted the radiographs into digital format using a matrix charge couple device line transparency scanner.
Digital images were stored at 300 pixels per inch with no enlargement (1:1) (4 x 3 centimeters) and at 300 ppi at 1:4 enlargement (8 x 6 cm). We made the prints on medical imaging paper (no. 132-4169, Eastman Kodak, Rochester, N.Y.) at 600 dots per inch using a dedicated medical ink jet printer (Model 1200, Distributed Medical Imager, Eastman Kodak). We have shown in a previous study that a scanned resolution of 300 ppi retains the diagnostic information in a bitewing radiograph as viewed on a standard monitor; a printer resolution of 600 dpi is the recommended printer setting for a standard, high-quality print stored at this resolution.5
We asked 14 dentists to diagnose caries in each of the image formats using the following five-point scale: 0 = no radiographic caries evident; 1 = caries limited to the enamel, 2 = caries penetrating less than one-half the dentin surface, 3 = caries penetrating more than one-half the dentin surface, and 4 = caries extending to the pulp. The dentists evaluated the radiographs (size 2 Ektaspeed, Eastman Kodak), small (1:1) prints and enlarged (1:4) prints following randomized block design.
Observers.
Of the 14 dentists who acted as observers in this study, two were board certified in public health, three in oral medicine and six in advanced general dentistry, and three had completed residencies in oral medicine. The dentists’ practice experience ranged from six to 26 years, with a mean of 18.4 years.
Viewing conditions.
The observers completed a diagnostic evaluation of the prints and radiographs after receiving a brief explanation of the study and data collection forms. The viewing conditions were standardized for each observer in a quiet room free from distractions. For radiographs, the observers used a light box in which extraneous light was masked. Each observer had the option of using a magnification lens (X-produkter, Malmö, Sweden) to view the radiographs. The image order was randomized for each block of images (small prints, enlarged prints and radiographs), and to minimize learning effects, the observers received the three image formats in random order.
Statistical analysis.
statistics have been accepted as the test of choice to assess interobserver consistency for radiographic caries diagnosis.6,7 In our study, we calculated weighted statistics using a software program (Medicalc, Version 6, Frank Schoonjans, Mariakeke, Belgium). Receiver operating characteristic (ROC) is used widely to assess the performance of radiologists or to compare the performance of different imaging modalities.8 The area under the ROC curve represents the diagnostic accuracy of the imaging system and simultaneously evaluates sensitivity and specificity.
In our study, we calculated diagnostic accuracy for each observer using a consensus decision as the gold standard. A panel of three radiologists who did not participate in the study determined this consensus standard. Of the 201 surfaces, only four had equivocal findings among the 14 observers; seven observers reported the surfaces as positive, and seven reported the surfaces as negative. The consensus standard established that 127 surfaces had no cariess, 37 had lesions limited to the enamel, 26 had lesions limited to the outer one-half of the dentin surface, nine had lesions extending beyond one-half of the dentin surface, and two had lesions extending to the pulp.
We derived areas under the ROC curves for the 14 dentists using a nonparametric method. We constructed ROC curves similarly for each image format, pooling data for the 14 observers. We tested the differences between the areas under the pooled ROC curves for each image format using a z-score test for correlated data.8
|
RESULTS
|
|---|
Fourteen dentists evaluated 201 nonrestored proximal surfaces (74 carious, 127 noncarious) in 15 bitewing radiographic images. The pooled sensitivity and specificity for the three image formats are given in Table 1
.
View this table:
[in this window]
[in a new window]
|
TABLE 1 POOLED SENSITIVITY AND SPECIFICITY VALUES FOR PROXIMAL CARIES DIAGNOSES BY THE 14 OBSERVERS FOR THE THREE IMAGE FORMATS.
| |
Interrater agreement.
We calculated inter-rater agreement for each image format using the weighted statistic (Table 2). The four agreement groups were < 0.41, poor agreement; = 0.41–0.60, moderate agreement; = 0.61–0.80, substan tial agreement; and = 0.81–1.0, near perfect agreement.9 Overall, the mean values for each image format were similar and ranged from moderate to substantial agreement. The mean interobserver agreement for radiographs was 0.66 (± 0.096 standard deviation [SD]; range, 0.41–0.82), for the enlarged prints, it was 0.69 (± 0.063 SD; range, 0.52–0.85), and for the small prints, it was 0.66 (± 0.097 SD; range, 0.40–0.85). A t test of the mean values revealed no significant differences among the three image formats. However, when we categorized the data into four groups, a Fisher exact test revealed proportionally more values in the "substantial agreement" category for the enlarged print format compared with radiographs and small prints (P = .0013) (Figure 1, page 327).
To make a claim of diagnostic equivalence, it is necessary to investigate not only the accuracy achieved with the image format but also the effect of the image format on interobserver variation.
ROC analysis.
We determined diagnostic accuracy using a nonparametric ROC analysis. Tables 3 (page 327) and 4 (page 328) show the area under the curve for ROC analysis for individual observers. Six of the 14 observers exhibited a significantly poorer performance with the smaller print, compared with the enlarged prints and the radiographs. When we pooled the ROC data for all of the observers, we found that the area under the ROC curve for the small prints was significantly different from that for the enlarged prints and radiographs when the ROC threshold was set at 1 (enamel caries) (Figure 2, page 328). When we set the ROC threshold at 2 (caries penetrating less than one-half the dentin surface), we found no significant differences among the three image formats (Figure 3, page 329).
View this table:
[in this window]
[in a new window]
|
TABLE 3 COMPARISON OF THE AREA UNDER THE ROC* CURVE FOR 14 OBSERVERS FOR ENAMEL CARIES (THRESHOLD SET AT ‘1’ ) AS VISUALIZED IN THREE IMAGE FORMATS.
| |
View this table:
[in this window]
[in a new window]
|
TABLE 4 COMPARISON OF THE AREA UNDER THE ROC* CURVE FOR 14 OBSERVERS FOR DENTINAL CARIES (THRESHOLD SET AT ‘2’) AS VISUALIZED IN THREE IMAGE FORMATS.
| |
View larger version (54K):
[in this window]
[in a new window]
|
Figure 2. Receiver operating characteristic curves for the 14 observers (pooled data), depicting the performance of radiographs, enlarged prints and small prints for the detection of proximal caries. The diagnostic threshold was set at 1, "enamel caries."
| |
View larger version (53K):
[in this window]
[in a new window]
|
Figure 3. Receiver operating characteristic curves for the 14 observers (pooled data), depicting the performance of radiographs, enlarged prints and small prints for the detection of proximal caries. The diagnostic threshold was set at 2, "dentinal caries."
| |
|
DISCUSSION
|
|---|
Bitewing radiographs are used routinely for diagnostic assessment of proximal caries. Studies have shown that various digital modalities are diagnostically comparable with Ektaspeed film for approximal caries detection.3,10–15 The emphasis of these studies was to establish diagnostic equivalence using monitor-based diagnosis. It is likely that with increased incorporation of direct digital technology into dental practice, monitor-based diagnoses will replace the traditional film-based approach.16,17 In the initial transition, digital record transmission between dental practices may be difficult. Some type of hard copy record will be required in locations where access to a computer and monitor is limited. Images printed using a proven ink jet system offer a low-cost solution for transferring digital diagnostic information or for providing an interim record of dental health status.18
Our primary objective in conducting this study was to determine if bitewing radiographic images printed on photographic paper are of sufficient quality to diagnose proximal caries. It has been shown that as much as 75 percent of the variance in diagnostic accuracy may depend on observer variability.19 Thus, to make a claim of diagnostic equivalence, it is necessary to investigate not only the accuracy achieved with the three image formats but also the effect of the image format on interobserver variation as we did in this study. An independent measure of the condition under study is necessary to establish the validity of a diagnostic method conclusively. A limitation of this study was that we used clinical radiographs and consequently could not histologically verify the presence of caries. This limitation was offset by the fact that our study was approximate to the clinical question of whether a printed image contains the same diagnostic information as a radiograph. We believe that the consensus standard used for validation was sufficient to test this hypothesis.
The primary difference we found among the three image formats was a lower sensitivity for the detection of enamel caries in the small print format. A diagnostic method that favors specificity at the expense of sensitivity is desired for caries diagnosis because the clinical outcome of a false positive diagnosis is the unnecessary restoration of a sound tooth. Because sensitivity was lowered and specificity was increased, and because this increase in false negatives was limited to enamel lesions, it can be argued that even the small prints may be acceptable for many purposes when a hard copy image is needed. Our findings are consistent with other published studies that report a similar higher level of disagreement and decreased diagnostic accuracy for enamel lesions.4,20,21
It is significant that when we set the ROC diagnostic threshold at 2 (dentinal caries), diagnostic accuracy was indistinguishable among the three image formats. Dentinal caries represents the threshold at which restorative decisions are critical, particularly in populations with high caries prevalence. On the other hand, detecting enamel lesions is important because small noncavitated lesions respond best to remineralization therapy and preventive strategies. It also is critical to identify small enamel lesions so that progression can be monitored because documenting lesion progression in a person at high risk of developing caries may be the best way to decide whether to restore a tooth.22,23
The enlarged prints and radiographs were clearly diagnostically indistinguishable in the ROC analysis. The decreased sensitivity of small prints to detect enamel caries and the slight improvement in inter-rater reliability in the enlarged prints suggest that printed image size is a critical technique factor for diagnostic quality in printed bitewing radiographic images. These findings are in agreement with another study that has shown that some enlargement in monitor-based images improved diagnostic sensitivity, but that too great an enlargement significantly increased the number of false positives.2 Because we found that enlarged prints (1:4) were diagnostically equivalent and that the detriment in performance of the small prints (1:1) was limited to enamel lesions in only six of the 14 observers, we believe that a convenient print size between 1:1 and 1:4, using the print parameters and paper used in our study, would produce prints of acceptable diagnostic quality.
|
CONCLUSIONS
|
|---|
Bitewing radiographic images printed on photographic paper in an easy-to-read size can be used to diagnose proximal caries. Printed images can be viewed with reflected light and as such have the advantage of being studied anywhere without the need for a light box or computer monitor. The disadvantage of a paper format is that the density of printed images may fade over time, particularly when exposed to light. The low cost of prints produced with an ink jet system makes it easy to print a new image when needed. The selection of a printer that is designed to print a full range of densities is critical, and it is important to adjust the printer settings to correspond to the steepest part of the densitometric curve.24 Similarly, choosing photographic paper that has the capacity to consistently display the full density range and has good archival characteristics is equally important.
Additional studies are needed to delineate the many technical factors that contribute to the diagnostic quality of printed formats. Investigations aimed at developing optimal printer parameters and verifying the archival characteristics of various printers and photographic papers represent important additional steps toward characterizing the quality of printed intraoral images.
|
FOOTNOTES
|
|---|
Dr. Otis is the director, Oral and Maxillofacial Radiology, University of Pennsylvania, The Robert Schattner Dental Center, 240 South 40th St., Philadelphia, Pa. 19104-6030. Address reprint requests to Dr. Otis.
Dr. Sherman is the director, Navy Dental Clinic, and the head, Oral Diagnosis Department, Naval Health Center, Pearl Harbor, Hawaii.
The authors wish to thank E. Paul Wileyto, Ph.D., for assistance in the statistical analysis, and the United States Navy Dental Corps officers who participated as observers in this study.
The opinions expressed in this article do not represent the views of the United States Department of Defense, the Department of the Navy or the United States Navy Dental Corps. Use of any commercial products in this study does not imply endorsement by the U.S. Government.
|
REFERENCES
|
|---|
- Arnold LV. The radiographic detection of initial carious lesions on the proximal surfaces of teeth, Part II: the influence of viewing conditions. Oral Surg Oral Med Oral Pathol 1987;64(2):232–40.[Medline]
- Moystad A, Svanaes DB, Larheim TA, Grondahl HG. Effect of image magnification of digitized bitewing radiographs on approximal caries detection: an in vitro study. Dentomaxillofac Radiol 1995;24(4):255–9.[Abstract]
- Svanaes DB, Moystad A, Risnes S, Larheim TA, Grondahl HG. Intraoral storage phosphor radiography for approximal caries detection and effect of image magnification: comparison with conventional radiography. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 1996;82(1):94–100.[Medline]
- Versteeg CH, Sanderink GC, Lobach SR, van der Stelt PF. Reduction in size of digital images: does it lead to less detectability or loss of diagnostic information? Dentomaxillofac Radiol 1998;27(2):93–6.[Abstract]
- Navarro M, Otis L. Video resolution requirements for novice observers for the diagnosis of dental-caries (abstract 1052). J Dent Res 1995;74:143.
- Fleiss JS. Statistical methods for rates and proportions. 2nd ed. New York: Wiley; 1981.
- Fleiss JL, Fischman SL, Chilton NW, Park MH. Reliability of discrete measurements in caries trials. Caries Res 1979;13(1):23–31.[Medline]
- Hanley JA, McNeil BJ. A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology 1983;148(3):839–43.[Abstract/Free Full Text]
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33(1):159–74.[Medline]
- Abreu M Jr, Mol A, Ludlow JB. Performance of RVGui sensor and Kodak Ektaspeed Plus film for proximal caries detection. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 2001;91(3):381–5.[Medline]
- Haak R, Wicht MJ, Noack MJ. Conventional, digital and contrast-enhanced bitewing radiographs in the decision to restore approximal carious lesions. Caries Res 2001;35(3):193–9.[Medline]
- Hintze H, Wenzel A, Frydenberg M. Accuracy of caries detection with four storage phosphor systems and E-speed radiographs. Dentomaxillofac Radiol 2002;31(3):170–5.[Abstract]
- Hintze H, Wenzel A. Influence of the validation method on diagnostic accuracy for caries: a comparison of six digital and two conventional radiographic systems. Dentomaxillofac Radiol 2002;31(1):44–9.[Abstract]
- Nair MK, Nair UP. An in-vitro evaluation of Kodak Insight and Ektaspeed Plus film with a CMOS detector for natural proximal caries: ROC analysis. Caries Res 2001;35(5):354–9.[Medline]
- White SC, Yoon DC. Comparative performance of digital and conventional images for detecting proximal surface caries. Dentomaxillofac Radiol 1997;26(1):32–8.[Abstract]
- Cederberg RA, Frederiksen NL, Benson BW, Shulman JD. Effect of different background lighting conditions on diagnostic performance of digital and film images. Dentomaxillofac Radiol 1998;27(5):293–7.[Abstract]
- Cederberg RA, Frederiksen NL, Benson BW, Shulman JD. Influence of the digital image display monitor on observer performance. Dentomaxillofac Radiol 1999;28(4):203–7.[Abstract]
- Lyttkens K, Kirkhorn T, Kehler M, et al. Evaluation of the image quality of ink-jet printed paper copies of digital chest radiographs as compared with film: a receiver operating characteristic study. J Digit Imaging 1994;7(2):61–8.[Medline]
- Svenson B, Welander U, Anneroth G, Soderfeldt B. Exposure parameters and their effects on diagnostic accuracy. Oral Surg Oral Med Oral Pathol 1994;78(4):544–50.[Medline]
- Svanaes DB, Moystad A, Larheim TA. Approximal caries depth assessment with storage phosphor versus film radiography: evaluation of the caries-specific Oslo enhancement procedure. Caries Res 2000;34(6):448–53.[Medline]
- Hintze H, Wenzel A, Jones C. In vitro comparison of D- and E-speed film radiography, RVG, and visualix digital radiography for the detection of enamel approximal and dentinal occlusal caries lesions. Caries Res 1994;28(5):363–7.[Medline]
- Verdonschot EH, Angmar-Mansson B, ten Bosch JJ, et al. Developments in caries diagnosis and their relationship to treatment decisions and quality of care: ORCA Saturday Afternoon Symposium 1997. Caries Res 1999;33(1):32–40.[Medline]
- Pitts NB. Diagnostic tools and measurements: impact on appropriate care. Community Dent Oral Epidemiol 1997;25(1):24–35.[Medline]
- Stephenson TF, Lincoln AJ, Mehnert PJ, Paul GJ. Comparison of X-ray film and photographic paper in recording CT images. J Comput Assist Tomogr 1984;8(6):1161–3.[Medline]
TOP
ABSTRACT
MATERIALS AND METHODS
