THE COURSE

TOP ABSTRACT THE NORTHWEST PRECEDENT NETWORK THE TRAINING PROGRAM THE COURSE ASSESSING THE COURSE CONCLUSIONS REFERENCES

 
We divide the course into four components to address the following fundamental issues:

– What is the question?

– What are the options?

– How do you evaluate the evidence?

– How do you conduct a study?

What is the question? One of the most fundamental issues in conducting good clinical research is understanding how to frame the research question of interest appropriately, and how then to translate it into a testable hypothesis. To aid the practicing dentist in this endeavor, we introduce the use of the PICO method. This terminology emphasizes identifying the patient’s or population’s (P) characteristics of interest, the intervention (I), the control or comparison (C) group and the outcome (O) on which comparisons will be based.

The PICO method is particularly helpful in framing questions that involve treatment interventions. For example, a clinician might suggest the following as a potential topic for a clinical research project: does regular use of chlorhexidine rinse prevent tooth loss in patients with periodontitis who are 65 years and older? Examination of the question using the PICO terminology reveals that the patient’s or population’s (P) characteristics of interest are specified (patients with periodontitis 65 years and older), the intervention (I) is specified (regular use of chlorhexidine rinse) and the outcome (O) is specified (tooth loss). However, there is nothing in the suggested research topic that specifies the control or comparison (C) group. The control group could be rinsing with fluoride, a placebo rinse with a taste similar to that of chlorhexidine or an antiseptic mouthrinse; alternatively, it could be using no rinse at all. Specification of what is intended for a control group is important in the design and analysis of the clinical questions, and using the PICO guidelines forces the clinician to be specific in the phrasing of the clinical question. This suggests that to frame the research question properly, one also must specify a control or comparison group with which to compare patients who use chlorhexidine regularly.

A second topic discussed under this first category is the role that behavior often plays in many oral health research questions. A patient is not merely a mouth full of teeth, and the practitioner-investigator is not merely treating or studying a biological event. Rather, a patient or participant is subject to many psychosocial and cultural influences that may affect his or her oral health. Acknowledging that cognition, emotion, temperament and motivation play roles in a person’s behavior that, in turn, affect his or her oral health is important in identifying the exact nature of the most relevant question. Distinguishing between disease as a biological event and illness as a subjective experience also may help in framing the appropriate question. Furthermore, psychosocial issues may be confounding risk factors or outcome variables. As practitioner-investigators may choose to include psychosocial variables in their study designs, we provide training in the selection and use of reliable and valid measures for their use in research studies.

What are the options? During this section of the program, we discuss the different kinds of clinical research study designs and the levels of evidence produced by each. In a given situation, it is desirable to identify the study design that produces the highest level of evidence and also is feasible to conduct. The hierarchy of evidence from study designs proceeds thus, from lowest to highest level of evidence: single case reports, case series, case-control studies, cohort studies, randomized controlled trials (RCTs). The study design that produces the highest level of evidence, the RCT, may not always be feasible. For example, if one is interested in testing whether the efficacy of a specific type of periodontal treatment is affected by smoking, an RCT in which patients are randomized to smoke or not smoke is not feasible or ethical. One has to select from among the study designs that are feasible the kind that will yield the most and highest level of information.

How do you evaluate the evidence? While practitioner-investigators in practice-based research networks do not have to function as bio-statisticians for their studies, they must understand the basic terminology and principles used in designing and analyzing studies so they can understand and help interpret the results. Thus, we try to ensure that they understand the basic concepts of confidence intervals and tests of hypotheses. We use inherent variability in disease measures as the underpinning for developing a confidence interval that expresses the amount of precision surrounding an estimated value. To explain statistical hypothesis testing, we draw an analogy between it and the legal system. In statistical hypothesis testing, a stated null hypothesis is assumed to be true, and the evidence from the study is evaluated by calculating the probability of achieving a result like that observed under that assumption. Similarly, in the legal system, a defendant is assumed to be innocent and the surrounding evidence is evaluated as to whether it contradicts that assumption beyond a reasonable doubt.⁶ We also use this analogy to introduce the concepts of type I error (convicting an innocent person) and type II error (letting a guilty person go free), as well as power (the probability of convicting a guilty person). Translated into hypothesis-testing terminology, this allows a discussion of how the sample size for a given study is determined by the size of the treatment effect one is trying to detect, the power one wishes to have for detecting that effect, the inherent variability in the measure one is using and the significance level (or type I error rate—the probability of incorrectly concluding that there is a treatment effect) one uses in the hypothesis test. Thus, the practitioners gain an appreciation of how all of these factors can influence the sample size required for a study. It also provides a basis for how the analysis of the data results in a conclusion, the underlying certainty of that conclusion and the difference between a conclusion that is statistically significant and one that implies clinical significance.

How do you conduct a study? The final didactic section of the training program consists of practical discussions of several topics that can influence the quality of a study and that, therefore, deserve special emphasis. These important topics include bias (what it is and what causes it), enrollment (which often is overlooked in importance and effect), methods of treatment assignment, patients’ adherence to the treatment protocol, measurement of the outcome of interest, attrition of participants as the study progresses, and the principles and process of obtaining informed consent. For each of these topics, we provide examples of how studies can become biased through a lack of attention to some of these details. A summary of how to perform a study and avoid introduction of bias includes the following advice:

– follow the enrollment protocol;

– establish a careful informed consent process;

– randomize treatment assignments (if appropriate);

– maximize patients’ adherence to assigned treatments;

– increase the accuracy of the outcome assessment;

– mask the treatment assignments when feasible;

– minimize attrition of patients.

After these discussions, we hold a general brainstorming session concerning topics for studies that practitioners have suggested when they signed up on our Web site. We address topics that were mentioned most often. We also encourage the practitioner-investigators to use PICO terminology to frame the appropriate question and to suggest an appropriate study design to address the question. This discussion reinforces the topics just covered and allows the practitioner-investigators to provide input on priorities for studies to be conducted in the network.

	ASSESSING THE COURSE

TOP ABSTRACT THE NORTHWEST PRECEDENT NETWORK THE TRAINING PROGRAM THE COURSE ASSESSING THE COURSE CONCLUSIONS REFERENCES

 
We presented the course twice as day-long conferences in early 2006, with a total of 98 dentists in attendance. During the second presentation of the course, the lectures were recorded, and the slides were incorporated into handouts. Now, as new members of the PRECEDENT network are identified, they receive a DVD version of the lectures (approximately 4.5 hours long) as well as a notebook with copies of all slides, and we ask them to view the lectures while reviewing the handouts. After they complete the training session, we ask them to attend a two-hour meeting with network investigators so they can ask any questions about the material and provide input on study topics, much as in the brainstorming sessions we hold at the end of the daylong conferences.

Rationale for the training program. It is reasonable to ask if this kind of training is really necessary to involve practicing dentists in practice-based research. One argument is that a one-day training program such as this will not provide adequate background for the practitioners to function as independent clinical researchers, and all that is needed for a specific protocol is to provide explicit instructions to practitioners regarding how to carry it out. It is our belief, however, that no protocol ever can be explicit enough to cover every scenario that might arise in carrying out a study, and at some point a practitioner will face a dilemma regarding what to do in a study for a situation not addressed in the manual of operations. When that occurs, we believe that someone who has been exposed to and understands the general underlying principles on which the research project is based, will not approach the dilemma as an automaton but, rather, will have improved judgment and be less likely to do something that threatens the integrity of the study.

An additional argument for requiring this kind of training before participation in practice-based research is that for a network to be successful, its members should feel a certain amount of ownership. If members understand the underlying principles being used to identify research topics, design the studies and analyze and interpret the results, then they are more capable of playing an active role in making decisions regarding network activities. By not simply treating practitioners as technicians, we reinforce a more collaborative role for them and, we hope, ensure increased satisfaction among, and future participation by, practitioner-investigators.

Practitioners also need to identify and appreciate any kind of adaptations they may need to make to incorporate research activities into their daily practice routines with minimal disruption. Understanding the principles covered in this training will help them do this in advance.

Finally, exposure to these research principles should encourage practitioners to use evidence-based principles routinely in their practice, even when they are not involved in a specific research project.

Evaluation of the program by practitioners. As stated earlier, we have presented this training program twice as one-day conferences, and several times using the DVD format with follow-up discussion meetings. At the conclusion of each of the conferences or meetings, we asked participants to evaluate the content of the training program by indicating whether they agreed or disagreed with four statements that suggest the potential impact of the course on future practice-based research. The table provides a summary of the responses from all of those who participated in either conference or DVD/meeting format.

Neither mode of presentation (in-person versus DVD) of the program was clearly superior to the other, as evidenced by the responses given. While we expected the in-person presentations to be more engaging, we found that several of those who used the DVD presentation expressed appreciation for the ability to return to and review some topics that were more challenging. In our discussions with practitioners at the completion of the two types of program, we found no discernible differences in their understanding and application of the research principles.

In the evaluations provided after completion of the in-person and self-paced DVD training, we asked participants for additional comments. Aside from specific comments and suggestions for individual speakers, practitioners’ general comments indicated that although the material could be somewhat foreign to them, they understood and appreciated the necessity of this training in enabling them to conduct research in their offices. Some typical comments were as follows:

– "Well done for a topic that can be dry, although I find it very interesting."

– "The course was worthwhile and necessary. Good overview of not that interesting a subject." In addition, many commented on how acquiring this new knowledge has increased their enthusiasm for creating evidence for the practice of evidence-based dentistry:

– "I feel very excited to participate in this type of research group."

– "Looks like a fun and satisfying new challenge [for] a 25-year wet-fingered dentist looking to return something to the profession, with a possible benefit to others in real life and in research."

	CONCLUSIONS

TOP ABSTRACT THE NORTHWEST PRECEDENT NETWORK THE TRAINING PROGRAM THE COURSE ASSESSING THE COURSE CONCLUSIONS REFERENCES

 
Practicing dentists who are interested in engaging in practice-based research appreciate the value of receiving training in research methodology that will better prepare them to conduct clinical research studies in their own offices. While listening to lectures about research methodology may not be as exciting as a hands-on clinical procedure course, these dentists generally see the need for this kind of orientation to help them conduct quality studies in their practices. A full one-day program focusing on research principles, followed by a discussion of how those principles apply to relevant studies, seems to solidify their interest in practice-based research.

Providing this background training already has paid dividends as far as obtaining practitioners’ input into identifying relevant clinical research topics and designing studies to address them. Investigators at the UW and OSHU proposed the first PRECEDENT study, a survey of the oral health status of a random sample of 20 patients from each participating practice, to help in future study planning. However, most subsequently identified study topics have been initiated by the practitioner members, and all have benefited from assessments by PRECEDENT practitioners, who rated them for relevance and made suggestions as to appropriate study designs. These studies (either under way, under development or approved for future development) include the evaluation of salivary characteristics to identify patients at high risk of experiencing caries, the treatments for and outcomes of cracked teeth, the comparison of mineral trioxide aggregate with calcium hydroxide in long-term outcomes of direct pulp caps, outcomes of and risk of complications resulting from third-molar extractions, and prevalence and treatment of chronic tooth sensitivity.

As we develop new studies, we will offer the opportunity for additional protocol-specific technical training to implement them. However, our emphasis is on developing studies that are simple to design and conduct, which should minimize the need for additional training and calibration. The mode of delivering additional protocol-specific training likely will vary with different studies; it could include Web-based videos as well as brochures with color illustrations. We have used the PRECEDENT annual meeting to introduce new studies and provide technical training in the use of the Web-based data-capture system, and we likely will continue to use that forum for additional study-specific training.

The training done so far has resulted in the collection of high-quality data in the first survey of dental patients. As of Jan. 15, 2008, we had collected data regarding approximately 1,432 patients from 75 dental practices, with few indications of missing, incomplete or invalid data. The high acceptance rate and overall positive evaluation of the training program by the member practitioner-investigators in conference and DVD format suggest that we should be able to continue using it for future members as they join the network. All indications are that the investment of our time in developing this course and of the members’ time in taking this training will continue to pay dividends in future years.