AADE believes the use of multiple sources of information designed to ensure reliable and valid decisions (portfolios) is inconsistent with their own interests and previously stated positions. That is their prerogative, and no further comment is necessary. (A somewhat different account of the failed ITEM meetings intended to find common ground between the American Dental Education Association and AADE can be found in the article by Ranney and colleagues.1)
The further implication that the AADE position is psychometrically sound does, however, bear scrutiny. The officers of the AADE say it does, but they offer no evidence other than reference to their own policy. The position I developed in my JADA article, and in a recent California Dental Association Journal article,2 is that decisions about who should receive a license to practice dentistry must be made by an independent agency with state authorization, and that the decisions must be based on reliable and valid data.
The testing community is confused on what it means to be independent, and the "one-shot" test to which they are bound fails to reach conventionally accepted standards for reliability and validity.
In my JADA article, I demonstrated that one-shot initial licensure examinations have measured reliability in the neighborhood of r = .40. The standard for high-stakes tests such as professional licensure is normally r = .80 and higher. The foreshortened format prevents valid measures of the range of competence that constitutes the standards of care in dentistry, a shortcoming that would be exaggerated if simulations replaced treatment of patients. A representative portfolio of tasks and multiple testing is necessary to overcome these limitations.
The testing agencies have not challenged the data I have published. It is the practice in the medical testing community, in nursing, optometry and other health care professions for testing experts to publish data regarding psychometric evidence independently in the journals of their peers. That does not happen in dentistry, where it is common for untrained individuals with political positions to interpret selective data. They don’t always get it right, and there is not the same culture of transparency in dentistry as in other fields.
But of all the professions identified in this paragraph, and the pilots and plumbers mentioned by Dr. Cosby, only dentistry relies on the one-shot test. The examining community is right to reject portfolios as being inconsistent with the current one-shot testing format. They are wrong in implying that acceptable alternatives do not exist.
There is also confusion over independence in licensure. By history and in name selection, the distinction between state dental boards that make initial licensure decisions and testing agencies that provide data in partial support of those decisions has become obscure. Regional testing agencies and the testing committees in states with their own examination systems do not grant licenses. Legally, that responsibility belongs to an independent group known variously as state dental boards.
Boards make licensure decisions based on evidence provided to them from independent sources. They have the right to delegate data collection to any appropriate source, and they currently delegate to testing agencies, schools, the ADA testing services, state law enforcement agencies and others. Boards have an obligation to use only those sources that can guarantee reliable and valid data according to standards determined by the boards. Presently, the only source that provides data that are known to be of unacceptable reliability and validity are the independent testing agencies that rely on one-shot examinations.
I do agree with one point made by Dr. Crosby and others in the testing community. To bring the boards that they represent up to standards of reliability and validity would be prohibitively expensive. If other sources did not exist, such as schools that can provide multiple, long-term, comprehensive patient care evaluation experiences—and presumably could do so under state board supervision and to state board standards—dentistry would be faced with a moral dilemma. We would have Hobson’s choice of faulty affordable information, or sound but expensive data.
The reason AADE must reject the concept of portfolios stems from its financial commitment to the current testing format. AADE General Assembly Resolution 10GA correctly finds that one-shot tests cannot be modified to simultaneously balance financial soundness on the one hand, with independence, reliability and validity on the other.
TOP
REFERENCES
