Psychometric Matters: Our Biggest (Unspoken) Fear, Part II

by Leon Gross

We all enjoy being right when we make predictions (as Yogi Berra said, “… especially about the future …”). This also applies to opining on unpopular or controversial issues. But sometimes we wish we were wrong, as I do regarding Part I of this column.

Part I lamented the organized memorization of test content for reproduction and distribution (MRD) as an insidious problem and the biggest fear of examining boards. The prescience of this concern was sustained by lawsuits filed since that publication involving the boards for three major health professions: podiatry, pharmacy, and of course, physical therapy. Although these lawsuits have distinguishing legal nuances, their fundamental commonality is the illegality of reproducing and distributing test items, and the resultant theft of the licensure boards’ intellectual property.

It is beyond the scope of this column to discuss the merits of these cases. What is relevant to this column is the close time proximity of these cases, suggesting not only high prevalence but also more seriously, a student culture that is unaware of or uninvolved with the corresponding ethical and legal implications of item MRD for public protection, and for the students themselves and their respective profession. As evidence of the public stake in this item MRD issue, the Pittsburgh Post-Gazette published a related article (Smydo, 2003), citing the public interest aspects.

So now what? With a student culture that values sharing and collaboration (much of which is good) in a broader Internet culture of global chat rooms, how can licensure boards meet this growing challenge? Here are some suggestions and a disclaimer: they do not come with a money-back guarantee.

With a student culture that values sharing and collaboration (much of which is good) in a broader Internet culture of global chat rooms, how can examining boards meet this growing challenge?

First, we need to recognize that as long as there are tests, students will have someone to demonize. With that label, ethics are often abandoned. The negative student perception of licensure tests is that they are obstacles to beginning one’s career. Boards need to articulate the positive aspect; that is, licensure tests provide a standardized, uniform opportunity to document entry-level competency, possibly with an expanded scope of practice. This function is the responsibility of licensure boards, not academic institutions. Regardless of the logic, this is the American model of education and licensure. Message to students: this is the way it is, and it is not going to change.

Another student attitude to be addressed is that item MRD is simply a prankish, non-malevolent activity. Here too, boards should clearly articulate that this is unacceptable, harmful to all parties, and consequential to participants. When a law is broken, “I didn’t know” is not a sufficient explanation, justification, or excuse.

Better dialogue is needed with academic institutions also. It is surprising how many faculties are unaware of the intellectual property issues. In fact, I have heard of situations in several professions in which students were asked by faculty to memorize items. To some extent, this is because faculties are sometimes evaluated by their students’ performance on board tests. Despite good intentions, the ethical and legal infringements of this activity must be addressed by boards and understood by academic institutions. In particular, an organized MRD effort with faculty involvement could taint the entire academic program. This is not an exaggeration -- one academic institution has already been named in the podiatry litigation.

This concludes the intellectual approach. A pragmatic one may resonate better. The most compelling reason for MRD is for examinees to capture items that may appear on a subsequent form of the test for which the examinees would have to sit if they fail their initial administration. What examinees probably do not know is that board item banks are many times larger than the number of items comprising a test. This size, plus the inclusion of new items (i.e., not previously used), significantly limits the number of repeat items from any one prior administration. Furthermore, the manner in which some boards “rest” items after prior use may actually preclude any items from the failed test being included on the next available 1-2 administrations.

To the extent that examinees are effective at item MRD, their efforts will help others, but not themselves. From a risk-reward perspective, the limited gain is hardly worth the consequences, regardless of the motivation.

FSBPT has forged a leadership role in coupling the seriousness of the crime with the consequences. The Federation has described item MRD as a form of cheating that compromises protection of the public, and is inconsistent with the trust bestowed on therapists for patient management. Therefore, in addition to monetary damages against item MRD participants, the Federation has announced that PT students who engage in item MRD are jeopardizing their ability to receive an academic degree, sit for the test, and become licensed. In essence, participants are risking their entire career as PTs.

These sanctions appear to be appropriately strong consequences and disincentives. I expect that with increasing awareness of the FSBPT lawsuit and outcome, these sanctions may be adopted by other federations of licensure boards. Vigorous protection of intellectual property is essential to the vitality of many industries such as pharmaceuticals, software, music, and now testing. Welcome to 21st century psychometrics.

References

Smydo, J. “Health fields fight cheating on tests,” Pittsburgh Post Gazette, August 3, 2003