Setting a Passing Score

Forum, Volume 21, Number 2

What Does the NPTE Passing Score Reflect?

Setting a passing score involves the process by which a performance standard is established. The passing score typically reflects the test score corresponding to a desired level of performance and is used for making decisions about what level of performance is high enough for a given purpose. The passing scores established for the National Physical Therapy Licensure Examinations (NPTE) reflect the level of performance required to provide minimally safe and competent physical therapy services by physical therapists and physical therapist assistants. Individuals scoring at or above the passing score have met the performance standard and are eligible for licensure, and individuals scoring below the passing score have failed to meet the performance standard and are not yet eligible for licensure. The term “passing score” is used interchangeably with the terms cut score and performance standard.

Passing Scores Do Not Exist Independently

There really is no passing score that exists independently. Rather, the passing score is the product of a deliberative process of the experience and the judgments of people who are qualified to make those judgments. Setting a passing score is a policy decision related to the mission of organization, in this case, to ensure that the public is protected from practitioners who are not competent. On the other hand, one must make certain the bar is not so high that it prohibits individuals who do have the necessary knowledge and skills from being able to enter the profession.

Numerous Steps are Taken to Ensure That the Score is Defensible

Establishing a performance standard is critical because of its impact on individuals, and numerous steps must be taken to ensure that the score is defensible. The Standards for Educational Psychological Testing contains criteria for establishing defensible passing scores, which have been endorsed by many organizations as the standard for practice both in terms of defensibility and best practices in our field. Included among the criteria are the qualifications of individuals participating in the setting of performance standards, the procedures by which performance standards are set, and the efforts made to validate the results of the standard setting process on an ongoing basis. All of these criteria combined are intended to ensure that the process is documented and defensible.

Establishing a passing score, including standards for passing the NPTE, involves six steps.

1. Deciding on a method that is appropriate for the test at hand.
2. Selecting qualified participants (individuals in the profession considered well qualified in their practice).
3. Training participants on the standard-setting method.
4. Providing feedback about participants’ judgments in carrying out the method.
5. Calculating a passing score.
6. Gathering validity evidence that is intended to bear on the question, “Do the folks who are awarded the license or the credential seem to be the kinds of folks who actually possess the kinds of characteristics important for practice?”

Choosing a method

Historically, the Federation has relied on criterion-referenced methods for setting passing scores since the early 1990s, particularly the modified-Angoff method.¹ This criterion-referenced method asks, “What are the levels of knowledge and skill that are required?” and sets the bar at that level. If every examinee had that level of knowledge or skill, the pass rate would be 100%.

The modified-Angoff technique is probably the most widely used method in health professions today, and it probably has the largest research base. Because of its ease of use and large research base, this method also is considered best practice.

Selecting participants

The next step is to select qualified participants who are representative of the profession. Participants are selected to be representative of the profession in terms of practice setting, specialty, geographic location, race and gender. Participants also are required to be knowledgeable about how the NPTE is developed.

Training Participants

The essential background required for the Angoff method is deliberation, discussion and formulation of a concept of a minimally competent or borderline level of knowledge or skill. For example, the statement reflecting minimal competence on the NPTE for physical therapists is “the minimal knowledge, judgment, technical, and interpersonal skills required to safely practice physical therapy.” It includes skills and knowledge on examination, evaluation, diagnoses, prognosis, intervention, and outcome assessment.

As part of their training, the panel is required to develop what is called a “key conceptualization” of the minimally qualified candidate – the hypothetical line between competent for entry into practice and not competent for entry into practice. What does that person look like? What can they do? What can they not do? Where is point you want to create a passing score between competent and not competent?

Rating Test Item Difficulty

After the panel is trained in using the modified-Angoff method and has developed key conceptualizations, panelists are asked to provide a rating of test items - the proportion of minimally competent examinees that would be expected to get that item correct. This judgment is made on each test question included on a test form.

Feedback

After initial ratings are made, panelists are encouraged to discuss their individual ratings. The raters consider the divergence, the heterogeneity, and the variability in their ratings, asking of themselves and other panelists: “Why did you rate it that way? Who gave this one such a high rating or a low rating? Why did that happen?” Through this discussion, panelists see if they are able to come to a consensus in their judgments. However, consensus isn’t required, it is just facilitated. In addition to feedback on other panelists' ratings, the panel may receive feedback on the proportion of minimally-competent test takers who answered each question correctly in the past, as well as the impact of these standards on the pass rate.

Setting New Scores in 2007

The passing score must be revisited periodically to ensure that it is responsive to changes in practice. At a minimum, the passing score is revisited whenever the scope of practice changes and the NPTE test is revised. For the NPTE, this process of reviewing the scope of entry-level practice, updating the test and revisiting the passing score occurs at least every five years. Passing scores were set for the NPTE in 2002 after the test had been updated as a result of a regular analysis of practice. The passing score for the NPTE for physical therapists was revisited in 2005 as a result of ongoing review.

The FSBPT is currently conducting an analysis of entry-level practice for physical therapists and physical therapist assistants, and the results of this analysis of practice will be used to update the test. New test forms will be assembled beginning in 2007 and new passing scores will be established in fall 2007.

After that process ends, the final passing score is recommended to the Federation Board of Directors. The Board of Directors will review the whole process, asking questions such as: Was the process implemented correctly? Was it done according to professional standards? Were there glitches? Were there modifications that were unexpected? After a thorough review, the Board of Directors will make its decision regarding the recommendation. If approved, it becomes the new passing score. Finally, new forms of the NPTE and new passing scores will be launched March 1, 2008.

¹In 2005, the Federation used an alternative criterion-referenced method to review the passing score for the NPTE for physical therapists called the Direct Consensus method.