Stakeholders should be aware of several important initiatives relating to the National Physical Therapy Examination (NPTE), including results from the Practice Analysis, progress towards new item types, refreshed content for the Practice Exam & Assessment Tool (PEAT), and other important updates. This article is based on a presentation by Lorin Mueller and Colleen Lettvin at the 2019 FSBPT Annual Meeting.
The National Physical Therapy Examination (NPTE) assesses basic, entry-level competence after graduation and is an important step in receiving a physical therapist or physical therapist assistant license. FSBPT staff and volunteers continually assess data, emerging trends, and additional factors when reviewing the content and format of the NPTE to ensure it is a strong tool for public protection. Several important initiatives were moved forward in 2019.
The Practice Analysis is a high-level description of what you have to know to be an entry-level physical therapist or physical therapist assistant. That knowledge translates directly into what’s on the NPTE.
FSBPT used to conduct the Practice Analysis every five years. However, the process was a massive undertaking, involving the creation of new task forces and procedures under a very tight timeline. It was also very expensive. Additionally, by only receiving data every five years, we were missing potential trends.
We re-examined the process and decided that overall we had a good framework, therefore, we could do away with extensive pre-planning through the Oversight Committee. We decided the first meetings of the Practice Analysis Task Force at each exam level weren’t necessary—the Exam Development Committee (EDC) could look at the previous survey and decide if any modifications needed to be made on a more regular basis and outside of a very tight timeline.
These changes also opened the door to conducting the analysis annually, which allows us a better view of the future. For example, if we know a category is becoming less and less relevant, we stop asking new item writers to create items in that category. While we’re not necessarily changing the content outline from year-to-year, the EDC has the ability to do that when needed as part of the NPTE Policies. We’ve just never given them regular, formal data on which to base their recommendations. We will still do a formal content outline update every five years—the next update will be in January 2023.
Amazingly, we’re able to do this yearly in a way that is cost-neutral. We are no longer flying in various task forces and we have a relationship with a contractor that ensures the process is efficient. Rather than re-inventing the wheel every five years, we’re simply maintaining a good design.
The 2019 Practice Analysis had a lot of interesting takeaways. Based on the results, the EDC is closely monitoring trends in the following areas:
The EDC is also re-examining our rules for references. Currently, we don’t allow references that are older than five years. However, the EDC would like to move toward a list of acceptable references instead of sticking with the hard five-year rule.
We’re in the process of creating two new item types: video and scenario-based items. Feedback from the 2016 practice analysis emphasized how much physical therapy was a movement-based profession, yet the test didn’t assess that. Video items are similar to the current graphic items. They are very brief, ten seconds or less. The candidate will need to access the video to answer the question.
Additionally, as health care technology improves, patients are living longer and they have more complicated medical conditions and histories. Entry-level physical therapists and physical therapist assistants will need to integrate several pieces of information in a short period of time and leverage clinical reasoning to make good decisions. We needed a good way to incorporate extended clinical scenarios that are more representative of these medically complex cases.
Scenario-based items use a template that includes the following pieces of information:
We’ve been working on both new item types since 2017. We tasked the Item Writing Task Force with drafting these item types in 2018. They’ve already created more than 300 video-based items and more than 800 scenario-based items that have been approved to move on to the EDC.
In terms of timing, we were originally thinking of introducing both the new items at the same time, but we’re moving away from that. One of the main challenges is Prometric’s exam delivery driver. They said videos aren’t a problem. However, surprisingly, scenario-based questions are. That said, they are working with us on solutions and we are hopeful that we can resolve the issues. However, as of now, we don’t have specific timing on when either of these new items types will be in effect, but we hope to have more definite information on timelines in late 2020.
Despite some challenges, we know we’re headed in the right direction. We’ve received really positive feedback from members and educators. The FSBPT Board of Directors is very supportive of this and has provided increased resources to make this happen. The volunteers, the lifeblood of our organization, have worked very hard to make these new item types a reality.
We use a retired form and a practice form with retired items for the Academic Practice Exam & Assessment Tool. Educators have recently come to us with concerns that people may be sharing passwords and that some items on the PEAT have been shared online. One way we have addressed this is by creating entirely new forms for the Academic PEAT. These new forms should be available in early 2020, if not sooner. We want to ensure that PEAT is an effective product for educators to adequately gauge what additional preparation, if any, their students need.
FSBPT proposed changes to the eligibility requirements for sitting for the NPTE. Specifically, we are looking at the requirements for candidates educated outside of the United States to have minimum scores on the Test of English as a Foreign Language and be evaluated on the latest version of the FSBPT Coursework Evaluation Tool. We are continuing to work with jurisdictions to improve consistency and we created a 2019 Resource Packet for Member Jurisdictions. (If you are a member, please contact us for a copy.) The packet details the rationale behind the new eligibility rules. Additionally, if your state is having an issue with one of the requirements, an FSBPT Board Member or staff can be available to help you through it.
FSBPT relies on a Technical Advisory Panel (TAP) with psychometric experts to provide us an outside view. In our most recent meeting, they suggested that we focus on getting more information about new item types and the NPTE research we’re doing out to the public. They emphasized we have some strong research, but we need to do a better job publicizing it.
Additionally, they suggested that we use average time spent responding to a question as an item review criterion. For example, if a question is taking a long time for a lot of candidates, it might just be confusing. Even though it might have good stats, we don't want a confusing item on the NPTE.
We are in “Stage 1” of a review by the Buros Institute for Assessment Consultation and Outreach. They look at the totality of what we do, from exam registrations to financial statements to test forms. We submitted hundreds of documents and they recently had a site visit. Overall they are very complimentary of the NPTE program—the leadership, staff, and volunteers. They noted we did well five years ago and have continued to improve. Similar to TAP, they also suggested we publicize more of our exam development efforts and research.
The NPTE is continuing to grow and evolve to ensure it is a strong tool for public protection. While there are many challenges ahead, our volunteers and members have been extremely supportive and continue to be the lifeblood of FSBPT.
Lorin Mueller, PhD
Managing Director of Assessment, Federation of State Boards of Physical Therapy
Lorin Mueller, PhD, is the Managing Director of Assessment at the Federation of State Boards of Physical Therapy with expertise in psychometrics and test development. Lorin oversees the development of the NPTE and other key assessment and research projects at FSBPT. Lorin has contributed his measurement expertise to such efforts as the development of high stakes tests for the selection of advanced mathematicians, medical personnel, elementary school teachers, air-traffic controllers, and human-resource professionals. Lorin previously worked at the American Institutes for Research (AIR) as a Principal Research Scientist, has served on several test development advisory boards, and is the current President of the Personnel Testing Council of Metro Washington.
Colleen Lettvin, PT
Assessment Content Manager, Federation of State Boards of Physical Therapy
Colleen Lettvin is the Assessment Content Manager at the Federation of State Boards of Physical Therapy. Colleen has been actively engaged in item and examination development as a long-term volunteer at FSBPT since 2005. Prior to joining FSBPT in 2016, Colleen was employed as senior physical therapist at Houston Methodist Hospital, where she played an integral role in developing the first American Physical Therapy Association (APTA)-accredited Critical Care Fellowship. Colleen has been a Board-Certified Specialist in Cardiovascular and Pulmonary Physical Therapy by the American Board of Physical Therapy Specialties (ABPTS) of the APTA since 2010 and received her Master of Science in Physical Therapy from Texas Woman’s University.