FSBPT uses multiple approaches to ensure the NPTE remains a fair and equitable tool to promote public protection. This article is based on a 2021 webinar presented by Lorin Mueller, Colleen Lettvin, Ashley Ray, Jenny Lapnawan, and Joanne Wagner.
The Federation of State Boards of Physical Therapy is charged with fulfilling its mission to protect the public by ensuring the ongoing excellence, reliability, defensibility, security, and validity of the national physical therapy examination or NPTE.
Passing the NPTE is a critical step in ensuring a candidate has the requisite knowledge to practice safely and effectively. There should be no barrier to any candidate taking the NPTE when considering factors they cannot control, such as race, ethnicity, gender assignment at birth, language, age, disability, or socioeconomic status.
FSBPT's job is to ensure that candidates have an unobstructed opportunity to demonstrate whether they have the knowledge and skill requirements to meet the minimum entry-level requirements to practice physical therapy safely and effectively. To do that, the NPTE and other assessments must contain questions that test only the knowledge of physical therapy practice and avoid US-centric jargon or language that would cause potential differential performance that may favor one candidate over another.
What terms might be unclear? For example, "football" means different things depending on location. Some think of it as a game in which an oval ball is moved forward by running or passing with the hands. Others defined football as a game with a round ball that may not be touched with the hands or arms. Therefore, describing a patient as having been injured playing football might confuse candidates from different parts of the world depending on their expectation
Ensuring fairness is a challenging process, and no one part of the process represents all the various perspectives at play. Accomplishing fairness requires attention throughout all stages of item development.
FSBPT has a rigorous item development process designed to support evidence of content validity at every stage of item development. FSBPT starts by recruiting a diverse group of licensed PTs and PTAs to volunteer to write items and serve on the FSBPT Examination Development Committees and other task forces and committees related to item writing. As a committee, FSBPT volunteers must be representative of the profession in terms of specialty, area of practice, practice setting, geographic location, gender, race and ethnicity, and other pertinent factors.
FSBPT employs a variety of recruitment efforts that include sending notices to specialty groups, advertising at conferences and workshops, and encouraging our existing volunteers, including volunteers who work with minority students, to recommend volunteers.
FSBPT provides formal training to our item writers, item writer coordinators, and committee members on an annual basis on how to ensure that items not only clearly reflect respect and sensitivity to all individuals but also reflect knowledge that is important to physical therapy. Questions must be linked to the NPTE content outlines, have only one unambiguously correct answer, and test knowledge that is relevant to public protection.
Training also includes how to help subject matter experts apply their unique content expertise to item writing and use terms that do not favor one candidate over another. An important aspect of training includes how to reduce construct irrelevant variance by eliminating extraneous or cueing content, for example, content in the question that suggests which answer is correct without requiring the knowledge that is being tested. FSBPT does this in myriad ways, including regularly sharing feedback on items that have been modified for fairness to groups reviewing items. As part of our ongoing training, FSBPT provides an item writing notebook that includes a checklist of qualifying criteria that reinforces how to minimize bias and maximize fairness across all items. FSBPT checks the items for fairness and for quality at multiple levels of item development. In addition to volunteers and committees, staff physical therapists verify the content of each item, and staff editors evaluate items for style and clarity.
The NPTE Style Manual enforces style conventions, which help provide a consistent testing experience for candidates. This allows them to focus on the content of the questions, rather than distracting or alarming details that might interfere with their thinking process.
For example, the NPTE Style Manual recommends the following:
Some of our preferred language is also intended to reduce bias in NPTE questions:
FSBPT also trains volunteers annually on how to use the bias and sensitivity scale, which assesses whether the item avoids language that may foster stereotypical, inappropriate, inflammatory, or derogatory terms towards racial, ethnic, gender, or any age groups. When in doubt, volunteers can flag items to undergo closer review by the Fairness Review Committee.
While FSBPT has many processes in place, it is important to test that these processes are, in fact, working. About twenty staff and volunteers and 300 candidates see an item before it becomes a scored item. Does that mean that someone will catch a problem? To answer this, FSBPT initiated a "secret shopper" experiment.
The Assessment Content Manager selected four item writing volunteers and asked them to intentionally submit problematic items under their names to their item writer coordinators. The experimental items included language that item writers, coordinators, assessment editors, and assessment content analysts were trained to identify and correct.
Only the volunteers and Fairness Review Committee were aware of the experiment. This helped create an environment where the Review Committee could fairly evaluate the quality. All four problematic items were identified and corrected before the items were approved. Based on the experiment's outcomes, FSBPT knew the training was working; problematic items were being caught.
FSBPT also surveys volunteer groups immediately following each meeting to collect information about the item and examination development process to help us internally identify and modify the way that FSBPT provides training and ongoing subject matter expert support.
FSBPT also aims to be fair in other aspects of the exam process. The Candidate Handbook and PEAT are both tools to help students prepare. The handbook offers clear, transparent advice and is freely accessible. PEAT is a practice exam simulating the NPTE and is available at a reasonable cost.
Finally, FSBPT also has external oversight. The Technical Advisory Panel, a group consisting of experts involved in academic or high-stakes examination programs, meets semiannually to provide insight and feedback. The Buros Center for Testing at the University of Nebraska at Lincoln holds FSBPT accountable by ensuring the organization meets accreditation standards. FSBPT collects candidate feedback in the post-administration survey and also seeks feedback from educators.
All of the current systems and checks can instill confidence, but that does not mean there is not more work to do. FSBPT recently engaged an external expert to help refresh the anti-bias tools. FSBPT continues to learn from educators, students, candidates, and other stakeholders and experts as they expand and build upon strategies to ensure fairness on the NPTE.
Jenny Lapnawan, PT, DPT, is an Assessment Content Analyst at the Federation of State Boards of Physical Therapy. Her primary duties include reviewing exam items and exam draft forms for the National Physical Therapy Examination and related assessments. Before joining FSBPT in 2018, Jenny was a Senior Physical Therapist at Houston Methodist Hospital and was previously a physical therapist at Florida Hospital Orlando. She currently works as a PRN physical therapist at Inova Fair Oaks Hospital in northern Virginia. Jenny graduated from the University of Florida Doctorate of Physical Therapy program in 2011 and is a Board-Certified Cardiovascular and Pulmonary Clinical Specialist.
Colleen Lettvin is the Assessment Content Manager at the Federation of State Boards of Physical Therapy. Colleen oversees the development of content for the National Physical Therapy Examination and other jurisprudence products such as Law examinations and Jurisprudence Assessment Modules. Colleen has been actively engaged in item and examination development as a long-term volunteer at FSBPT since 2005. Prior to joining FSBPT in 2016, Colleen was employed as a Senior Physical Therapist at Houston Methodist Hospital, where she played an integral role in developing the first American Physical Therapy Association (APTA)-accredited Critical Care Fellowship. Colleen has been a board-certified specialist in cardiovascular and pulmonary physical therapy by the American Board of Physical Therapy Specialties (ABPTS) of the APTA since 2010 and received her master of science in physical therapy from Texas Woman’s University.
Lorin Mueller, PhD, joined the Federation of State Boards of Physical Therapy (FSBPT) in 2011 as its Managing Director of Assessment. Prior to joining FSBPT, Lorin spent ten years as Principal Research Scientist at the American Institutes for Research in Washington, DC. Lorin received his PhD in industrial and organizational psychology with a specialization in statistics and measurement in 2002 from the University of Houston.
Ashley Ray, MA, is the Assistant Director of Exam Services at the Federation of State Boards of Physical Therapy. At FSBPT, her primary responsibilities include customer service, managing fixed-date exam administration tasks, and handling exam accommodations for candidates. Ashley joined FSBPT in 2011, after completing her undergraduate degree. Ashley received a Masters of Arts in Industrial and Organizational Psychology in 2015, with a concentration in Exam Development and Employee Training.
Joanne Wagner, MA, ELS, is Editorial Project Manager in the Assessment Department of the Federation of State Boards of Physical Therapy. She provides oversight of editorial work and editorial quality assurance for the NPTE and NPTE-related documents and is responsible for copyrighting processes for the NPTE. Prior to joining FSBPT, she was an editor of scientific journals and textbooks for a specialty medical association. She received certification as an editor from the Board of Editors in the Life Sciences.