The questions in this section were compiled from the “This Quarter’s Question” in the Quarterly Faculty Newsletter, which is distributed via email to all PT and PTA faculty who request a copy.
I have heard that the questions on the NPTE take a long time to develop and validate. Since the profession of physical therapy is rapidly changing and is becoming much more evidence based, how do you assure that the questions on the exam are current?
It is critical that a high stakes licensing examination covers current practice. While it is a challenge to maintain this currency, the item writers and exam committees work very hard to assure the exam reflects current practice. Currency is maintained via the following steps:
The exam blueprint or content outline is revised at a minimum of every five years. The exam blueprint determines the content of the exam and assures that the content is relevant to current practice.
Currency is stressed at item-writer workshops; item writers are required to reference questions to recognized authoritative texts that have been published within the past five years.
Items are not used where there may be conflicting references or viewpoints in the literature
An item goes through committee review multiple times and each time, it is reviewed for currency.
The item goes through a final review for currency (among other things) prior to each and every time it is being released as an operational question.
PT Passing Standard Review
In 2005, the Federation conducted a Standard Review for the PT exam with a resulting adjustment in the passing standard (passing score). Why did the Federation not conduct a standard review for the PTA exam?
It is the Federation of State Board of Physical Therapy Board of Director’s responsibility to assure that the quality and integrity of the NPTE is always maintained. In doing this, there are many quality assurance processes and procedures that are continually being conducted. In reviewing some of the outcomes of the quality assurance procedures, there were several things that pointed to the need for a standard review for the PT exam. These included variable pass rates from 1996 through 2004 and significant stakeholder feedback from PT program faculty and employers. Neither of these sources was observed for the PTA exam.
Linking Admissions Criteria to Performance on NPTE
As a PTA program, we maintain very high admissions criteria for our students. However, with the increased pressure to admit diverse populations as well as the pressure to admit most anyone at the community college level, we are having a hard time justifying this standard. Have there been studies linking admission criteria to performance on the PTA exam?
Most of the studies looking at admission criteria and pass/fail performance on the NPTE have been focused on physical therapist programs. However, several researchers are beginning to look at this topic related to PTA programs. You may want to contact your colleagues to see what efforts are being conducted related to NPTE performance of PTA programs.
Are NPTE Questions Getting Harder?
I have heard that the NPTE questions have gotten harder over the last couple of years due to better item writer training. Is this true?
The pass rate for the NPTE for physical therapists dropped in 2003. Many stakeholders have asked if more difficult questions are being included on the NPTE, which in turn has made the examination more difficult to pass. Individuals have speculated that as FSBPT improves the quality of items written for the NPTE, the reliability of these items improves, and it becomes more and more difficult for candidates to determine the correct answer through the powers of deduction and logic.
High quality items are more difficult to answer correctly for students who do not know the material. However, test questions on the NPTE are not becoming more difficult. The FSBPT monitors item- and form-level difficulty as a standard part of its psychometric procedures. Test forms are assembled to be as similar as possible with respect to content and difficulty. Forms administered from 2001 through 2005 vary slightly in average item difficulty but do not show a consistent trend toward increasing difficulty. It is important to note that even if the average item-level difficulty across forms had increased, this should not influence the pass rate because all test forms are equated so that a form with more difficult items will have a lower passing score than a form with easier items.
Do NPTE Items Have More than One Right Answer?
I have been told that the exam is structured in such a way that there are two "right" answers, but that one is a better choice. Is the exam scored using a partial credit model to account for an individual who might select the less favorable of the correct choices? Or is it an all-or-none credit system?
There is only one correct answer for each question. The item writers and exam committees go through great lengths assuring that there is one correct answer and three incorrect answers. Each option must be backed up with a rationale and reference as to why it is either correct or incorrect. Beyond this, statistics are collected on the item that also helps support the one correct answer. There is no “partial credit” for an answer. It is either correct or incorrect.
Recalled NPTE Questions
A question from a student: As an exam candidate who just completed my examination, I recall several questions on the exam where I was not sure of the answer. Is it ok for me to discuss the questions with my instructor?
On the surface, it may seem natural for you to go to a faculty member to discuss specific questions on your licensing examination. You may want to be assured that you answered the item correctly. You may also be motivated to “learn from mistakes” and a discussion of the question could be a productive learning experience. This is particularly true if you find you have failed the exam. You certainly want to pass the next time! However, by doing this you are putting yourself and your instructor in jeopardy.
Every item on the exam is copyright protected. When a candidate takes the exam, the candidate agrees not to share any question or part of a question with anyone else. Good intentions do not nullify this agreement or the copyright protection of the exam. The bottom line is that it is illegal for a candidate to share a test question with an instructor or anyone else.
You might say: “Ok, so it is technically illegal, but what is the harm if I discuss the item with my instructor? I can trust her. She would not share the information. So, no harm no foul.” Basically, it is not for you to decide when you can break the law or when you can’t. The law is the law.
A parallel question from an instructor: Can I talk to my students about the NPTE after they have taken it?
Certainly general comments about the exam are not a violation of copyright law. However, if your students discuss specific items or specific characteristics of the exam with you, they are in violation of the confidentiality terms that they agree to when they take the exam. Included in these terms is not disclosing the content of examination items. Legal action could be taken against candidates who violate these terms including criminal prosecution. Ultimately the student may risk being able to ever obtain a license.
Educators may unwittingly cross the line by asking students questions about the exam or by listening to a student share a question from the exam. It is important that educators defend the integrity of the licensure process by making students and colleagues aware of these critical issues and reporting violations when they occur. You can also report any violations to security@fsbpt.org.
Educators can find accurate information regarding the exam in the “For Candidates/Licensees” section of the website. The entire candidate Handbook is found at “For Candidates/Licensees”/ “NPTE.” The test content outline can be found at “For Candidates/Licensees”/ “NPTE” / “Exam Development.” More detailed information on the NPTE can be obtained by attending a NPTE Workshop for Faculty. Information on upcoming workshops can be found in the “News and Events” section.
Enhancing NPTE test items
Currently, the NPTE contains questions that include graphics. Does the Federation have plans to add other testing technologies such as “hot spots” or video?
The Federation has been exploring other testing formats for some time. While we have the technical capabilities of utilizing these formats, we are not planning on adding these types of test questions in the immediate future. These types of questions create a host of additional issues and complexities. For example, a question that requires the examinee to review to a video clip of a gait pattern and determine the dysfunction may seem like the perfect question for a physical therapy licensing examination. However, before such a question can be used, several issues have to be addressed. How much time needs to be allowed for a question that requires the viewing of a video as compared to a standard multiple choice question? How many times should the candidate be allowed to re-run the video? Is the video clip clear enough that all candidates are able to see the action taking place clearly? How does one account for the psychometric differences between the video question and other questions?
Another reason we are waiting prior to implementing additional test question formats is to continue to allow our tests to remain stable for a period of time before implementing any significant test format changes.
In spite of these factors, we do believe there may be some advantages to utilizing different test formats at some point in the future and will continue to review the literature and address the questions mentioned above. Ultimately, it is critical to determine whether or not these additional test formats provide us with better information related to the competency of the entry-level physical therapist.
Can you provide the background on the 20 PT NPTE scores that were invalidated in 2007? The following information was taken from the “News and Events” section.
Forensic Analysis Conducted to Investigate Effect of Trafficking in Recalled Test Items Leads to Invalidation of 20 Candidate Test Scores
On Friday, August 17, 2007, the Board of Directors of the Federation of State Boards of Physical Therapy approved the invalidation of 20 candidates’ National Physical Therapy Exam (“NPTE”) test results. This decision resulted from an extensive forensic analysis of the test performances of all candidates who sat for the NPTE between March 1, 2005 and June 5, 2007.
The forensic analysis, conducted by Caveon, a test security company, was commissioned in response to the unlawful trafficking of NPTE questions by Philippines-based exam prep centers. Through its own private investigation efforts, as well as Philippines government surveillance and raids of two Manila test centers in January 2007, FSBPT has confirmed that the centers have distributed to customers compilations of actual NPTE test questions memorized and shared by prior test takers (“recalled items”). In an effort to assess the potential effects of this practice of using recalled test items, Caveon analyzed approximately 23,512 test performances of all NPTE candidates, regardless of place of education.
Caveon’s analysis conclusively establishes that at least twenty individuals benefited unfairly from advance access to recalled test items. All twenty candidates are Philippines-educated, some but not all of whom are already licensed to practice physical therapy. FSBPT’s assessment and review of the Caveon forensic analysis is continuing, so as to determine whether additional candidate score invalidation is appropriate.
In identifying these twenty candidates, the forensic analysis used three statistical indices to identify aberrant candidate performances. First, performance on compromised test questions (those known to be compromised by distribution at Philippines-based test prep centers) was compared to performance on non-compromised test items. Second, the similarity among candidate response choices was examined, with higher degrees of similarity suggesting the possibility of prior knowledge of test content. Third, the analysis computed the probability that each test taker had attended a course at which recalled items were used. In each case, the percentage of candidates flagged as aberrant was highest for Philippines-educated test takers.
FSBPT limited the universe of “aberrant” test performances under each of the three indices to those test results whose likelihood of occurring by chance was at least 1 in 10,000 (one in ten thousand). The twenty invalidated candidate scores are those that appeared aberrant based on all three statistical indices. The likelihood of aberrant performance on all three statistical indices is extremely unlikely and at least less than 1 in one million.
“As with every decision we’ve made in addressing the troubling use of recalled items, the FSBPT Board did not take this action lightly,” stated E. Dargan Ervin, Jr., FSBPT President. “We made the decision only after careful consideration of the issues and in light of the overwhelming statistical data that calls into question the legitimacy of these scores.”
School Pass Rate Reports
Why do you post ultimate pass rates for individual programs and not first time pass rates on the public site?
The intent of the FSBPT Board of Directors is to post first time pass rates as well as ultimate pass rates. However due to some of the historical fluctuations in pass rates, the Board decided to hold off on posting first-time pass rates until they became more stable. The Board will revisit the posting of first -time pass rates in two to three years.
Why does my listing of students keep changing? (AKA – A Candidate’s Travels through the FSBPT Database)
Imagine a program director’s frustration as a student appears and disappears from her school report when she specifically asked for a correction. This apparently spontaneous appearance and disappearance is related to the structure of the FSBPT database and the various sources of information for the database including the students, their schools and their respective licensing authorities.
When a student first enters the database, the information generally comes from one of two sources; the school scantron (blue bubblesheet) or from the student as he/she registers for the examination or another FSBPT product. As educators you are surely familiar with the challenges in students using scantron forms. The FSBPT routinely sees situations where a student has written the correct information on the scantron form but then did not correctly fill in the corresponding “bubbles.” This can cause a student to show up in the wrong school and/or in the wrong graduation class. When these errors are reported FSBPT staff makes the requested corrections.
In December 2005, the FSBPT modified its database structure to prevent students from updating their school name and graduation date after it is entered in the system. Prior to December 2005, every time a student interacted with the FSBPT database to register for the NPTE or law exams, request PEAT, request transfers etc, the student was required to enter school information and graduation date. This allowed students to inadvertently make mistakes when inputting their information. Now when students come to the FSBPT database they are presented with the information that currently exists in their FSBPT record. The student can update address information or name info but cannot update their school or graduation date. If a change is required the student must call the FSBPT to make the update.
While the process has been tightened, there are still opportunities for information to be updated which can cause a student to move from a school or graduation year. A primary way is when a jurisdiction licensing authority supplies licensure information to the FSBPT. When the licensure information is received, the FSBPT updates the database with the jurisdiction’s data. If there are inaccuracies in the jurisdiction’s information or if a candidate has gone to school in more than one jurisdiction, this can create problems for school reports.
The FSBPT continues to evaluate the process for recording student information to try and reduce the complexity and potential for errors. One area currently being investigated is replacing the scantron process with an online process. Look for information on this evolving process in future newsletters.
How do I order school reports?
School administrators will need to go back to “For Faculty” and click on “School Reports” to order school report subscriptions.
How do I find out my school code and password in order to log on?
If you have not yet subscribed or don’t know/don’t remember your school code and password, please send an email to schoolreports@fsbpt.org. From there, go to “For Faculty”/ “School Reports” on www.fsbpt.org and log in to either request a free subscription (reports are mailed twice a year at specific intervals determined by the Federation) or purchase an online subscription.
What kind of reports do you offer?
We offer four types of reports. Before you order any subscription, it is a good idea to review the sample reports first. That way you can be sure you’re ordering the report that provides the information you really want. Go to “For Faculty”/ “School Reports” on www.fsbpt.org. Look to the right of the SCHOOL ADMINISTRATOR LOGIN box. You will find report descriptions and links to sample reports.
1. Basic Mail Subscription (Free): Includes two "Basic Reports" that are sent via mail at specific intervals determined by the Federation. If you would like to receive the free basic mail subscription, you must subscribe to it through this online website. This report is available for the current year and it will include information for two prior years.
2. Basic Online Subscription ($100): Includes two "Basic Reports" that are accessible via the Schools Home website. Basic reports show pass rates for most current graduating classes and two prior years. They include names of student in those graduating classes, although they do not include individual student’s scores on the NPTE.
3. Enhanced Subscription ($200): Includes two "Enhanced Reports" that are accessible via the Schools Home website. The format of the enhanced report is the same as the Basic Reports but in addition, Enhanced Reports provide individual students’ scores on each attempt to pass the exam.
4. Content Area Subscription ($200): Includes two "Content Breakdown Reports" that are accessible via the Schools Home website. This report is available for graduating classes of 2003 until present. It compares your first time test takers to first time test takers from all U.S. CAPTE accredited schools during the same period on each area of the examination.
Can I pay with a purchase order?
The online system has two payment options; credit card or invoice. If you pay by credit card, your reports are available to be run the same day. If you prefer to have your institution send a check, you can choose to pay by invoice. When you choose to pay by invoice, the last screen of the purchase process is the official invoice. The invoice can be printed and submitted to your accounts payable department so that a check can be issued and mailed to the Federation with the invoice. Please do not send a purchase order - use the invoice you printed at the time you registered online for your subscription.
What is the “My Students” listing?
The “My Students” listing is a critical component of the school report process. Immediately prior to running each report you should review “My Students” to verify that all the students on the listing did indeed graduate from your institution. If you find a student listed that did not graduate from your program, there is a link to “Request a Correction” for each graduating class on the My Students page. Remember to review the “My Students” listing prior to running every report.
Additionally, you can view the “My Students” listing to see how many candidates have tested (and when they tested) so that you can decide when the best time is to run your report. For instance, you probably would not want to run a report that includes candidates for 2007 if only a small portion of those candidates have tested.
Why do my reports expire after 30 days?
The Federation simply does not have the “storage space” to maintain all school reports for an unlimited period of time. Instead, you should print the reports or to save them as Excel files on your own computers so that you have a permanent record.
If my reports expire after 30 days, why have a subscription for a year?
The first reason is so that you can choose when you would like to run your reports during the year. You may choose to run one report mid-year and another report at the end of the year. The second reason is that by having a subscription, you can request additional reports at a discounted rate of only $50.
PEAT
Do you offer discounts for group purchases of the Practice Exam & Assessment Tool (PEAT)?
Yes. In order to qualify for a group PEAT discount, 15 or more students must be registered. If the group pays by check, the cost will be $56 per PEAT ($58 per PEAT if paid by credit card).
To purchase PEAT as a group, you’ll need to send an email to peat@fsbpt.org. Please request a registration form and instruction sheet. Each student will need to register as individuals, indicating “pay by check” for the method of payment. The students will each need to provide the school with a social security and transaction number (obtained once the student has signed up for PEAT).
Once the students have signed up (everyone must be registered in order to process the payment), a confirmation email will be sent to each individual with his/her product key. The school will also receive the returned registration list with each student’s product key and transaction code.
HINTS
Many times, students will see the following message when they log on to PEAT. “Thank you for visiting FSBPT. PEAT is now running in another browser window.” In this case, the student will need to disable their pop-up blocker in order to see PEAT. They should allow pop-ups for the duration of the PEAT.
PEAT can only be accessed through Microsoft’s Internet Explorer browser – no other browser will work.
PEAT expires 30 days after the first practice exam has been accessed. However, students can download and print the supplemental materials and this won’t count toward the 30 days.
Once the student has taken the practice exam and reviewed the performance feedback on the first exam, they cannot go back to the first practice exam or rationale. They can only move forward to the second practice exam.
An extension is available for 15 days if the student has not completed both practice exams. The extension fee is $35 and can be purchased by going to “Home” and selecting the “Purchase PEAT” shortcut.
Confirmation emails for PEAT purchases are generally received within 2-3 hours. The exception is YAHOO accounts (they seem to be sent to junk mail or filtered as spam). If the student hasn’t received the confirmation email, they can call or email FSBPT and receive the product key and confirmation email.
State Practice Acts
We’d like to be able to see what our state’s practice act allows, such as whether it allows temporary licensure or direct access to physical therapy services. Does the Federation maintain that kind of regulatory information?
Yes, you can find all kinds of regulatory information in the Jurisdiction Licensure Reference Guide on the Federation’s public website. This guide compares some of the major similarities and differences among the jurisdictions’ practice acts. It may be used as a reference for educational programs, students, licensing boards, professional associations, and others. Information is listed in aggregate and by individual jurisdiction. To review the information, go to “Regulatory Tools” and click on “Licensure Reference Guide.”
