We have tried to ensure that the Performance Report (click here to view an example) is easily understood and self-explanatory. However, experience suggests that a few aspects of the report deserve additional comment.
Concerning Our Terminology — Peer Group
IPPT Programs vary in the word they use to identify the set of students with whom a student is compared when assessing her performance and progress. Most programs use "Cohort" or "Class", as in "a student in the Class of 2012". Here we use "peer" and "peer group" as synonyms for whatever word your Program uses to identify your current peer group.
Frequently Asked Questions
|How do I interpret the basic measures?|
|My Z-Score — What's that all about?|
For students at the medical schools
|What is my Transformed Percentage Adjusted Correct?|
A N S W E R S
Q: How do I interpret the basic measures?
A: We suggest that you think of the basic measures as follows..
| items attempted (#),
attempted / number of items (%)
| your assessment of the extent of your knowledge;
ie, what you think you know
|items correct (#)||what you know|
| adjusted correct (#),
Percentage Adjusted Correct
|what you know minus our mandatory statistical penalty to discourage guessing|
|adjusted correct / attempted (%)||how well you can self-assess; what you know relative to what you think you know|
Q: My Z-Score — What's that all about?
A: A z-score is any quantity expressed relative to its group mean (m) and the standard deviation (sd) of that mean. Z-scores are often called standardized scores, because the quantity in question has been made relative to a given standard.
Here the quantity is your number of correct responses minus our penalty
for guessing, ie, your number "adjusted correct".
The group is everyone who sat the test who is at your current level
in the curriculum; ie, your peer group.
The general formula for a z-score is provided on your Performance Report
but here it is again:
z = (your score minus the group mean) / standard deviation of the group mean
The great attraction of the z-score of your PAC (Percentage Adjusted Correct) is that the z-PAC expresses your performance relative to that of your peers. That student-to-peers linkage is essential to determining whether you are progressing or progressing at a satisfactory rate.
The great benefit from expressing your PAC as a z-score arises from the fact that, by definition, any complete set of z-scores has a mean of "0" and a standard deviation (of that mean) of "1". These characteristics facilitate several easy and useful findings about one's performance on any one test and across multiple tests.
- Knowing that the peer group's standardized mean is "1", a student with a positive z-PAC knows immediately that her score is above the average score for her peer group. A student with a negative z-PAC knows that her score is below that average score.
Knowing that the peer group's standardized standard deviation is "1"..
- the student can easily judge how far her score on any one test is from the average score for her peer group.
- the student, her advisor and her Program can easily compare her performance across multiple tests, ie, her progress, because each test's standardized peer mean and standard deviation are the same constants, "0" and "1" respectively.
Thus a z-PAC that rises with each successive test is a clear, easily understood indication that the student is improving relative to her peers, ie, evidence of progress.
Conversely, a z-PAC that falls with each successive test is a clear, easily understood indication that the student is not keeping up with her peers, even though, as is usually the case, her raw (actual) PAC increases with each test.
For students at the medical schools
at Algarve and McMaster universities..
Q: What is my Transformed Percentage Adjusted Correct?
A: Your Program uses a second standardized measure, the Transformed Percentage Adjusted Correct (t-PAC) to track progress by both students and peer groups across successive tests. The Program adopted this measure because Program leaders want to eliminate (statistically) the impact of two factors on group performance data:
- variation in test difficulty due to variation in item difficulty, and
- variation in the academic backgrounds and knowledge of successive entry cohorts.
The Transformed PAC is like a z-score in that your raw PAC score is expressed relative to a group mean and standard deviation. However, in this case, the group in question is a hypothetical peer group with a mean and standard deviation determined in part from historical data spanning multiple peer groups and multipe past tests.
You could say that the t-PAC expresses your raw (actual) PAC score relative to the mean PAC that would likely be achieved by an average group of students at your level of the curriculum sitting a test of average difficulty.
The McMaster MD Program's hypothetical peer group means and standard deviations are listed below by test sequence and position in the curriculum. The corresponding set for the four-year Algarve program are omitted for the moment because the current values are under review.
THE TEST PEER GROUP ------------------------ ------------------ Seq in Acad Seq % Adjusted Correct Curr. Year in AY Mon Mean SD ============================================ 1 1 1 Sep 10 5 2 1 2 Feb 20 5 3 1 3 May 30 5 4 2 1 Sep 40 5 5 2 2 Feb 48 5 6 2 3 May 55 5 7 3 1 Sep 58 5 8 3 2 Feb 60 5 ============================================ Acad Year = AY = Academic Year
McMaster MD students can compute their t-PAC for any test using the above data and the following formula:
t-PAC = (A x B) + C
A = the z-score of your PAC for the test in question (displayed on your Performance Report)
B = the peer group sd (also displayed on your zone report)
C = the peer group mean (also displayed on your zone report)
Your Transformed PAC is the same relative distance from the hypothetical peer mean as your raw (actual) PAC is from the mean of your actual peer group.
The benefit for you, your advisor or a review committee of the Transformed PAC is more apparent when examining your Zone Report. The t-PAC is provided on your Performance Report so that you can more easily relate the t-PAC on your Zone Report to your actual PAC, which is displayed only on your Performance Report.