Introduction

We have tried to ensure that the Performance Report (click here to view an example) is easily understood and self-explanatory. However, experience suggests that a few aspects of the report deserve additional comment.

Concerning Our Terminology — Peer Group

IPPT Programs vary in the word they use to identify the set of students with whom a student is compared when assessing her performance and progress. Most programs use "Cohort" or "Class", as in "a student in the Class of 2012". Here we use "peer" and "peer group" as synonyms for whatever word your Program uses to identify your current peer group.

Frequently Asked Questions

How do I interpret the basic measures?
My Z-Score — What's that all about?

For students at the medical schools
at Algarve and McMaster universities..

What is my Transformed Percentage Adjusted Correct?

A N S W E R S

Q:  How do I interpret the basic measures?

A:  We suggest that you think of the basic measures as follows..

items attempted (#),
attempted / number of items (%)
your assessment of the extent of your knowledge;
ie, what you think you know
items correct (#) what you know
adjusted correct (#),
Percentage Adjusted Correct
what you know minus our mandatory statistical penalty to discourage guessing
adjusted correct / attempted (%) how well you can self-assess; what you know relative to what you think you know

Q:  My Z-Score — What's that all about?

A:  A z-score is any quantity expressed relative to its group mean (m) and the standard deviation (sd) of that mean. Z-scores are often called standardized scores, because the quantity in question has been made relative to a given standard.

Here the quantity is your number of correct responses minus our penalty for guessing, ie, your number "adjusted correct". The group is everyone who sat the test who is at your current level in the curriculum; ie, your peer group. The general formula for a z-score is provided on your Performance Report but here it is again:
z = (your score minus the group mean) / standard deviation of the group mean

The great attraction of the z-score of your PAC (Percentage Adjusted Correct) is that the z-PAC expresses your performance relative to that of your peers. That student-to-peers linkage is essential to determining whether you are progressing or progressing at a satisfactory rate.

The great benefit from expressing your PAC as a z-score arises from the fact that, by definition, any complete set of z-scores has a mean of "0" and a standard deviation (of that mean) of "1". These characteristics facilitate several easy and useful findings about one's performance on any one test and across multiple tests.

  1. Knowing that the peer group's standardized mean is "1", a student with a positive z-PAC knows immediately that her score is above the average score for her peer group. A student with a negative z-PAC knows that her score is below that average score.
  2. Knowing that the peer group's standardized standard deviation is "1"..
    • the student can easily judge how far her score on any one test is from the average score for her peer group.
    • the student, her advisor and her Program can easily compare her performance across multiple tests, ie, her progress, because each test's standardized peer mean and standard deviation are the same constants, "0" and "1" respectively.

Thus a z-PAC that rises with each successive test is a clear, easily understood indication that the student is improving relative to her peers, ie, evidence of progress.

Conversely, a z-PAC that falls with each successive test is a clear, easily understood indication that the student is not keeping up with her peers, even though, as is usually the case, her raw (actual) PAC increases with each test.

For students at the medical schools
at Algarve and McMaster universities..

Q:  What is my Transformed Percentage Adjusted Correct?

A:  Your Program uses a second standardized measure, the Transformed Percentage Adjusted Correct (t-PAC) to track progress by both students and peer groups across successive tests. The Program adopted this measure because Program leaders want to eliminate (statistically) the impact of two factors on group performance data:

  • variation in test difficulty due to variation in item difficulty, and
  • variation in the academic backgrounds and knowledge of successive entry cohorts.

The Transformed PAC is like a z-score in that your raw PAC score is expressed relative to a group mean and standard deviation. However, in this case, the group in question is a hypothetical peer group with a mean and standard deviation determined in part from historical data spanning multiple peer groups and multipe past tests.

You could say that the t-PAC expresses your raw (actual) PAC score relative to the mean PAC that would likely be achieved by an average group of students at your level of the curriculum sitting a test of average difficulty.

The McMaster MD Program's hypothetical peer group means and standard deviations are listed below by test sequence and position in the curriculum. The corresponding set for the four-year Algarve program are omitted for the moment because the current values are under review.

        THE  TEST             PEER GROUP
------------------------  ------------------
Seq in  Acad   Seq        % Adjusted Correct
 Curr.  Year  in AY  Mon      Mean   SD
============================================
   1      1     1    Sep       10    5
   2      1     2    Feb       20    5
   3      1     3    May       30    5
   4      2     1    Sep       40    5
   5      2     2    Feb       48    5
   6      2     3    May       55    5
   7      3     1    Sep       58    5
   8      3     2    Feb       60    5
============================================
Acad Year = AY = Academic Year

McMaster MD students can compute their t-PAC for any test using the above data and the following formula:

t-PAC = (A x B) + C

in which:

 A = the z-score of your PAC for the test in question (displayed on your Performance Report)

 B = the peer group sd (also displayed on your zone report)

 C = the peer group mean (also displayed on your zone report)

Your Transformed PAC is the same relative distance from the hypothetical peer mean as your raw (actual) PAC is from the mean of your actual peer group.

The benefit for you, your advisor or a review committee of the Transformed PAC is more apparent when examining your Zone Report. The t-PAC is provided on your Performance Report so that you can more easily relate the t-PAC on your Zone Report to your actual PAC, which is displayed only on your Performance Report.