Visit ManhattanGMAT.comMore from the GMAC Summit
Here’s more detail on the GMAC Summit from MGMAT Instructor Stacey Koprince:
Last week, I attended the GMAC Test Prep Summit, a biennial gathering held by GMAC specifically for test prep companies. Most of the things we discussed are probably only of interest to those of us who work in the industry. There were a few things, though, that would be quite useful for students to know.
When Dr. Lawrence Rudner (GMAC’s Vice President of Research & Development and Chief Psychometrician) speaks, it’s definitely in all of our best interests to listen. He’s basically responsible for the construction and continued operation of a valid GMAT CAT. All data and quotes in this article courtesy:
Rudner, Lawrence M. (2009). GMAT Psychometrics. Materials presented at the 2009 GMAC Test Preparation Summit, New York City, NY. October 15, 2009.
Penalty for leaving questions blank
The slide that I think was most important began with this headline:
“Myth: It is more important to respond correctly than it is to finish.”
The slide showed a graph with the integers 0 through 5 and the label “# of not-reached items” on the x-axis, and the percentiles 50 through 100 (in 10-point increments) on the y-axis. (If it’s hard to visualize, go ahead and draw yourself a little representation of that.)
There are 6 points on the graph: the first at 0 and 70th percentile, the second at 1 and approximately the 66th percentile, the third at 2 and approximately the 63rd percentile, the fourth at 3 and the 60th percentile, the fifth at 4 and approximately the 57th percentile and the final at 5 and approximately the 55th percentile.
Essentially, the slide is showing what would happen to the score of a student who might have expected a 70th percentile score if that person left blank 1, 2, 3, 4, or 5 questions at the end of the section: the score drops in a big way.
The bottom of the slide says “Fact: There is a severe penalty for not completing the GMAT test.”
Larry also told us that there are different consequences depending upon the scoring level of the student. For lower-scoring students, running out of time doesn’t have as big an impact on the score. For higher-scoring students, it makes a big difference –and the higher the student was before running out of time, the more of a difference it makes to the final score. This applies both when the high-scoring student leaves items blank and when the high-scoring student guesses but gets most of the questions wrong.
What can we take away from this? Well, the specific numbers shown in Larry’s chart may not be the precise penalty for every student who runs out of time on the test; according to him, it varies based upon the student’s scoring level. I think we can assume that the numbers are probably fairly accurate, then, for the given scenario (a 70th percentile test-taker), but not necessarily for testers with much higher or lower scores.
Because Larry also told us that running out of time “matters more” for higher-level students, I think we can also assume that the penalty would be at least as high, and probably higher, for 80th or 90th or 99th percentile hopefuls. Clearly, we absolutely cannot afford to spend a bunch of extra time earlier in the section – that chart showed a 15 percentile-point drop for a 70th percentile student who left 5 questions blank at the end!
Score Improvements On A 2nd Official Test
Larry also provided some data about the kinds of score changes that people achieve from their first official test to their second official test. On average, students taking the test for the 2nd time show a 34-point overall increase; this data has been published by GMAC previously. The standard deviation on the statistic is fairly high: 65 points. (This means that, while +34 is the average score change, there’s quite a bit of variability among individual students.)
About 31% of test-takers unfortunately experience a score decrease or stay the same on the 2nd test. About 28% achieve a score increase of 10 to 40 points and 27% achieve a 50 to 90 point increase. About 14% achieve a 100+ point increase (though only 3% achieve a 150+ point increase).
These data points are all from a 1st to 2nd official test. GMAC has found that, on subsequent tests, the average score change is close to zero; in other words, after the 2nd test, it’s not typical to see substantial changes in score.
There are also some trends in terms of who tends to achieve the best score increases from a 1st to a 2nd official test. People who did not finish the quant section the first time around record an average 49-point improvement on the 2nd test, while those who didn’t finish the verbal section record an average 48-point improvement. (No data was given on the average improvement for those who did not finish both the quant and verbal sections the first time around.)
People with below-average scores on the first test record an average 44-point improvement on the 2nd test. Finally, people with what are called “high discrepancy scores” record an average 42-point improvement on the 2nd test. A “high discrepancy score” has to do with a discrepancy between the tester’s GMAT score and his/her undergrad GPA. (Larry didn’t tell us what qualifies as enough of a discrepancy to earn the “high discrepancy” label.)
The take-away here is two-fold. First, the average change is a 34-point increase, which bodes well for those who want to take the test a second time. Second, if you fall into one of the “higher score increase” categories noted above, then it may be even more worthwhile for you to take the test a 2nd time. (Of course, you have to do the work to improve – though we don’t know for sure how people achieved the increased scores, I highly doubt it happened without some very hard work!)