4 Steps to Get the Most out of your CATs (part 2)

Stacey Koprince —  February 17, 2014 — Leave a comment

gmat-cats-computer-adaptive-test-2Last week, we talked about the first two elements of getting the most out of your CATs

#1: How NOT to use your practice CATs

#2: How to analyze your Strengths and Weaknesses with respect to timing

This week, we’re going to dive even further into Strengths and Weaknesses using the Assessment Reports.

#3: Run The Reports

Note: this article series is based on the metrics that are given in ManhattanGMAT tests, but you can extrapolate to other tests that give you similar performance data.

In the ManhattanGMAT system, click on the link “Generate Assessment Reports.” The first time, run the report based solely on the most recent test that you just did; later, we’ll aggregate data from your last two or three tests.

The first report produced is the Assessment Summary; this report provides the percentages correct for the five main Q and V question types, as well as average timing and difficulty levels. Here’s an example; see whether you can spot any problem areas.

 

237&238 - image #1

Most people will immediately say “Oh, this student is better at PS than DS.” Why? “Because the percentage correct is higher for PS.”

Actually, that’s the wrong takeaway for this particular set of results. It’s crucial to compare three data points at once: the percentage correct, the time spent, and the difficulty levels.

It’s true that the PS percentage correct is significantly higher. But look at that timing: the student is spending a lot longer on PS. DS is averaging well below the normal time of 2 minutes. Further, check out those difficulty levels; the correct answers are at the same level of difficulty!

It’s likely that the student’s DS percentage correct is artificially depressed because he’s just not spending enough time. I’d tell this student to start looking for careless errors—everywhere, but specifically on DS problems. He needs to learn to cut himself off on the harder PS problems (that he’s getting wrong anyway!) and reallocate that time to DS.

These kinds of results should catch your eye:

-       Percentages correct below approximately 50%, especially when coupled with lower average difficulty levels and higher average times. Note that I don’t consider DS a straight weakness for the above student, even though the percentage correct is below 50%. The other two data points indicate that the real culprit is the timing.

-       Average timing that is 30 seconds (or more) higher or lower than the expected average. The above student needs to take a look at timing on Word Problems (too slow) and Reading Comprehension (too fast).

-       A big discrepancy (more than 30 seconds) in average time for correct vs. incorrect questions of the same type; it’s normal to spend a little extra time on incorrect questions (because those are probably the harder ones!), but not a ton—that just means you’re being stubborn.

The second and third reports sort the questions by Question Format and Difficulty. Here’s an example of the Verbal report:

237&238 - image #2

Spot anything?

The student has done a phenomenal job on SC. I am wondering, though, why he was rushing so much on CR and RC. He should check these for careless mistakes. Or maybe mental fatigue was a factor? It might be worth checking out the Problem List again to see if he was speeding up as the test went on (and getting more wrong); that’s a classic sign of mental fatigue.

In these two reports, look for:

-       average timing that is 30 seconds (or more) higher or lower than the expected average, and whether that is happening on correct or incorrect questions (or both)

-       lower percentages correct on lower-level questions than on higher-level questions

In particular, these two things might appear together. If that happens, you might be spending too much time on incorrect higher-level questions and not enough time on lower-level questions (that you then get wrong because you’re rushing).

Note: the timing averages for Reading Comprehension can be misleading because the first question for each passage includes the time to read the passage itself. For Reading Comp, you need to dive back into the problem list to look at each problem individually in order to get a true picture of what happened.

Finally, we’re going to take a look at the fourth and fifth reports (Quant by Content Area and Topic, Verbal by Verbal Type and Topic). First, though, I suggest that you run the reports based on your last 2 or 3 tests rather than just your last test. We’re diving deep into the details with these final two reports, so there will be lots of categories with only one or two questions unless we add more data to the report.

If your last test was more than six weeks ago, though, then the data might be too old. You may also decide to look at these last two assessment reports based just on the last test first, and then run the reports again using your last two tests—your choice. If you use only one test, be aware that your analysis may need to be flexible for those sub-categories with only 1 question. If you get 0% of 1 question right, that doesn’t mean that area is a big weakness!

The fourth and fifth reports show all of the questions broken out by question type and sub-type or sub-topic. You’re going to use these reports, coupled with everything you’ve learned so far, to complete the 4th and final step of your analysis.

#4: Fill Your Buckets

You are going to place all of these problems into 3 buckets. These buckets roughly correspond to Great, Prioritize This, and Ugh. Note that the guidelines I give are approximate. If something is only slightly higher or lower than it should be, and you feel comfortable with it, then you can still count that in a “better” bucket.

Before you dive in, you may want to run the assessment reports again. Often, certain categories will contain only 1 or 2 problems, so you will be able to do better analysis with more data. If you’ve taken other tests within the past month, include them at this stage. Don’t include more than 3 tests (you shouldn’t be taking that many within a month anyway!).

Okay, run the program again and go to the fourth and fifth reports.

Bucket 1. Great! These are my strengths.

You get these questions right roughly within the expected timeframe (or faster).

Make sure that you actually knew what you were doing for each problem and didn’t just get lucky! Going forward, problems in this bucket are not high on your priority list, but there may still be things you can learn:

-       faster ways to do the problem

-       ways to make educated guesses (so that you can use the thought process on harder problems of the same type)

-       how to quickly recognize future problems of the same type

You may want to move on to more advanced material in these areas.

Bucket 2. Prioritize This.

These problems comprise three broad categories:

A. Careless mistakes. You knew how to do the problem but made some kind of error along the way. Figure out what error you made, why you made it and how you can minimize the chances of repeating that type of error.

B. Efficiency. You get these right but take too long to do so (but not ridiculously too long—if you took way too long, put this in bucket 3 instead). Which part took too long? Where did you get hung up? Find a more efficient way to solve next time.

C. Holes in foundation. You missed the problem and it was lower in difficulty than problems you usually get right. (Check the difficulty rating on your problem list.) You need to return to the fundamentals—the rule, the formula, the technique, whatever it is that caused you to miss this problem.

In general, prioritize the above three categories over all others. These represent the best areas for your potential improvement.

Bucket 3: Ugh. Just get these wrong faster.

You usually miss these problems—these are weaknesses or too hard for you. Often, you spend too much time getting these wrong. Alternatively, you sometimes get these right but spend WAY too long to do so.

The best remedy at this point is literally to get these wrong faster. I’m entirely serious. You’re getting them wrong anyway, so take less time to get them wrong! Re-allocate that time to questions from one of the other buckets, where additional time is more likely to make a difference.

In most cases, you likely knew you couldn’t do the problem but you didn’t want to let go. If you’re doing your job, the GMAT should be offering you questions that are too hard or will take too long to solve. Your task is to recognize when this happens and let the problems go. Go back to the first half of this article and read (or re-read!) the time management articles.

You may eventually be able to learn how to do some of these correctly and within normal time, but set these aside for now and concentrate on Bucket 2. On the real test, you’re still going to have several “My goal is to get problems like this wrong fast” categories.

One more thing: frequency

For all of the above, don’t forget to think about the frequency with which the material is tested. You might be terrible at 3D geometry (as I am), but that category is so rare that it’s not even worth studying. If, on the other hand, you’re also not so great at exponents and roots, you need to start studying; those topics are common. (If you’re not sure what is more or less frequently tested, ask your teacher or ask on the forums.)

No, wait, another thing: the 2nd Level of GMAT prep

Now that you know where to concentrate, how should you study? Go back to your books or lessons for any fundamental content issues. For any strategy or technique issues (e.g., how to know what the problem was actually testing), learn about the 2nd Level of learning to take the GMAT.

The 4 Steps to Getting the Most out of your CATs

(1) Stop Taking So Many CATs. Get a minimum of 2 weeks’ worth of study out of each one.

(2) Learn your strengths and weaknesses. Start with an analysis of your timing, since bad timing will kill your score no matter how good you are with the actual material.

(3) Run the reports to dive into the content and question types. It’s critically important to evaluate your performance across all three main axes at once—percentage correct, timing, and difficulty.

(4) Fill Your Buckets. Concentrate on Bucket 2. Every time you take a test, you’ll have new Buckets; you’ll be able to see your progress and adjust accordingly.

Studying for the GMAT? Take our free GMAT practice exam or sign up for a free GMAT trial class running all the time near you, or online. And, be sure to find us on Facebook and Google+, LinkedIn, and follow us on Twitter!

Stacey Koprince

Posts

Stacey Koprince is an Instructor and Trainer as well as the Director of Online Community for Manhattan Prep. She's also a management consultant who specializes in corporate strategy. She has been teaching various standardized tests for more than fifteen years and her entire teaching philosophy can be summed up in five words: teaching students how to think.

No Comments

Be the first to start the conversation.

Leave a Reply