Too Hard

Whoops, Tim already wrote a blog post with this same name. I'm using it anyway.

I was the problem czar for the Harvard-MIT November Tournament, which happened on Sunday, November 7, 2010. As problem czar, I was responsible for making sure that enough problems got written on time for the contest to run. That in itself was a huge task, made no easier by the relatively small amount of help that the February problem czars gave me. I personally wrote the majority of the test, with help from Travis in some key areas (he wrote many of the geometry problems), and from Jacob in not as key areas (he kept giving me problems about Bayesian inference...).

66 problems later, I get the (somewhat delayed) response from testsolvers: "Too Hard."

What is too hard?

Honestly I am probably too biased to comment, but I don't believe that it's possible for something to be too hard on its own. Being too hard can cause other issues, such as a lack of distinguishing power, but being too easy can also cause that. And I am fully aware of the issues that come from a test that lacks distinguishing power.

So when I see a complaint that something is too hard, what do I think? I feel that it is extremely more likely that some aspect of the undertaker is off rather than that the task is actually too hard. And it seems like others are the opposite -- they would much rather blame the test than the test takers, especially when they are the test taker.

Actually, I do that too. Lots of my blog posts last year were about things that I disagreed with in tests. Was it really the tests that were at fault or was it me? I think it was some of both. I don't think it's deniable that the HMMT calculus test was unable to distinguish between the top 4 competitors (hint: they tied), but at the same time it wasn't like the test gave them absolutely zero opportunity to distinguish themselves (the tie was at 28, not 50).

Regardless, I feel like there is this idea that if lots of people get two or fewer problems then the test is too hard. So, people say, make there be a few problems that everyone will get. But why? Now I've reduced my contest from 10 problems to 7. That doesn't seem useful at all.

I've rarely seen someone look at a test and say ``This is too easy,'' but many people will look at a test and say ``This is too hard.''