The number one thing that most administrations miss out when they schedule testing is the question of what they really want to know, as you’re pointing out. They should be testing the thing their instruction is designed to improve, because the whole reason for testing is to see whether it really did anything or not. It’s scary to think of several days of work for several people, all for – what?[/quote]
Thanks. You seem to share some of my reservations.
I’ve been making myself unpopular by asking the “What’s this for?” question, and havn’t had much of a reply beyond the repetition of Key Performance Indicator as a sort of mantra.
I’ve also suggested that if they’re going to do it, they need to use blind testing. No response to that so far. They might not have understood, or they might be concerned that it’ll stop them getting the result they want, if MOE window-dressing is indeed the objective.
The details of the proposed testing aren’t clear, which is understandable given we’re in the end-of-term grading marathon, but the outline of the tasks and criteria proposed for the four testers is:-
Read a text aloud : Pronunciation Intonation and stress
Describe a picture : All of the above, plus Grammar Vocabulary and Cohesion
Respond to questions : All the above,plus Relevance of content and Completeness of content
Respond to questions using information provided : All of the above
Propose a solution : All of the above
Express an opinion : All of the above
Seems likely to be non-trivial to me. Also seems unlikely that grammar, vocabulary and cohesion, for example, are going to be significantly improved by a 7-day “cultural exchange” programme.
At the risk of sounding like I’m trying to re-invent the Heisenberg Principle, IF you got a (statistically? t-test?) significant improvement on the second test, how do you know its not due to the practice effect of the successive testing?
The test, after all, is inherently more relevant to itself than it is to the summer school activities
If you don’t get a significantly significant improvement, so what?
Is that evidence that the Summer School is a waste of time?
If it is, are the “clients” going to have a use for that unpalatable conclusion?
There’s no point in asking a question if you can’t interpret or use the answer.