What do Chinese tests actually test?

Had a test in Chinese class this morning (Taida); it threw me into a depressive state for the rest of the afternoon. Here we are, lower advanced students doing a course on magazine and news articles (original materials), and teacher comes up with a test scenario in which 40% of the mark is derived from zao juzi (‘make a sentence, no matter how stupid, with the word or sentence structure provided’)!!

So innane and frustrating it was all I could do to stop myself from crying right there in the classroom. I was stupid enough to expect a series of short answer QUESTIONS on the text, or perhaps even a comprehension extract on a related topic, but no. The same old, sloppy crap (along with a listening section that was completely unrelated to the original article!). If I pulled this stunt in an English class here I’d be sacked in a second.

I’ve been studying Chinese for the best part of the last decade, and no matter how progressive some teachers may appear on the surface, when push come to shove what really counts is how many characters you can memorise. Fuck me; what a total waste of my time! I want a test that makes me think, rather than one that makes me crawl around in my (admittedly small) brain for that one weird character which you have to use 'cause laoshi said it was important…

:help: fear I’m going :loco:

Don’t blame the teachers, most of them aren’t trained. A bachelor’s degree in Chinese (from a Chinese university) doesn’t make you a qualified CSL teacher, and even a qualification as a CSL teacher doesn’t necessarily make you a curriculum writer or a test writer.

While I hear your frustration with this sort of test, an even more deep-rooted problem in Taiwan testing circles is the failure to realize that a single test item should measure a single thing. Otherwise you don’t know what you’re measuring. A good example would be a reading comprehension passage where the answer choices are four chengyu. So, if the student gets it wrong, does it mean he didn’t understand the reading passage? Or is it that he happened to miss class the day they “learned” some of the chengyu? Either way, while it gives raw numbers to be used for, say, ranking students for scholarships or “proving” one thing or another, it doesn’t do what a test is supposed to do in an ideal state – that is, show the instructor where the student(s) need reinforcement and what has been mastered.

Even listening comp is not immune. I’ve taken LC tests that were more like mental math challenges (good for Asian students, I suppose?) “Lao Zhang bought three four-cent stamps, four six-cent stamps, and one eight-cent stamp.” Now that you’re adding up the prices, “How many stamps did Lao Zhang buy in all?”

:noway:

I had a test this week in which one of the answers was dependant on how many decibels (fenbei) was loud (apparently more than 70) and how many was quiet (apparently less than 30). How does that knowledge affect my ability to speak Chinese?

Thankfully years of Chinese tests has enhanced my ability to guess multiple choice answers and I got around it :sunglasses:

If you need to memorize characters see this thread:
forumosa.com/taiwan/viewtopic.php?t=38059

If you want a sample chinese test try this out:
cpt.mtc.ntnu.edu.tw/tma2/

[quote=“ironlady”]Don’t blame the teachers, most of them aren’t trained. A bachelor’s degree in Chinese (from a Chinese university) doesn’t make you a qualified CSL teacher, and even a qualification as a CSL teacher doesn’t necessarily make you a curriculum writer or a test writer.

While I hear your frustration with this sort of test, an even more deep-rooted problem In Taiwan testing circles is the failure to realize that a single test item should measure a single thing. Otherwise you don’t know what you’re measuring.[/quote]

Having been hired to write a lot of the reading sections for standardized tests for a certain state board of education (in case anybody noticed I was gone the last three months ^^) I feel the need to point out that accurately measuring just-one-thing on a language-based test is very hard, and is often done poorly. [To be fair, I don’t have any special training at doing this, and am writing within the confines of some specifications that were decided way over my head – perhaps there are people who really do this well. But the specs I’m receiving are vague and clearly describe questions which a student could fail for a whole host of reasons, and these are from professional American educators… and also decided by committees.]

Anyway, I’m just saying that in my experience, it isn’t Chinese tests that are the problem; it’s standardized testing in general. Any standardized test is highly susceptible to test-mining, to using a process of elimination to identify correct answers, to a guessing-factor, even just to matching. Any standardized test is also highly susceptible to a student missing questions that s/he would’ve been able to handle perfectly adequately in a more realistic context – by an equivalent answer, or because the answer choices weren’t understood, or through any of a number of other reasons. In general, any test that relies primarily on receptive knowledge, rather than productive knowledge, doesn’t really tell you about productive ability… which you’d think would be obvious. Oh well.

Zao juzi is at least asking you to do something that resembles a communicative task. Few conversations (beyond deciding what to get for breakfast) involve multiple-choice responses…

but a test for a newspaper class ought to at least have questions that relate to the relevant articles??

Anyway, I’m curious – what do better-designed test questions look like? Ironlady, what’ve you seen in your experience as an actual trained educator?

Haven’t seen many well-designed ones in my experience. :smiley:

You’re absolutely right, though – I didn’t mean to infer that it’s ONLY a Chinese test problem. But there are more resources for ESL tests and so forth, and just the nature of things dictates that the more resources, the better your chance of getting someone who knows what they’re doing, as those people usually command a fairly good salary, after all.

It’s tough writing monolingual tests for a second language. If you have the luxury to write the test partly in English, for example, then you can truly test reading or listening. The student reads the passage in Chinese, then answers English questions on the content. You know if the student fails to pick the right answer (assuming your choices are solidly written) the problem most likely is that he or she didn’t understand the Chinese passage. At least you have eliminated the idea that he DID understand the passage but did not understand the question or one or more of the choices.

Even “zao juzi” is testing multiple things. What if the target word is used correctly but something else in the sentence is wrong? Does the teacher take off points for characters that are written wrong? How “wrong” is “wrong” when it comes to writing a character? When I taught 2nd year Chinese at Texas, I used to put a section on the tests where the students could write any way they could make me understand what words they were using. Characters, Pinyin, whatever. I was looking ONLY at whether the sentence made sense. In one way that made sense as I couldn’t do individual oral testing because of the number of students in the class, so this was the nearest thing.

Of course language is such a complex thing that it’s almost impossible to reduce it to a neat formula that will yield a number. Unfortunately, that’s what many programs want or need (for grades, rankings, etc.) I much prefer the testing method that goes with TPRS (the method I teach) – the tests and quizzes are intended only for teacher feedback. If the teacher is doing his or her job, you get at least 80% of students scoring 80% or higher. Else, you need to go back and find out what you didn’t present enough or well enough. The idea that someone can’t learn a language is astounding to me when our brains are wired to do so. Some students may have problems with written forms, but spoken forms are within the reach of nearly everyone. I had special-needs kids and a Deaf student in my classes in the US and they did just fine.

Personally I only test vocab, as grammar will be acquired in a certain order anyway, no matter what you teach in which order. If you try to pound late-acquired stuff into a learner’s head at an early stage, you get overanalysis. IMHO better to be patient and focus on things that let the learner get out and about and use the language, which will give him more input and more motivation to keep on. (Okay, I digress…) :blush:

[quote=“Toe Tag”]If you want a sample Chinese test try this out:
cpt.mtc.ntnu.edu.tw/tma2/[/quote]Thanks for the link Toe Tag. That test provides useful listening material even if one doesn’t want to try a test per se.