Is it possible to prove a student essay is AI?

Plagiarism used to be easy to detect (and subsequently fail a student for). However, ChatGPT written essays are proving a challenge. While I can strongly suspect a student with a basic proficiency in English cannot write a flawless, if extremely generic and impersonal, essay… it’s very difficult to prove. And if I can’t prove an essay is AI slop, then I can get in serious trouble if a student complains that I failed them based on my bias or without solid evidence. I try to use sites like ZeroGPT but they’re problematic and the same text can sometimes yield two different results (one essay was 80% suspected of being AI, and then I checked it again and it was 50%). How are the rest of you uni and high school teachers handling this scourge during grading time?

1 Like

There isn’t.

7 Likes

Use many cctv

Place it beside him or her

2 Likes

All writing is done during class time. A sad waste of teaching time, but I can’t see any other way.

10 Likes

I thought there has to be algorithm out there that can detect AI. You can check for hallucination and fact check every little thing out there, but the problem is it can be REALLY hard to find the errors when AI written stuff just sounds so good.

I do timed writes. They get a paper with five segments of lines per line ( __ __ __ __ __) and ten minutes to write, one word per line [segment]. They have to write something that makes sense/can’t just word vomit nonsense. If they do nothing or copy from their books hidden under their desk or just write random vocabulary words then I have them come to me at another time that day and do it correctly (a zero would just encourage them to not try.) I tally up how many words they write. My goal is for them to be able to write something and they need a lot of practice writing in order to get better at writing. They do get better at writing when we do this consistently and my standards are enforced (letting them copy nonsense or just write random words would do nothing to support them).

My theory is that students use AI

  1. Because they’re lazy A.F.
    But also:
  2. Because usually they don’t have any idea how to actually write.

So giving them space to learn how to write (having them just practice writing) takes away the “I can’t do it” attitude that every other adult in their lives instills in them.

6 Likes

AI written stuff reads very strange to me. I assume it will become more natural over time.

There are no errors to find, which of course is the tell but is unprovable.

2 Likes

If someone’s making bullet point with no good reason to do so, I’m suspecting AI.

4 Likes

It’s really easy to spot, very difficult to prove.

Check out the bs on Facebook at the moment. Everything and everyone is “iconic”, for example.

4 Likes

Some good replies here, but unfortunately I think @Marco is right.

It’s supposed to be a research paper, which they need time at home to prepare for, so having them do it in class wouldn’t be reasonable. Oh well, have to grit my teeth and give the girl who answers “Saturday” when I ask what she did last weekend an 88 … :man_facepalming: Had to be a little creative to even get her that low as there’s no real grammatical errors, but I made it a point to emphasize how impersonal and lacking in personal anecdotes or examples her essay was at least. Doubt she’ll get the message. She probably can’t even read my feedback.

9 Likes

What’s the purpose here? Couldn’t it just be a blank paper and ten minutes to write?

Do you think I’m in control of my school’s CCTV? And that I can just position it behind any student at random? :face_with_monocle:

Get them to produce the research paper at home. Then they have to produce a synopsis of their own work in class time.

4 Likes

Anyway, is writing not becoming irrelevant? As well as thinking.

6 Likes

I mean, it’s not the worst idea. Though you’d have the typical deers-in-headlights either buckling under the pressure or giving perfunctory answers (“hello teacher, my essay is about pollution”… sits down) that again, I can’t determine much based on.

Reading and writing is a great joy in my life, and always will be. Whether higher education has collectively given up and decided we should all be a generation of brain rotted tech bros and gals… that I can’t answer.

2 Likes

As an interest/hobby it’s great. Arguably no longer necessary for most people.

In theory, there should be a way to check if the essay was written or just copy and pasted. I guess students could get round that by copying word by word from the LLM created essay. Most genuine native speaker essays would show some editing hisory where they checked or rewrote pieces of work. I guess some students would still fake this even if it took longer than actually doing the work.

I remember at university I pulled an all nighter to get one philosophy essay in and then the computer had a glitch whereby my entire few thousand word essay dissappeared. I managed to sit there and rewrite the whole thing again from memory before dawn, but even in that case, I don’t think I typed the essay in word by word with no editing or rearrangement of arguments needed. But maybe some people’s brains would be able to pull off an essay fully structured without needing to edit a single paragraph.

I guess @nz 's method is the only way, and I like that it actually addresses the cause of this issue, and thus the test and the teaching go hand in hand, like sporting events and the training in between. Perhaps @BiggusDickus 's method of a synopsis would help if it had to be written in class without reference to the original, but again students could simply memorise a prewritten text after the first time this method was used.

1 Like

No way to know. Detector are really bad, especially with non native because low perplexity.
You could ask them to share the Google Docs with the edit history so you can check how the paper evolved. But they could cheat that too.

3 Likes

They need to produce multiple copies of their essay from beginning to end.

  1. Brain storming ideas and topics handed in. (photocopy it)

  2. First rough draft handed in. (photocopy it)

  3. Second rough draft handed in. (photocopy it)

  4. Final submission.

Doing it this way is kind of like a math problem. Show your work you fucking cheater! You can’t just write the answer to a complicated math problem. show the progression like a proof.

If in fact it’s a research paper, then this shouldn’t be unreasonable to make them do this.

Then you’ll have definite proof of cheating or not.

When I say photocopy, of course I mean take a picture with your phone.

13 Likes