Well, they would need to know something about the topic. The easiest way for them to do this would be to read about it. When picking what to read they would need to be able to estimate if it contained valid information. The meta-skills that would be developed would be:
* Understanding the quality of sources
* Understanding the different between primary and secondary sources.
* Establishing chains of evidence and tracing them through large datasets.
* Integrating information from different sources and using correlation to establish validity.
Basically, it would be a crash course in how to do research, and the best possible preparation for living in a world where we drown in uncertain information and still have to make the best choices that we can.
For bonus points, at the end of the process they would have a better understanding of the subject matter than somebody who had only read about it.
Someone was also demonstrating over the weekend that literally cut and pasted ChatGPT text output is easily detected as LLM output with near certainty. https://huggingface.co/openai-detector/
That said, adult me could probably use ChatGPT to stitch together a 1000 word high school paper pretty easily (and probably evade plagiarism/LLM detectors). But, as you say, I could probably get a lot of the way there by other means as well and putting words on the page is mostly not the hard part anyway.
If you play with it a bit, there are definitely patterns.
Here's one I did over the weekend: 99.98% fake.
The Battle of Agincourt was a significant event in English and French history, as it was a major victory for the English army against the French during the Hundred Years' War. The battle took place on October 25, 1415, near the town of Agincourt in northern France.
The English army, led by King Henry V, was vastly outnumbered by the French forces, which were estimated to be around 30,000 soldiers compared to the English army of just 6,000. Despite the odds, the English were able to emerge victorious thanks to their superior tactics and weaponry.
The English army was composed of mostly longbowmen, who were able to shoot arrows with great accuracy and power. The French, on the other hand, relied heavily on heavily-armored knights, who were less effective in the muddy and wet conditions of the battlefield. The English were also able to take advantage of the narrow front of the battlefield, which allowed them to focus their firepower and overwhelm the French forces.
Whereas the first couple paragraphs from a book I'm working on came out 0.02% fake.
* Understanding the quality of sources
* Understanding the different between primary and secondary sources.
* Establishing chains of evidence and tracing them through large datasets.
* Integrating information from different sources and using correlation to establish validity.
Basically, it would be a crash course in how to do research, and the best possible preparation for living in a world where we drown in uncertain information and still have to make the best choices that we can.
For bonus points, at the end of the process they would have a better understanding of the subject matter than somebody who had only read about it.