My agents kept teaching wrong physics with full confidence

[u/Gleb_SV]

I’m building a thing where a team of agents puts together a short interactive lesson on some topic. Code part was fine. What killed me was facts.

Some real ones I caught:

• phantom traffic jams have “no cause”. no, they grow from tiny fluctuations, there’s just no crash or light
• Cambrian = first animals. no, it’s when animals boomed
• catenary arch is pure compression. only under its own weight

Everything sounded right and looked polished. That’s the scary part, I almost shipped a few.

Now I have a list of these traps and a check step before anything goes out, but I still don’t fully trust it.

Anyone else generating educational content with agents? How do you catch the confident wrong stuff

[u/absolutenobody]

Not technically educational but I have an agent process 75 trivia questions per week for a pub quiz kind of thing. The questions (and answers) come from several large data sets online. Some of the q/a pairs are more than 20 years old and thus potentially no longer accurate.

I give the agent very detailed, very aggressively-worded instructions to review every question and every answer as though they are an insufferable know-it-all subject-matter-expert in each area, update any question that is ambiguous or subject to interpretation, update any answer that is not unambiguously and uniquely true, and to flag any question that can't be answered without qualifications like "technically" or "probably". It writes a review of each q/a and its changes to a markdown file.

So far I haven't caught it producing (or passing) any wrong info, nor have any of the quiz participants. And they are absolutely the kind of people who "well, actually" when they think they can get a point, haha.

[u/imitsi]

You can’t one-shot things like that. You need to ask it to be launching independent agents (frontier models) to verify accuracy.

[u/J_E_E_VACATION]

let me guess, free shit model? I use opus to do actual photonics research and it is absolutely astonishing.

[u/Gleb_SV → u/J_E_E_VACATION]

Almost xD

I'm using gemini-3.8-flash; I can't use Opus yet. it's too expensive

[u/J_E_E_VACATION → u/Gleb_SV]

DM me if you want a free week of claude subscription.

[u/Gleb_SV → u/J_E_E_VACATION]

I'm building with Claude Code, but I'm using Gemini in the my product.
The reasoning level seemed medium to me, I think. I'm still experimenting with prompts.

Right now I can't afford a high reasoning level: it takes a lot of time and costs a lot of money on the API. I need information and good visuals with interactivity at the same time, built within a limited amount of time and money. That's the difficulty right now.

[u/war4peace79 → u/Gleb_SV]

Sorry, I'll be blunt: cheap tools result in cheap outcome. To make an analogy, it's like hiring preschool students for undergrad work.

[u/J_E_E_VACATION → u/Gleb_SV]

That's why.
What reasoning level?
Also, are you asking it to research and cite it's sources for all information it finds?

[u/ComprehensiveShake76]

Confident wrong facts are the hardest class because nothing fails. I'd split the writer from the checker and make the checker work from sources, not memory: pull every factual claim out as one line, and a second pass has to find a source for it or mark it unsupported. Unsupported claims get cut or rewritten, not argued about. Your trap list is worth a lot, since the classic misconceptions are exactly what models repeat. I'd turn it into tests, so every lesson on a topic gets checked against that topic's known traps before it ships. For anything going to students, a human spot check on a sample is still worth it.

[u/Gleb_SV → u/ComprehensiveShake76]

this is honestly the most useful comment i got this week. im turning it into the actual plan: separate writer and checker, claims one per line, source it or cut it, and the trap list as tests

one question: would you run the checker per section or once on the whole lesson? per section feels more accurate but it multiplies the calls

[u/ComprehensiveShake76 → u/Gleb_SV]

Per section, but only for extraction. Pull the claims out section by section so nothing gets lost in a long lesson, then batch all of them into one sourcing pass, so the call count grows with claims, not sections. If cost bites, a topic that passes clean a few times can drop to a sample check. Glad it helped.