when prediction pretends to be understanding
evening,
someone told me.. “a dog can catch a ball without understanding physics.” been stuck in my head.
the dog thing
three moves:
- see. dog eyes the ball.
- do. dog lands where the ball lands.
- say. “see? no physics degree.”
fair. no collie ever cracked a textbook. but inside that skull is a rough gravity model, trained by evolution and fetch. that’s a model, even if it’s implicit.
compression & the big claim
the twitter fight was about compression. tighter zip ⇒ better prediction. cue the catchy line:
more prediction ⇒ more understanding
it fits on a slide, but it trims too much
prediction: given t, nail t + n.
understanding: say why the rules are like that.
why folks got spicy
people smell oversimplification quick. claiming “prediction = understanding” flattens a deep idea, they push back.
okay, llms?
want to know if gpt-6 “gets” chemistry? you’d have to:
- freeze on one model checkpoint (these things age fast).
- agree on what “explain” even means, not just next-token hits.
- read the fresh interpretability papers before tweeting hot takes.
wrapping up
- dogs: embodied probabilistic models.
- zip algorithms: pattern maximizers.
- llms: large statistical engines edging toward causality.
all three predict. only some creep into explain. the space between is where science, philosophy, and ml wrestle.
next time someone says “prediction isn’t understanding,” ask: cool, so what tips it over? when that answer’s clear, we’re getting somewhere.. maybe past what a dog can do.