SlopShape: Identifying AI-Generated Commercial Web Content

AI-written company blog posts share a tidy structure that survives rewording and reveals which model wrote them.

96.1

macro-F1

Detection after AI rewording

Structural detection barely drops when every AI post is rewritten.

How it works

Word detectors are fragile
Word-level AI detectors fail once text is reworded, and they can't say which model wrote it.
Measure post structure
An LLM scores 13,500 blog posts on 176 structural features: order, evidence, voice, purpose.
Structure gives AI away
An XGBoost classifier on these features hits 97.0 macro-F1 and names the source model 68.6% of the time.

From the paper