The /explorer/ SPA endpoints were the source of all spider-trap traffic. A robots.txt Disallow is a better fix than serving 404s — it prevents crawlers from entering the infinite URL tree in the first place, avoids serving large numbers of 404s that hurt SEO, and doesn't break legitimate deep links. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|---|---|---|
| .. | ||
| changelog.d | ||
| explanation | ||
| how-to | ||
| reference | ||
| tutorials | ||
| index.md | ||
| quartz.config.ts | ||
| quartz.layout.ts | ||