/robots.txt
Robots.txt Editor & Inspector
Validate directive syntax, generate rules for AI crawlers, and test path permissions.
Fetch live
https://
Validation Diagnostics
Checks run as you type — directive order, path syntax, and unrecognized rules.
Generate Rules
Pick which crawlers to allow or block, then insert rules into the editor.
AI & search crawlers12 selected
Path scope
Test Access
Check a path against rules currently in the editor using longest-match precedence.
User-agent
Path
Sitemaps found