robots.txt Generator

Generate a robots.txt file for your website with presets and custom rules.

Presets

Custom rules

User agent

Directive

Path

Sitemap URL

Crawl-delay (seconds)

Delay between each robot request (not supported by Google)

robots.txt file preview

User-agent: *
Allow: /

Other tools you might like