robots.txt generator
Builds a robots.txt file. Add one rule block per user-agent you want to target — "*" applies to every crawler.
What is robots.txt generator?
robots.txt tells search engine crawlers which parts of a site they're allowed to index — a courtesy request, not a security measure, but the standard way to keep crawlers out of admin areas or duplicate content.
How to use it
Add a rule group per user-agent (or "*" for all crawlers), list disallowed and allowed paths, and optionally point to your sitemap.
Example
User-agent: *
Disallow: /admin/
Sitemap: https://example.com/sitemap.xml