Robots.txt Generator
Build a correct robots.txt with presets for common setups. Free, private, no sign-up — runs in your browser.
Robots.txt Generator
Generate a valid robots.txt visually: allow or block all crawlers, protect specific folders, target individual bots (including AI crawlers like GPTBot), and declare your sitemap.
About this Robots.txt Generator
robots.txt lives at your domain root and tells crawlers which paths not to fetch. The presets cover the common cases, including the modern need to opt out of AI training crawlers (GPTBot, CCBot, Google-Extended and others) while staying fully indexed in search. Remember: robots.txt controls crawling, not indexing — use a noindex meta tag to keep a page out of results.
Frequently asked questions
Where do I put robots.txt?
Will Disallow remove a page from Google?
Should I set Crawl-delay?
More SEO Tools.
Meta Tag Generator
Generate complete meta, Open Graph and Twitter Card tags. Free, private, no…
Schema Markup Generator
JSON-LD structured data for articles, FAQs, products and more. Free, private, no…
Sitemap Generator
Crawl your website and generate a complete XML sitemap automatically. Free, private,…
Age Calculator
Exact age in years, months, days — plus next-birthday countdown. Free, private,…
