Popular
Website / SEO Tools
Meta Tag Generator
Generate title, description, keywords, canonical and Open Graph meta tags.
Open toolWebsite / SEO Tools
Robots.txt Generator
Choose which crawlers to allow, list the paths to block, add your sitemap, and download a valid robots.txt.
Good to know
Frequently asked questions
Where does robots.txt go?
At the root of your domain, for example https://example.com/robots.txt. Crawlers only look there — a file in a subfolder is ignored.
Does Disallow hide a page from Google?
No. It asks crawlers not to fetch the page, but a blocked URL can still appear in results if other sites link to it. Use a noindex meta tag to keep a page out of the index.
Should I include a sitemap line?
Yes if you have one. It helps search engines find all your URLs, and it is the only directive that takes a full absolute URL.
Is my file uploaded?
No. The text is built in your browser and downloaded directly from it.