Question 1

What is a robots.txt file?

Accepted Answer

A robots.txt file tells search engine crawlers which pages or sections of your site they can or cannot crawl. It is placed at the root of your website.

Question 2

How do I validate my robots.txt?

Accepted Answer

Paste your robots.txt content into the input area and click Validate. The tool will check for syntax errors, analyze rules, and test URL access.

Question 3

What errors does this validator detect?

Accepted Answer

It detects syntax errors, invalid directives, missing line breaks, conflicting rules, unreachable paths, and warns about common misconfigurations.

Question 4

Can I test if a URL is blocked?

Accepted Answer

Yes. Enter a URL path and select a user-agent to test whether it is allowed or blocked by your robots.txt rules.