Robots.txt + Sitemap Conflict Checker
Enter your domain and the tool fetches your robots.txt and sitemap, then flags every URL in the sitemap that your robots.txt would block. One of the most common silent indexing bugs at enterprise scale.
Technical SEO Intelligence
Pages blocked by robots.txt should never be in your sitemap.
If a URL is in your sitemap, you're telling Google you want it indexed. If that same URL is Disallow'd in robots.txt, Google can't crawl it to confirm. The two signals contradict each other. Google eventually ignores the sitemap entry and may demote your sitemap's reliability score. This tool finds every such conflict.
Input
We'll check
yoursite.com/robots.txt and yoursite.com/sitemap.xml.Next steps
Check the whole site, not one URL
SEORCE Technical Suite crawls every page and ranks technical issues by impact, so you fix what matters first.
Frequently asked questions
What conflict does this tool catch?
URLs listed in your XML sitemap that your robots.txt disallows from crawling. A classic contradiction that wastes crawl budget and signals a broken config.
Why does the conflict happen?
Usually a disconnect between SEO ops and dev. Someone adds a Disallow rule to robots.txt without updating the sitemap generator, or vice versa.
Will Google penalise this?
Not a direct penalty, but affected URLs sit in Search Console as "Indexed, though blocked by robots.txt" which hurts the overall health report.
How do I decide which to change?
If the URL should be indexable, remove the Disallow. If it shouldn't, remove it from the sitemap.
Can wildcard rules in robots.txt cause this?
Yes.
Disallow: /*?* accidentally blocks every URL with a query string, including ones in your sitemap.Does the tool handle sitemap-index files?
Yes. It follows nested sitemaps automatically.
What about x-robots-tag headers?
The tool checks robots.txt only. Use the Sitemap Indexability Checker for a full indexability review.
How often should this run?
After every release that changes robots.txt or the sitemap generator. Monthly for sites with active changes.
What is the fix order?
Fix robots.txt first (cheap to change), regenerate sitemap second (needs CMS cooperation).
Does Bing respect robots.txt the same way?
Yes. Bing follows the same robots.txt spec. Fix for Google, fix for Bing.
Can I block legitimate URLs on purpose?
Yes, you may want to block search result pages, filter combinations, print views. Just make sure those aren't in the sitemap.
Free · no account
SEORCE for Chrome, Edge and Firefox
Audits the page you are on: headings, meta tags, schema, indexability, and whether AI crawlers can read it. Export the result as a PDF. Free, with no account.
It audits one page at a time and does not track AI answers. About the extension
Across the whole site
One check here. Every page in SEORCE.
This tool does one job. SEORCE Technical Suite crawls your whole site with fast and secure crawlers and ranks technical issues by impact. AutoFix applies fixes for meta tags, schema, canonical tags, image alt text, headings and internal links for orphan pages on your live site, through one script tag, and the list keeps growing.
3,000+ users · Free forever plan, no card needed