why-site.com
robots.txt

Robots Exclusion Standard data for why-site.com

Resource Scan

Scan Details

Site Domain why-site.com
Base Domain why-site.com
Scan Status Ok
Last Scan2026-01-09T22:23:50+00:00
Next Scan 2026-02-08T22:23:50+00:00

Last Scan

Scanned2026-01-09T22:23:50+00:00
URL https://why-site.com/robots.txt
Domain IPs 77.72.1.29
Response IP 77.72.1.29
Found Yes
Hash 5f9b674969f8d19e3d3ae2e4e6d074f464814d35b109e82fd462a521755da7e9
SimHash 68005a128bb2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://why-site.com/wp-sitemap.xml