topcompilation.com
robots.txt

Robots Exclusion Standard data for topcompilation.com

Resource Scan

Scan Details

Site Domain topcompilation.com
Base Domain topcompilation.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-11-09T08:56:17+00:00
Next Scan 2026-02-07T08:56:17+00:00

Last Successful Scan

Scanned2024-03-26T15:51:36+00:00
URL https://topcompilation.com/robots.txt
Domain IPs 104.21.20.239, 172.67.194.222, 2606:4700:3031::6815:14ef, 2606:4700:3034::ac43:c2de
Response IP 104.21.20.239
Found Yes
Hash 364d3190b97a1478b577475da56c3add1b804850117d13ac112f4a5808215c91
SimHash 7f1d5420ca90

Groups

*

Rule Path
Disallow /wp-admin/

amazonbot
petalbot
semrushbot
mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap /sitemap_index.xml

Comments

  • Sitemap files