cookiesandcups.com
robots.txt

Robots Exclusion Standard data for cookiesandcups.com

Resource Scan

Scan Details

Site Domain cookiesandcups.com
Base Domain cookiesandcups.com
Scan Status Ok
Last Scan2024-05-14T23:53:34+00:00
Next Scan 2024-05-21T23:53:34+00:00

Last Scan

Scanned2024-05-14T23:53:34+00:00
URL https://cookiesandcups.com/robots.txt
Domain IPs 104.26.6.219, 104.26.7.219, 172.67.72.146
Response IP 104.26.6.219
Found Yes
Hash dc39ba86aa43e183596d0b2859d6f310a68aebb77658d28b5c5c64ce672f07e2
SimHash 68019a0003b2

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://cookiesandcups.com/sitemap_index.xml