safehouse.everydayspy.com
robots.txt

Robots Exclusion Standard data for safehouse.everydayspy.com

Resource Scan

Scan Details

Site Domain safehouse.everydayspy.com
Base Domain everydayspy.com
Scan Status Ok
Last Scan2025-10-04T01:52:10+00:00
Next Scan 2025-10-18T01:52:10+00:00

Last Scan

Scanned2025-10-04T01:52:10+00:00
URL https://safehouse.everydayspy.com/robots.txt
Domain IPs 162.159.128.53, 162.159.138.52
Response IP 162.159.138.52
Found Yes
Hash 5548b077f62bfad376136c7c2eaf6090d3c0921565385819b0df63585bfb7210
SimHash 62f4d8408b89

Groups

botify
spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

mj12bot

Rule Path
Disallow /

*

Rule Path
Disallow /search?
Disallow /list?
Disallow /sign_in?
Disallow /sign_up?
Disallow /*?*filters=*
Disallow /*comments
Disallow /spaces/20842027*
Disallow /spaces/14584925*
Disallow /spaces/18258670*
Disallow /spaces/16174434*
Disallow /spaces/16945412*
Disallow /spaces/18282321*
Disallow /spaces/16814220*
Disallow /spaces/14373266*
Disallow /spaces/14000845*
Disallow /spaces/16174394*
Disallow /spaces/17026292*
Disallow /spaces/19253142*
Disallow /spaces/16174436*
Disallow /spaces/14496553*
Disallow /spaces/14584901*
Disallow /spaces/14592034*
Disallow /spaces/16822531*
Disallow /spaces/20707066*
Disallow /spaces/18283695*
Disallow /spaces/13578641*
Disallow /spaces/20012562*
Disallow /spaces/18283605*
Disallow /communities/

Other Records

Field Value
sitemap https://safehouse.everydayspy.com/sitemap.xml