dealmagpie.com
robots.txt

Robots Exclusion Standard data for dealmagpie.com

Resource Scan

Scan Details

Site Domain dealmagpie.com
Base Domain dealmagpie.com
Scan Status Ok
Last Scan2024-09-21T17:19:21+00:00
Next Scan 2024-09-28T17:19:21+00:00

Last Scan

Scanned2024-09-21T17:19:21+00:00
URL https://dealmagpie.com/robots.txt
Redirect https://www.dealmagpie.com/robots.txt
Redirect Domain www.dealmagpie.com
Redirect Base dealmagpie.com
Domain IPs 18.155.68.101, 18.155.68.11, 18.155.68.14, 18.155.68.36
Redirect IPs 18.155.68.101, 18.155.68.11, 18.155.68.14, 18.155.68.36, 2600:9000:23d2:1a00:10:9fc4:d600:93a1, 2600:9000:23d2:3800:10:9fc4:d600:93a1, 2600:9000:23d2:800:10:9fc4:d600:93a1, 2600:9000:23d2:8800:10:9fc4:d600:93a1, 2600:9000:23d2:aa00:10:9fc4:d600:93a1, 2600:9000:23d2:c200:10:9fc4:d600:93a1, 2600:9000:23d2:ea00:10:9fc4:d600:93a1, 2600:9000:23d2:ec00:10:9fc4:d600:93a1
Response IP 18.155.68.101
Found Yes
Hash b0c6af4a21092e70257f50e3f02f234c0f2d54dff450fedc4d9cf1b576006eb4
SimHash b7574e0a58da

Groups

mj12bot

Rule Path
Disallow /

comscore

Rule Path
Disallow /

admantx

Rule Path
Disallow /

python

Rule Path
Disallow /

panscient

Rule Path
Disallow /

semrush

Rule Path
Disallow /

guzzlehttp

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

obot

Rule Path
Disallow /

etaospider

Rule Path
Disallow /