hedgeapple.com
robots.txt

Robots Exclusion Standard data for hedgeapple.com

Resource Scan

Scan Details

Site Domain hedgeapple.com
Base Domain hedgeapple.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-08-30T19:35:37+00:00
Next Scan 2024-11-28T19:35:37+00:00

Last Successful Scan

Scanned2024-01-11T19:05:45+00:00
URL https://hedgeapple.com/robots.txt
Domain IPs 2600:9000:269b:2c00:19:a31e:ec0:93a1, 2600:9000:269b:7c00:19:a31e:ec0:93a1, 2600:9000:269b:8c00:19:a31e:ec0:93a1, 2600:9000:269b:a400:19:a31e:ec0:93a1, 2600:9000:269b:ca00:19:a31e:ec0:93a1, 2600:9000:269b:e00:19:a31e:ec0:93a1, 2600:9000:269b:e200:19:a31e:ec0:93a1, 2600:9000:269b:ea00:19:a31e:ec0:93a1, 52.222.174.117, 52.222.174.33, 52.222.174.6, 52.222.174.86
Response IP 3.160.196.9
Found Yes
Hash 65899509c3a8533f66d1f89c695fb5d96d498ae57f48133c74dd426a6e1fa14a
SimHash 4805dd11d675

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10

ahrefsbot
ahrefssiteaudit
bingbot
facebot
googlebot
googlebot-image
mediapartners-google
msnbot
msnbot-media
rogerbot
semrushbot
slurp
twitterbot
yahoo-blogs
yahoo-mmcrawler

Rule Path
Allow /
Disallow /cart/
Disallow /checkout/
Disallow /accounts/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://hedgeapple.com/sitemap.xml

Comments

  • Disallow all
  • But allow only important bots