Log In Sign Up

my.embarkvet.com
robots.txt

Robots Exclusion Standard data for my.embarkvet.com

Resource Scan

Scan Details

Site Domain my.embarkvet.com
Base Domain embarkvet.com
Scan Status Ok
Last Scan2025-09-12T21:45:22+00:00
Next Scan 2025-09-26T21:45:22+00:00

Last Scan

Scanned2025-09-12T21:45:22+00:00
URL https://my.embarkvet.com/robots.txt
Domain IPs 34.194.62.92, 54.158.208.137
Response IP 54.158.208.137
Found Yes
Hash 6867b525b35eec06a0495db2d3e28ec59a43303e34b5b996a7bd8fd4725da8dc
SimHash 80918362cbf6

Groups

*

Rule Path
Allow /dog/
Allow /dogs/
Allow /vetreport/
Allow /vet-report/
Allow /related/
Disallow /
Disallow /dog/*?compareMix=*

twitterbot

Rule Path
Allow /breed-game/
Allow /dog/
Allow /dogs/
Allow /ogimage/
Allow /ogimage/v2
Allow /ogimage-breed-game
Disallow /
Disallow /dog/*?compareMix=*

mj12bot

Rule Path
Disallow /

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15
Back to top

Comments

  • Tell Mozilla bots to crawl at a slower rate (per second)
Back to top