pro.homeadvisor.com
robots.txt

Robots Exclusion Standard data for pro.homeadvisor.com

Resource Scan

Scan Details

Site Domain pro.homeadvisor.com
Base Domain homeadvisor.com
Scan Status Ok
Last Scan2025-09-19T08:04:01+00:00
Next Scan 2025-10-03T08:04:01+00:00

Last Scan

Scanned2025-09-19T08:04:01+00:00
URL https://pro.homeadvisor.com/robots.txt
Domain IPs 104.17.224.8, 104.17.225.8
Response IP 104.17.224.8
Found Yes
Hash 774e71772ff01db096dd9fee62e89eb3b88071e4dad9c9379378f0e7b15a2587
SimHash c522d83417c1

Groups

omniexplorer_bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

*

Rule Path
Disallow /classes/
Disallow /cgibin/
Disallow /templates/
Disallow /rfw/
Disallow /ManualFilterTools/
Disallow /sas/
Disallow /allconnect/
Disallow /sitesearch/
Disallow /cost/search/
Disallow /cost/embed/
Disallow /xmd/
Disallow /m/
Disallow /rfs/glossary/
Disallow /rfs/serviceprofessional/xmprofile/
Disallow /eng/
Disallow /sem/
Disallow /servlet/RateContentServlet*
Disallow /smod/
Disallow /lo/
Disallow /ld/
Disallow /designmine/all/search/
Disallow /designmine/photo/admin
Disallow /partner-listings*
Disallow /partner-profile/
Disallow /designmine/photos/*/report/
Disallow /prematch/*
Disallow /r/search/
Disallow /r/?s=
Disallow /tloc/
Disallow /clp/
Disallow /c.*
Disallow /rfs/
Disallow /servlet/
Disallow /prodredir/
Disallow /emc.*
Disallow /rated.*
Disallow /usp/
Disallow /biz/

Comments

  • robots.txt for http://pro.homeadvisor.com/

Warnings

  • `noindex` is not a known field.