Log In Sign Up

au.boats.com
robots.txt

Robots Exclusion Standard data for au.boats.com

Resource Scan

Scan Details

Site Domain au.boats.com
Base Domain boats.com
Scan Status Ok
Last Scan2024-05-09T10:35:12+00:00
Next Scan 2024-05-23T10:35:12+00:00

Last Scan

Scanned2024-05-09T10:35:12+00:00
URL https://au.boats.com/robots.txt
Domain IPs 13.226.228.113, 13.226.228.20, 13.226.228.30, 13.226.228.47
Response IP 18.165.171.54
Found Yes
Hash e069030300a52a0432eaf686de8a22838f3e5a428a8fc89aa92f22346d60a479
SimHash 1d1019b8cbe7

Groups

*

Rule Path
Disallow /my-boats/
Disallow /sites/
Disallow /listing/
Allow /transport/
Disallow /transport/?b=*
Allow /boat-transport/index.jsp
Disallow /boat-transport/index.jsp?yw_country=*
Disallow /boats-for-sale/*sort%3D
Disallow /boats-for-sale/*currency%3D
Disallow /boats-for-sale/*length%3D
Disallow /boats-for-sale/*price%3D
Disallow /boats-for-sale/*year%3D
Disallow /boats-for-sale/*condition%3Dall-in-stock
Disallow /boats-for-sale/*condition%3Dnew-in-stock
Disallow /boats-for-sale/*condition%3Dnew-models
Disallow /boats-for-sale/*hull-material%3D
Disallow /boats-for-sale/*fuel-type%3D
Disallow /boats-for-sale/*created%3D
Disallow /boats-for-sale/*media-type%3D
Disallow /boats-for-sale/*activity%3D
Disallow *keyword%3D
Disallow *type-class%3D
Disallow *typeClass%3D
Disallow *?p=
Disallow *?locale=
Disallow *?redirect=
Disallow /boats.serv*
Disallow /engines/engine_details_frame.jsp*
Disallow /dealer/
Disallow /boats/contact/email_seller.html
Disallow /252108799/Boats*
Back to top

Other Records

Field Value
sitemap https://au.boats.com/sitemap.xml
Back to top

Comments

  • Prod Test
  • The use of robots, crawlers, spiders, rovers, scrapers or other automated
  • means to access or mine data from any Boats Group hosted web sites
  • (including, but not limited to, boats.com, yachtworld.com, boatwizard.com,
  • and soldboats.com) without the express written permission of Boats Group
  • is strictly prohibited.
  • Boats Group may permit access to Boats Group hosted web sites for the
  • limited purpose of including content in publicly available search engines,
  • but only if those search engines are not specific to the marine industry
  • and do not frame or otherwise obscure any Boats Group hosted web sites.
  • Please read our Terms of Use at
  • https://www.boatsgroup.com/terms-of-use/ for more information.
Back to top