unite.geocaching.com
robots.txt

Robots Exclusion Standard data for unite.geocaching.com

Resource Scan

Scan Details

Site Domain unite.geocaching.com
Base Domain geocaching.com
Scan Status Ok
Last Scan2024-06-25T03:04:37+00:00
Next Scan 2024-07-25T03:04:37+00:00

Last Scan

Scanned2024-06-25T03:04:37+00:00
URL https://unite.geocaching.com/robots.txt
Redirect https://www.geocaching.com/robots.txt
Redirect Domain www.geocaching.com
Redirect Base geocaching.com
Domain IPs 63.251.163.200
Redirect IPs 63.251.163.200
Response IP 63.251.163.200
Found Yes
Hash a9aca420aefced01f7989ba4a71d75c18a39be36e7ba8beede1f056b9de480b9
SimHash 641cce5563b7

Groups

dotbot

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

twiceler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

scoutjet

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

*

Rule Path
Disallow /iis/*
Disallow /login/*
Disallow /admin/*
Disallow /map/*
Disallow /play/results
Disallow /email/*
Disallow /my/*
Disallow /gps/*
Disallow /datastore/*
Disallow /api/*
Disallow /bait.asp
Disallow /hotels/*

Other Records

Field Value
sitemap http://www.geocaching.com/sitemaps/sitemap-index.xml

Comments

  • Disallow all unnecessary content from the search