hiking-site.nl
robots.txt
Robots Exclusion Standard data for hiking-site.nl
Resource Scan
Scan Details
| Site Domain | hiking-site.nl |
| Base Domain | hiking-site.nl |
| Scan Status | Ok |
| Last Scan | 2024-11-13T13:05:59+00:00 |
| Next Scan | 2024-11-20T13:05:59+00:00 |
Last Scan
| Scanned | 2024-11-13T13:05:59+00:00 |
| URL | https://hiking-site.nl/robots.txt |
| Domain IPs | 158.106.139.211 |
| Response IP | 158.106.139.211 |
| Found | Yes |
| Hash | 990caac9ea94ee4e21c398497933a563bafe0a3018b5e9c7a3fbd3538a59eebb |
| SimHash | da0a496dc9ec |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /ads/ |
| Disallow | /css/ |
| Disallow | /cgi-bin/ |
| Disallow | /includes/ |
| Disallow | /phorm/ |
| Disallow | /ads |
| Disallow | /css |
| Disallow | /cgi-bin |
| Disallow | /graphics |
| Disallow | /includes |
| Disallow | /phorm |
| Disallow | /rkoome |
| Disallow | /cc.php |
| Disallow | /listmessenger/ |
| Disallow | /listmessenger/public/ |
anawave
emailcollector
emailsiphon
extractorpro
flashsite
go-get-it
grab-a-site
hotcargo
httploader
linkwalker
memoweb
nearsite
netattache
radview
radview/httploader
second site
secondsite
sitesnagger
spidybot
teleport
teleport pro
visual web
visualweb
wbi_client
webcompass
webcopy
webdownloader
webretriever
websnake
webvcr
webwhacker
voyager
webzip
wget
shopwiki
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://hiking-site.nl/sitemap.xml |
| sitemap | https://hiking-site.nl/prikbord/sitemap.php |