volcano-expedition.com
robots.txt

Robots Exclusion Standard data for volcano-expedition.com

Resource Scan

Scan Details

Site Domain volcano-expedition.com
Base Domain volcano-expedition.com
Scan Status Ok
Last Scan2024-11-02T15:42:24+00:00
Next Scan 2024-11-09T15:42:24+00:00

Last Scan

Scanned2024-11-02T15:42:24+00:00
URL http://volcano-expedition.com/robots.txt
Domain IPs 78.46.4.63
Response IP 78.46.4.63
Found Yes
Hash b34a9ffc20e0645a3bad3c3dee5d4f882848374fd343d88d28ce506e9f0b95ff
SimHash 481c4a31cf81

Groups

*

Rule Path
Disallow /tmp/
Disallow /typo3temp/
Disallow /api/
Disallow /*-showMore.html
Disallow /*/latlng/*/*/*/*.html
Disallow /place/*/archive/*
Disallow /*/place/*/archive/*
Disallow /*/lugar/*/archive/*
Disallow /de/ort/*/archive/*
Disallow /bluehour/
Allow /api/getEarthquake.php

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /*.html$
Allow /*/$
Disallow /tmp/*
Allow /api/
Disallow /*-showMore.html$

Other Records

Field Value
crawl-delay 0.5

twitterbot

Rule Path
Allow /*.html$

slurp

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /*.html$

Comments

  • Disallow: /uploads/
  • Disallow: /fileadmin/
  • Disallow: /*/archive/*
  • Disallow: /*/archive/*
  • Yandex
  • Allow: /ru/
  • Disallow: /