volcanoadventures.com
robots.txt

Robots Exclusion Standard data for volcanoadventures.com

Resource Scan

Scan Details

Site Domain volcanoadventures.com
Base Domain volcanoadventures.com
Scan Status Ok
Last Scan2024-11-05T02:26:04+00:00
Next Scan 2024-11-12T02:26:04+00:00

Last Scan

Scanned2024-11-05T02:26:04+00:00
URL https://volcanoadventures.com/robots.txt
Domain IPs 78.46.167.85
Response IP 78.46.167.85
Found Yes
Hash 01cb0fe32ee42e209363cb1598156bd9065692e8a7041925f870b7f649447296
SimHash 485452706588

Groups

*

Rule Path
Disallow /tmp/
Disallow /typo3temp/
Disallow /api/
Disallow /*-showMore.html
Disallow /*/latlng/*/*/*/*.html
Disallow /place/*/archive/*
Disallow /*/place/*/archive/*
Disallow /*/lugar/*/archive/*
Disallow /de/ort/*/archive/*
Disallow /*quake*/p1*.html
Disallow /*quake*/p2*.html
Disallow /*quake*/p3.html
Disallow /*quake*/p4.html
Disallow /*quake*/p5.html
Disallow /*quake*/p6.html
Disallow /*quake*/p7.html
Disallow /*quake*/p8.html
Disallow /*quake*/p9.html
Disallow /*sism*/p1*.html
Disallow /*sism*/p2*.html
Disallow /*sism*/p3.html
Disallow /*sism*/p4.html
Disallow /*sism*/p5.html
Disallow /*sism*/p6.html
Disallow /*sism*/p7.html
Disallow /*sism*/p8.html
Disallow /*sism*/p9.html
Disallow /bluehour/
Allow /api/getEarthquake.php

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /*.html$
Allow /*/$
Disallow /tmp/*
Allow /api/
Disallow /*-showMore.html$

Other Records

Field Value
crawl-delay 0.5

twitterbot

Rule Path
Allow /*.html$

slurp

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /*.html$

Comments

  • Disallow: /uploads/
  • Disallow: /fileadmin/
  • Disallow: /*/archive/*
  • Disallow: /*/archive/*
  • Yandex
  • Allow: /ru/
  • Disallow: /