earthquakes.volcanodiscovery.com
robots.txt

Robots Exclusion Standard data for earthquakes.volcanodiscovery.com

Resource Scan

Scan Details

Site Domain earthquakes.volcanodiscovery.com
Base Domain volcanodiscovery.com
Scan Status Ok
Last Scan2024-05-29T00:55:53+00:00
Next Scan 2024-06-05T00:55:53+00:00

Last Scan

Scanned2024-05-29T00:55:53+00:00
URL https://earthquakes.volcanodiscovery.com/robots.txt
Domain IPs 104.26.10.77, 104.26.11.77, 172.67.73.194, 2606:4700:20::681a:a4d, 2606:4700:20::681a:b4d, 2606:4700:20::ac43:49c2
Response IP 104.26.11.77
Found Yes
Hash 01cb0fe32ee42e209363cb1598156bd9065692e8a7041925f870b7f649447296
SimHash 485452706588

Groups

*

Rule Path
Disallow /tmp/
Disallow /typo3temp/
Disallow /api/
Disallow /*-showMore.html
Disallow /*/latlng/*/*/*/*.html
Disallow /place/*/archive/*
Disallow /*/place/*/archive/*
Disallow /*/lugar/*/archive/*
Disallow /de/ort/*/archive/*
Disallow /*quake*/p1*.html
Disallow /*quake*/p2*.html
Disallow /*quake*/p3.html
Disallow /*quake*/p4.html
Disallow /*quake*/p5.html
Disallow /*quake*/p6.html
Disallow /*quake*/p7.html
Disallow /*quake*/p8.html
Disallow /*quake*/p9.html
Disallow /*sism*/p1*.html
Disallow /*sism*/p2*.html
Disallow /*sism*/p3.html
Disallow /*sism*/p4.html
Disallow /*sism*/p5.html
Disallow /*sism*/p6.html
Disallow /*sism*/p7.html
Disallow /*sism*/p8.html
Disallow /*sism*/p9.html
Disallow /bluehour/
Allow /api/getEarthquake.php

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /*.html$
Allow /*/$
Disallow /tmp/*
Allow /api/
Disallow /*-showMore.html$

Other Records

Field Value
crawl-delay 0.5

twitterbot

Rule Path
Allow /*.html$

slurp

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bingbot

Rule Path
Allow /

yandex

Rule Path
Allow /*.html$

Comments

  • Disallow: /uploads/
  • Disallow: /fileadmin/
  • Disallow: /*/archive/*
  • Disallow: /*/archive/*
  • Yandex
  • Allow: /ru/
  • Disallow: /