Log In Sign Up

interglacial.com
robots.txt

Robots Exclusion Standard data for interglacial.com

Resource Scan

Scan Details

Site Domain interglacial.com
Base Domain interglacial.com
Scan Status Ok
Last Scan2025-06-08T06:19:12+00:00
Next Scan 2025-07-08T06:19:12+00:00

Last Scan

Scanned2025-06-08T06:19:12+00:00
URL https://interglacial.com/robots.txt
Domain IPs 188.166.68.29
Response IP 188.166.68.29
Found Yes
Hash 61897c2ccabda469798900522479f20aae4cec7a409495b4662db0b9b2e8a5d4
SimHash 385ddd028955

Groups

zyborg

Rule Path
Disallow /

jyte_feed_fetcher

Rule Path
Disallow /

webclipping.com

Rule Path
Disallow /

wget

Rule Path
Disallow /rss/
Disallow /d/

googlebot

Rule Path
Disallow /rss/*.pl$
Disallow /*.ps$
Disallow /temp/
Disallow /clock/
Disallow /subrosa/
Disallow /photos/
Disallow /d/
Disallow /kwiki/
Disallow /rtf/rtf_book_examples/
Disallow /jerkcity_jumble/

msnbot

Rule Path
Disallow /rss/*.pl$
Disallow /*.css$
Disallow /*.exe$
Disallow /*.ps$
Disallow /temp/
Disallow /clock/
Disallow /subrosa/
Disallow /photos/
Disallow /d/
Disallow /kwiki/
Disallow /rtf/rtf_book_examples/
Disallow /jerkcity_jumble/

lmspider (lmspider@scansoft.com)

Rule Path
Disallow /

lmspider lmspider@scansoft.com

Rule Path
Disallow /

owlin bot v. 3.0

Rule Path
Disallow /

owlin bot

Rule Path
Disallow /

owlin

Rule Path
Disallow /

*

Rule Path
Disallow /rss/internet-drafts/
Disallow /rss/rfc/
Disallow /temp/
Disallow /clock/
Disallow /subrosa/
Disallow /photos/
Disallow /egads/navajo/
Disallow /d/
Disallow /kwiki/
Disallow /rtf/rtf_book_examples/
Disallow /jerkcity_jumble/
Back to top

Comments

  • See http://www.robotstxt.org/robots.txt Time-stamp: "2014-09-08 00:29:22 MDT"
  • Useless bot:
  • See http://www.kahunaburger.com/blog/archives/000117.html
  • The all-agents should always be last
  • Ethnic divisions,
  • screaming demands --
  • A little bit like you and me!
  • I guess that we are in love.
  • Stay away!
  • Stay away!
  • Stay away, you might end up like me!
  • It's the pain that keeps us alive!
  • But that beauty is all that we need to survive.
  • That damn beauty is all that we need to survive.
  • - David Byrne, "They are in Love"
Back to top