Log In Sign Up

hermetic.com
robots.txt

Robots Exclusion Standard data for hermetic.com

Resource Scan

Scan Details

Site Domain hermetic.com
Base Domain hermetic.com
Scan Status Ok
Last Scan2024-09-28T20:02:22+00:00
Next Scan 2024-10-05T20:02:22+00:00

Last Scan

Scanned2024-09-28T20:02:22+00:00
URL https://hermetic.com/robots.txt
Domain IPs 104.21.39.167, 172.67.146.198, 2606:4700:3033::ac43:92c6, 2606:4700:3035::6815:27a7
Response IP 104.21.39.167
Found Yes
Hash cda8dedafced8e2619aeae9b70f42eae754a6692511e5e2e2fb3a05a229ebfc9
SimHash 6a1dd8158115

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ia_archiver-web.archive.org
archive.org_bot
ia_archiver
googlebot
msnbot
bingbot
baiduspider
slurp
yandex
duckduckgo
mastodon
applebot
feedly
googlebot-image
facebookexternalhit
yandexbot
sogou
stractbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://hermetic.com/?do=sitemap
Back to top

Comments

  • Block OpenAI
  • Block Google Bard AI
  • Block Common Crawl
Back to top