Log In Sign Up

hrmtc.com
robots.txt

Robots Exclusion Standard data for hrmtc.com

Resource Scan

Scan Details

Site Domain hrmtc.com
Base Domain hrmtc.com
Scan Status Ok
Last Scan2024-06-07T15:15:20+00:00
Next Scan 2024-06-14T15:15:20+00:00

Last Scan

Scanned2024-06-07T15:15:20+00:00
URL https://hrmtc.com/robots.txt
Domain IPs 104.21.95.102, 172.67.144.72, 2606:4700:3034::6815:5f66, 2606:4700:3036::ac43:9048
Response IP 104.21.95.102
Found Yes
Hash 8f6156256e2eac7302d5204177635ff6240e1d9ba5cc4736ff4123de0f672a6e
SimHash 2a3cd815813f

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ia_archiver-web.archive.org
archive.org_bot
ia_archiver
googlebot
msnbot
bingbot
baiduspider
slurp
yandex
duckduckgo
mastodon
applebot
feedly
googlebot-image
facebookexternalhit
yandexbot
sogou
stractbot
pinterestbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • Block OpenAI
  • Block Google Bard AI
  • Block Common Crawl
Back to top