Log In Sign Up

allthingsplc.info
robots.txt

Robots Exclusion Standard data for allthingsplc.info

Resource Scan

Scan Details

Site Domain allthingsplc.info
Base Domain allthingsplc.info
Scan Status Ok
Last Scan2024-11-01T07:56:14+00:00
Next Scan 2024-12-01T07:56:14+00:00

Last Scan

Scanned2024-11-01T07:56:14+00:00
URL https://allthingsplc.info/robots.txt
Domain IPs 35.164.160.26
Response IP 35.164.160.26
Found Yes
Hash a01efdba9d4568aba6ca4e89b7302a41d2f103daaf95ca25986e69db4a42ddc4
SimHash 5234582ac013

Groups

deepcrawl

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /
Back to top

Comments

  • disable deepcrawl....
  • disable ahrefs bot
  • disable petalbot
  • disable AI bot traffic
Back to top