soc.pyrox.dev
robots.txt

Robots Exclusion Standard data for soc.pyrox.dev

Resource Scan

Scan Details

Site Domain soc.pyrox.dev
Base Domain pyrox.dev
Scan Status Ok
Last Scan2024-06-07T00:37:03+00:00
Next Scan 2024-06-08T00:37:03+00:00

Last Scan

Scanned2024-06-07T00:37:03+00:00
URL https://soc.pyrox.dev/robots.txt
Domain IPs 2a01:4ff:f0:98bf::1, 5.161.140.5
Response IP 5.161.140.5
Found Yes
Hash 2df1c5fddbfa4efb69ec7e87f648e15c8e9f7a762835b568fd3ca2824012cfea
SimHash 2a3ccbc181f4

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

storebot-google

Rule Path
Disallow /

googleother

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

Comments

  • explicit disallows because some bots are assholes that need that