thelchat.net
robots.txt

Robots Exclusion Standard data for thelchat.net

Resource Scan

Scan Details

Site Domain thelchat.net
Base Domain thelchat.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-04T08:07:40+00:00
Next Scan 2024-08-03T08:07:40+00:00

Last Successful Scan

Scanned2024-04-06T08:05:42+00:00
URL https://thelchat.net/robots.txt
Domain IPs 104.21.5.92, 172.67.133.63, 2606:4700:3031::6815:55c, 2606:4700:3031::ac43:853f
Response IP 172.67.133.63
Found Yes
Hash 819f4d9a0008c6cdbb6be67ce351c50bf759e2f55df2d98152463be5480b640b
SimHash 6cb9d10086d6

Groups

neevabot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

yahoo-blogs/v3.9

Rule Path
Disallow /

*

Rule Path
Disallow
Disallow /cgi-bin/

Other Records

Field Value
crawl-delay 120

ahrefs

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

amazon*

Rule Path
Disallow /

amazon data services

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /do-not-crawl/

Other Records

Field Value
sitemap https://thelchat.net/forum