tldrsec.com
robots.txt

Robots Exclusion Standard data for tldrsec.com

Resource Scan

Scan Details

Site Domain tldrsec.com
Base Domain tldrsec.com
Scan Status Ok
Last Scan2025-10-15T05:18:55+00:00
Next Scan 2025-11-14T05:18:55+00:00

Last Scan

Scanned2025-10-15T05:18:55+00:00
URL https://tldrsec.com/robots.txt
Domain IPs 104.26.8.163, 104.26.9.163, 172.67.73.46, 2606:4700:20::681a:8a3, 2606:4700:20::681a:9a3, 2606:4700:20::ac43:492e
Response IP 104.26.9.163
Found Yes
Hash cb30629dcb8693ae55774fd8689e361d7e585e95b688bd41bd1d27a9a0f6075b
SimHash 6f1cdcf2ab10

Groups

amazonbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /nogooglebot/

*

Rule Path
Disallow /login

adsbot-google

Rule Path
Disallow /login

nutch

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /login

Other Records

Field Value
crawl-delay 10

ahrefssiteaudit

Rule Path
Disallow /login

Other Records

Field Value
crawl-delay 10

mj12bot

Rule Path
Disallow /login

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://tldrsec.com/sitemap.xml