Log In Sign Up

domains.tntcode.com
robots.txt

Robots Exclusion Standard data for domains.tntcode.com

Resource Scan

Scan Details

Site Domain domains.tntcode.com
Base Domain tntcode.com
Scan Status Ok
Last Scan2024-09-22T01:11:29+00:00
Next Scan 2024-10-22T01:11:29+00:00

Last Scan

Scanned2024-09-22T01:11:29+00:00
URL https://domains.tntcode.com/robots.txt
Domain IPs 104.26.0.125, 104.26.1.125, 172.67.73.60, 2606:4700:20::681a:17d, 2606:4700:20::681a:7d, 2606:4700:20::ac43:493c
Response IP 104.26.1.125
Found Yes
Hash 11b207472ab08048d6b8628525dc6d9c7bf62bd23d2452fbf64a5a6a899886de
SimHash a2929103ec14

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Comments

  • a google validator shows warning icon next to Crawl-delay and says "rule ignored by google bot"
  • left it on hopefully others will use it
  • remove it if causing unwanted crawling blocks
Back to top