litquotes.com
robots.txt

Robots Exclusion Standard data for litquotes.com

Resource Scan

Scan Details

Site Domain litquotes.com
Base Domain litquotes.com
Scan Status Ok
Last Scan2025-10-26T20:22:05+00:00
Next Scan 2025-11-02T20:22:05+00:00

Last Scan

Scanned2025-10-26T20:22:05+00:00
URL https://litquotes.com/robots.txt
Domain IPs 104.21.36.126, 172.67.194.92, 2606:4700:3035::ac43:c25c, 2606:4700:3037::6815:247e
Response IP 172.67.194.92
Found Yes
Hash 4273da113ccb8ceb35c4a27e367f51e22d001edb394a505aa57dc31f116027cd
SimHash 7bb16b444f9d

Groups

*

Rule Path
Disallow /cgi-bin

mozilla/5.0 (compatible;contxbot/1.0)

Rule Path
Disallow

Other Records

Field Value
sitemap http://www.litquotes.com/sitemap.xml

Comments

  • All robots will spider the domain
  • Disallow directory /cgi-bin/