Log In Sign Up

readerscave.com
robots.txt

Robots Exclusion Standard data for readerscave.com

Resource Scan

Scan Details

Site Domain readerscave.com
Base Domain readerscave.com
Scan Status Ok
Last Scan2024-11-11T02:51:55+00:00
Next Scan 2024-11-18T02:51:55+00:00

Last Scan

Scanned2024-11-11T02:51:55+00:00
URL https://readerscave.com/robots.txt
Redirect https://www.readerscave.com/robots.txt
Redirect Domain www.readerscave.com
Redirect Base readerscave.com
Domain IPs 104.21.16.231, 172.67.216.209, 2606:4700:3031::6815:10e7, 2606:4700:3031::ac43:d8d1
Redirect IPs 104.21.16.231, 172.67.216.209, 2606:4700:3031::6815:10e7, 2606:4700:3031::ac43:d8d1
Response IP 172.67.216.209
Found Yes
Hash e4a1ac178245e85b2649831aec00df2a9b3a89164579b830fbd97c3fc581b3dd
SimHash c1144ae0f23d

Groups

ninjabot

Rule Path
Allow /

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

duggmirror

Rule Path
Disallow /

hypestat

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /trackback/
Disallow /comments/
Disallow /author/
Disallow /tag/*
Disallow /tag/
Disallow */trackback/
Disallow */feed/
Disallow */comments/
Allow /wp-content/uploads/
Back to top

Other Records

Field Value
sitemap https://www.readerscave.com/sitemap_index.xml
Back to top

Comments

  • Ninja Bot
  • Google Image
  • Google Adsense
  • Google Ads Bot
  • Google Mobile
  • digg mirror
  • Hypestat
  • global
Back to top