folkstreams.net
robots.txt

Robots Exclusion Standard data for folkstreams.net

Resource Scan

Scan Details

Site Domain folkstreams.net
Base Domain folkstreams.net
Scan Status Ok
Last Scan2025-11-28T19:12:45+00:00
Next Scan 2025-12-05T19:12:45+00:00

Last Scan

Scanned2025-11-28T19:12:45+00:00
URL https://folkstreams.net/robots.txt
Redirect https://www.folkstreams.net/robots.txt
Redirect Domain www.folkstreams.net
Redirect Base folkstreams.net
Domain IPs 104.21.56.69, 172.67.179.206, 2606:4700:3030::ac43:b3ce, 2606:4700:3035::6815:3845
Redirect IPs 104.21.56.69, 172.67.179.206, 2606:4700:3030::ac43:b3ce, 2606:4700:3035::6815:3845
Response IP 104.21.56.69
Found Yes
Hash f9b221ae4cf485e2bfc76c6568a6ba4819bf7e6ae553c8a8aed882030db2dd99
SimHash 09744e925a24

Groups

*

Rule Path
Disallow /admin/
Disallow /contrib/
Disallow /media/
Disallow /mockup/
Disallow /scripts/
Disallow /sources/
Disallow /styles/
Disallow /wiki/

googlebot

Rule Path
Disallow /pub/search2.php
Allow /wiki/

mediapartners-google

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

twiceler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /vafp/admin/
Disallow /vafp/classes/
Disallow /vafp/css/
Disallow /vafp/images/
Disallow /vafp/img/
Disallow /vafp/media/
Disallow /vafp/templates/
Disallow /vafp/templates_c/
Disallow /vafp/video/
Disallow /publicdev/
Disallow /admin3/

Comments

  • adsense search engine has been seen in our logs
  • 3000 per year
  • Cuil
  • VAFP subsite