der-farang.com
robots.txt

Robots Exclusion Standard data for der-farang.com

Resource Scan

Scan Details

Site Domain der-farang.com
Base Domain der-farang.com
Scan Status Ok
Last Scan2024-11-13T19:23:14+00:00
Next Scan 2024-11-20T19:23:14+00:00

Last Scan

Scanned2024-11-13T19:23:14+00:00
URL https://der-farang.com/robots.txt
Domain IPs 104.21.34.127, 172.67.161.64, 2606:4700:3033::6815:227f, 2606:4700:3037::ac43:a140
Response IP 172.67.161.64
Found Yes
Hash aff190388207e0ebcb52c430105b780054db9bf77b73a50f710fc40a2649d5db
SimHash 2007d00bad33

Groups

*

Rule Path
Disallow /tmp/
Disallow /.externalToolBuilders/
Disallow /.settings/
Disallow /applications/
Disallow /docs/
Disallow /lang/
Disallow /library/
Disallow /tests/
Disallow /log/
Disallow /de/comment/list/box-preview/
Disallow /en/
Disallow /th/
Allow /

mj12bot

Rule Path
Disallow /

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • add one sitemap per active language
  • Sitemap: http://domain.com/en/sitemap.xml
  • Sitemap: http://domain.com/en/sitemap.xml