Log In Sign Up

bedrijventekoop.nl
robots.txt

Robots Exclusion Standard data for bedrijventekoop.nl

Resource Scan

Scan Details

Site Domain bedrijventekoop.nl
Base Domain bedrijventekoop.nl
Scan Status Ok
Last Scan2024-06-15T04:09:16+00:00
Next Scan 2024-07-15T04:09:16+00:00

Last Scan

Scanned2024-06-15T04:09:16+00:00
URL https://bedrijventekoop.nl/robots.txt
Redirect https://www.bedrijventekoop.nl/robots.txt
Redirect Domain www.bedrijventekoop.nl
Redirect Base bedrijventekoop.nl
Domain IPs 188.166.37.215
Redirect IPs 188.166.37.215
Response IP 188.166.37.215
Found Yes
Hash 6c1694c35d96f5bba599111ff23f2cb34755632dfae95a42031fb861c954012d
SimHash 6145d5e5a5d2

Groups

*

Rule Path
Disallow /login
Disallow /admin/
Disallow /create-password
Disallow /artikel
Disallow /en/pdf/business/
Disallow /pdf/business/
Disallow /specialist?*
Disallow /totaaloverzicht?*

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
Back to top

Other Records

Field Value
sitemap https://www.bedrijventekoop.nl/sitemap.xml
Back to top

Comments

  • www.robotstxt.org/
  • www.google.com/support/webmasters/bin/answer.py?hl=en&answer=156449
  • 304: Make sure all JS and CSS files are NOT blocked by Robots.txt
  • Disallow: /bundles/
  • Disallow: /css/
Back to top