Log In Sign Up

kerstversiering.be
robots.txt

Robots Exclusion Standard data for kerstversiering.be

Resource Scan

Scan Details

Site Domain kerstversiering.be
Base Domain kerstversiering.be
Scan Status Ok
Last Scan2024-10-26T06:32:50+00:00
Next Scan 2024-11-25T06:32:50+00:00

Last Scan

Scanned2024-10-26T06:32:50+00:00
URL https://kerstversiering.be/robots.txt
Redirect https://www.kerstversiering.be/robots.txt
Redirect Domain www.kerstversiering.be
Redirect Base kerstversiering.be
Domain IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Redirect IPs 151.101.130.132, 151.101.194.132, 151.101.2.132, 151.101.66.132
Response IP 199.232.46.132
Found Yes
Hash 96c299e769777221f709f35c23f8837e22bb19a4a9779e2be3579eb32586b0d3
SimHash 437ed370ed99

Groups

*

Rule Path
Disallow /*/q/*_ASC*
Disallow /*/q/*_DESC*
Disallow /*/q/*
Disallow /search/
Disallow /*/search/

mj12bot

Rule Path
Disallow /

vagabondo

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

exabot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

cfnetwork

Rule Path
Disallow /

twengabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

speedyspider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.kerstversiering.be/sitemap/sitemap_be.xml
Back to top

Comments

  • Website Sitemap
  • Crawlers Setup
  • Paths used by Magento's sorting or layered navigation
  • Disallow all filters, search queries, sorting and pagination
  • Disallow the search
  • Disallow with store code
  • ----------------------------------------------------
  • Block bad crawlers
  • ----------------------------------------------------
  • third party search engine bot
  • Web-crawler by the company 'WiseGuys' to gather unstructured webdata
  • Chinese Search Engine
  • French Search Engine
  • Russian Search Engine
  • User-agent: AhrefsBot
  • Disallow: /
Back to top