pbmdiscounter.nl
robots.txt

Robots Exclusion Standard data for pbmdiscounter.nl

Resource Scan

Scan Details

Site Domain pbmdiscounter.nl
Base Domain pbmdiscounter.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-22T16:19:52+00:00
Next Scan 2026-01-20T16:19:52+00:00

Last Successful Scan

Scanned2023-09-11T01:11:43+00:00
URL https://pbmdiscounter.nl/robots.txt
Domain IPs 104.21.34.20, 172.67.153.54, 2606:4700:3030::ac43:9936, 2606:4700:3031::6815:2214
Response IP 104.21.34.20
Found Yes
Hash e24a46ccd07fa792f3a837f8f036b884dbaf50720c9fc074229a43efcbba12ce
SimHash a900685a0896

Groups

*

Rule Path
Disallow /*?*filters=
Disallow /*?*sort=
Disallow /*?*direction=
Disallow /*?*amount=
Disallow /*?*layout=
Disallow /*?*category=
Disallow /*?*advice=
Disallow /*?*SID=
Disallow /catalogsearch/*
Disallow /review/product/listAjax/
Disallow /*?*color=
Disallow /*?*price=
Disallow /*?*property=
Disallow /*?*size=
Disallow /*?*manufacturer=
Disallow /*?*cat=
Disallow /shoppingcart
Disallow /checkout
Disallow /favorites
Disallow /account
Disallow /zoeken

Other Records

Field Value
sitemap https://pbmdiscounter.nl/uploads/sitemap/sitemap.xml

Comments

  • Filter and overview page parameters
  • Unknown parameter
  • Old site
  • Pages irrelevant for crawl