selency.nl
robots.txt

Robots Exclusion Standard data for selency.nl

Resource Scan

Scan Details

Site Domain selency.nl
Base Domain selency.nl
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-31T11:47:11+00:00
Next Scan 2024-07-30T11:47:11+00:00

Last Successful Scan

Scanned2024-04-02T11:44:57+00:00
URL https://selency.nl/robots.txt
Redirect https://www.selency.nl/robots.txt
Redirect Domain www.selency.nl
Redirect Base selency.nl
Domain IPs 18.155.68.102, 18.155.68.17, 18.155.68.30, 18.155.68.50
Redirect IPs 13.35.18.122, 13.35.18.123, 13.35.18.28, 13.35.18.69
Response IP 13.35.18.69
Found Yes
Hash 2513a51876f46cfe7cfd22d1b1508e4a1f96fa8cea314c51888322900e832d4f
SimHash 259c5dc4fc3e

Groups

*

Rule Path
Disallow /search.html*
Disallow /api/
Disallow /mon-compte/
Disallow /se-connecter*
Disallow /login
Disallow /forgot-password
Disallow /mot-de-passe-oublie
Disallow /register
Disallow /add-to-cart
Disallow /panier
Disallow /payment_success
Disallow /ma-commande
Disallow *?query=*
Disallow *?q=*
Disallow /checkout/
Disallow /*?SID=
Disallow /*___from_store%3Ddefault

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.1

Other Records

Field Value
sitemap https://www.selency.fr/sitemap.xml

Comments

  • Crawlers Setup
  • Crawlers Setup for V1 Pages
  • Paths (clean URLs)
  • Paths (no clean URLs)