selency.co.uk
robots.txt

Robots Exclusion Standard data for selency.co.uk

Resource Scan

Scan Details

Site Domain selency.co.uk
Base Domain selency.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-05-28T06:45:10+00:00
Next Scan 2024-07-27T06:45:10+00:00

Last Successful Scan

Scanned2024-03-30T06:43:46+00:00
URL https://selency.co.uk/robots.txt
Redirect https://www.selency.co.uk/robots.txt
Redirect Domain www.selency.co.uk
Redirect Base selency.co.uk
Domain IPs 18.155.68.115, 18.155.68.66, 18.155.68.80, 18.155.68.82
Redirect IPs 108.156.133.101, 108.156.133.112, 108.156.133.46, 108.156.133.84
Response IP 108.156.133.101
Found Yes
Hash 2513a51876f46cfe7cfd22d1b1508e4a1f96fa8cea314c51888322900e832d4f
SimHash 259c5dc4fc3e

Groups

*

Rule Path
Disallow /search.html*
Disallow /api/
Disallow /mon-compte/
Disallow /se-connecter*
Disallow /login
Disallow /forgot-password
Disallow /mot-de-passe-oublie
Disallow /register
Disallow /add-to-cart
Disallow /panier
Disallow /payment_success
Disallow /ma-commande
Disallow *?query=*
Disallow *?q=*
Disallow /checkout/
Disallow /*?SID=
Disallow /*___from_store%3Ddefault

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 0.1

Other Records

Field Value
sitemap https://www.selency.fr/sitemap.xml

Comments

  • Crawlers Setup
  • Crawlers Setup for V1 Pages
  • Paths (clean URLs)
  • Paths (no clean URLs)