seasaltcornwall.com
robots.txt

Robots Exclusion Standard data for seasaltcornwall.com

Resource Scan

Scan Details

Site Domain seasaltcornwall.com
Base Domain seasaltcornwall.com
Scan Status Ok
Last Scan2024-09-19T07:05:51+00:00
Next Scan 2024-10-19T07:05:51+00:00

Last Scan

Scanned2024-09-19T07:05:51+00:00
URL https://seasaltcornwall.com/robots.txt
Redirect https://www.seasaltcornwall.com/robots.txt
Redirect Domain www.seasaltcornwall.com
Redirect Base seasaltcornwall.com
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 199.232.45.124
Found Yes
Hash d5c37c1d7157baff1f9f731570dc8086a30a8e57885ec89f016ae3a2872c055b
SimHash 6305d55fe270

Groups

*
googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

dotbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

pinterestbot

Rule Path
Disallow /index.php/
Disallow /checkout/
Disallow /app/
Disallow /lib/
Disallow /*.php$
Disallow /pkginfo/
Disallow /report/
Disallow /var/
Disallow /catalog/
Disallow /customer/
Disallow /sendfriend/
Disallow /review/
Disallow /*SID%3D
Disallow /*?product_list_order=
Disallow /*?fit_clothing=
Disallow /*?base_colour=
Disallow /*?price=
Disallow /*?size=
Disallow /*%26productListFilters%3Dbase_colour
Disallow /*%26productListFilters%3Dsize
Disallow /*%26productListFilters%3Ddepartment
Disallow /*%26productListFilters%3Dgender
Disallow /*%26productListFilters%3Dfit_clothing
Disallow /*%26productListFilters%3Dfit_footwear
Disallow /*%26productListFilters%3Dgender
Disallow /*%26productListFilters%3Dklevu_price
Disallow /*?productListFilters=base_colour
Disallow /*?productListFilters=size
Disallow /*?productListFilters=department
Disallow /*?productListFilters=gender
Disallow /*?productListFilters=fit_clothing
Disallow /*?productListFilters=fit_footwear
Disallow /*?productListFilters=gender
Disallow /*?productListFilters=klevu_price

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://www.seasaltcornwall.com/media/sitemap/uk_sitemap.xml
sitemap https://www.seasaltcornwall.com/media/sitemap/ie_sitemap.xml
sitemap https://www.seasaltcornwall.com/media/sitemap/row_sitemap.xml

Comments

  • Exclusions for UK and ROW filters
  • Exclusions for Ireland filters