clearpay.co.uk
robots.txt

Robots Exclusion Standard data for clearpay.co.uk

Resource Scan

Scan Details

Site Domain clearpay.co.uk
Base Domain clearpay.co.uk
Scan Status Ok
Last Scan2025-04-29T03:33:10+00:00
Next Scan 2025-05-06T03:33:10+00:00

Last Scan

Scanned2025-04-29T03:33:10+00:00
URL https://clearpay.co.uk/robots.txt
Redirect https://www.clearpay.co.uk/robots.txt
Redirect Domain www.clearpay.co.uk
Redirect Base clearpay.co.uk
Domain IPs 104.16.72.7, 104.16.73.7, 104.16.74.7, 104.16.75.7, 104.16.76.7, 2606:4700::6810:4807, 2606:4700::6810:4907, 2606:4700::6810:4a07, 2606:4700::6810:4b07, 2606:4700::6810:4c07
Redirect IPs 104.16.72.7, 104.16.73.7, 104.16.74.7, 104.16.75.7, 104.16.76.7, 2606:4700::6810:4807, 2606:4700::6810:4907, 2606:4700::6810:4a07, 2606:4700::6810:4b07, 2606:4700::6810:4c07
Response IP 104.16.72.7
Found Yes
Hash cbe5d3e01675e1eb218a153b2af2546e1d5dac5eb5a89b4483303245da227d3b
SimHash bd710d55ce07

Groups

*

Rule Path
Disallow /builder/
Disallow /site-staging/

Other Records

Field Value
sitemap https://www.afterpay.com/sitemap.xml

Comments

  • Disallow crawling of the builder pages