clearpay.com
robots.txt

Robots Exclusion Standard data for clearpay.com

Resource Scan

Scan Details

Site Domain clearpay.com
Base Domain clearpay.com
Scan Status Ok
Last Scan2025-04-28T08:07:40+00:00
Next Scan 2025-05-05T08:07:40+00:00

Last Scan

Scanned2025-04-28T08:07:40+00:00
URL https://clearpay.com/robots.txt
Redirect https://www.clearpay.com/robots.txt
Redirect Domain www.clearpay.com
Redirect Base clearpay.com
Domain IPs 104.18.0.70, 104.18.1.70, 2606:4700::6812:146, 2606:4700::6812:46
Redirect IPs 104.18.0.70, 104.18.1.70, 2606:4700::6812:146, 2606:4700::6812:46
Response IP 104.18.0.70
Found Yes
Hash cbe5d3e01675e1eb218a153b2af2546e1d5dac5eb5a89b4483303245da227d3b
SimHash bd710d55ce07

Groups

*

Rule Path
Disallow /builder/
Disallow /site-staging/

Other Records

Field Value
sitemap https://www.afterpay.com/sitemap.xml

Comments

  • Disallow crawling of the builder pages