surgo.com
robots.txt
Robots Exclusion Standard data for surgo.com
Resource Scan
Scan Details
Site Domain | surgo.com |
Base Domain | surgo.com |
Scan Status | Ok |
Last Scan | 2024-09-16T02:21:36+00:00 |
Next Scan | 2024-10-16T02:21:36+00:00 |
Last Scan
Scanned | 2024-09-16T02:21:36+00:00 |
URL | https://surgo.com/robots.txt |
Redirect | https://www.surgo.com/robots.txt |
Redirect Domain | www.surgo.com |
Redirect Base | surgo.com |
Domain IPs | 65.9.112.65, 65.9.112.69, 65.9.112.96, 65.9.112.97 |
Redirect IPs | 52.222.144.108, 52.222.144.40, 52.222.144.64, 52.222.144.79 |
Response IP | 18.165.140.124 |
Found | Yes |
Hash | 8ee51fdd3e84c04fd97e0fc8a02fd408e9b178ed7ddb1ad15b34ea858f80bc97 |
SimHash | 47049070d230 |
Groups
adsbot-google
Rule | Path |
---|---|
Disallow | /root$ |
Disallow | /password-reset$ |
Disallow | /cart$ |
Disallow | /checkout$ |
Disallow | /checkout/ |
Disallow | /register$ |
*
Rule | Path |
---|---|
Disallow | /root$ |
Disallow | /password-reset$ |
Disallow | /cart$ |
Disallow | /checkout$ |
Disallow | /checkout/ |
Disallow | /register$ |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |