bid.awsurplusauction.com
robots.txt
Robots Exclusion Standard data for bid.awsurplusauction.com
Resource Scan
Scan Details
| Site Domain | bid.awsurplusauction.com |
| Base Domain | awsurplusauction.com |
| Scan Status | Ok |
| Last Scan | 2025-12-06T05:50:26+00:00 |
| Next Scan | 2026-01-05T05:50:26+00:00 |
Last Scan
| Scanned | 2025-12-06T05:50:26+00:00 |
| URL | https://bid.awsurplusauction.com/robots.txt |
| Domain IPs | 13.35.238.119, 13.35.238.61, 13.35.238.72, 13.35.238.74 |
| Response IP | 13.35.238.72 |
| Found | Yes |
| Hash | bd3dd715c0c6c6082616c6527131f2b43adff0d1555e5a7bab8b2a179c15ff63 |
| SimHash | ac808aa102d2 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api/file |
| Disallow | /api/graphql |
| Disallow | /api/json |
| Disallow | /api/soap12 |
| Disallow | /auctions/ask-question |
| Disallow | /auctions/register |
| Disallow | /auctions/tell-friend |
| Disallow | /login |
| Disallow | /logout |
| Disallow | /register/auction-lot-item-changes |
| Disallow | /register/confirm-bidder-options |
| Disallow | /register/confirm-shipping |
| Disallow | /register/registration-confirm |
| Disallow | /register/revise-billing |
| Disallow | /register/special-terms-and-conditions |
| Disallow | /register/terms-and-conditions |
| Disallow | /search |
| Disallow | /signup/index |
| Disallow | /signup/verify-email |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 120 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://bid.awsurplusauction.com/sitemap/cache/single-index.xml |
Comments