abe.com
robots.txt
Robots Exclusion Standard data for abe.com
Resource Scan
Scan Details
Site Domain | abe.com |
Base Domain | abe.com |
Scan Status | Ok |
Last Scan | 2024-09-08T03:49:16+00:00 |
Next Scan | 2024-10-08T03:49:16+00:00 |
Last Scan
Scanned | 2024-09-08T03:49:16+00:00 |
URL | https://abe.com/robots.txt |
Redirect | https://www.abebooks.com/robots.txt |
Redirect Domain | www.abebooks.com |
Redirect Base | abebooks.com |
Domain IPs | 75.2.69.186, 99.83.223.161 |
Redirect IPs | 13.225.4.113, 13.225.4.116, 13.225.4.49, 13.225.4.88 |
Response IP | 13.225.4.113 |
Found | Yes |
Hash | b1a412ce65a9cbfe2434deedc5bc8c5732bc7e189a84198ce1e5edab0fee4b9a |
SimHash | ba45f9454616 |
Groups
*
Rule | Path |
---|---|
Disallow | /servlet/ |
Disallow | /abe/ |
Disallow | /abep/ |
Disallow | /cgi/ |
Disallow | /search/ |
Disallow | /docs/Newsletters/BD |
Disallow | /es/docs/BooksellerCentral/informativos |
Disallow | /collections/ |
googlebot
Rule | Path |
---|---|
Allow | /search/ |
Allow | /servlet/SearchResults |
Allow | /servlet/BookDetailsPL |
Allow | /servlet/BookstoreSearch |
Allow | /servlet/BrowseListingsResults |
Allow | /servlet/BrowseListingsEntry |
Allow | /servlet/ListingDetails |
Allow | /servlet/StoreFrontDisplay |
Allow | /abe/ListingDetails |
Allow | /servlet/DWRestService/ |
Allow | /servlet/RecommendationsApi |
Allow | /servlet/HighlightInventory |
Allow | /collections/ |
Disallow | /servlet/ |
Disallow | /abe/ |
Disallow | /abep/ |
Disallow | /cgi/ |
Disallow | /docs/Newsletters/BD |
Disallow | /es/docs/BooksellerCentral/informativos |
Disallow | /*IMAGE_URL$ |
msnbot
Rule | Path |
---|---|
Allow | /search/ |
Allow | /servlet/SearchResults |
Allow | /servlet/BookDetailsPL |
Allow | /servlet/BookstoreSearch |
Allow | /servlet/BrowseListingsResults |
Allow | /servlet/BrowseListingsEntry |
Allow | /servlet/ListingDetails |
Allow | /servlet/StoreFrontDisplay |
Allow | /abe/ListingDetails |
Allow | /servlet/DWRestService/ |
Allow | /collections/ |
Disallow | /servlet/ |
Disallow | /abe/ |
Disallow | /abep/ |
Disallow | /cgi/ |
Disallow | /docs/Newsletters/BD |
Disallow | /es/docs/BooksellerCentral/informativos |
bingbot
Rule | Path |
---|---|
Allow | /search/ |
Allow | /servlet/SearchResults |
Allow | /servlet/BookDetailsPL |
Allow | /servlet/BookstoreSearch |
Allow | /servlet/BrowseListingsResults |
Allow | /servlet/BrowseListingsEntry |
Allow | /servlet/ListingDetails |
Allow | /servlet/StoreFrontDisplay |
Allow | /abe/ListingDetails |
Allow | /servlet/DWRestService/ |
Allow | /collections/ |
Disallow | /servlet/ |
Disallow | /abe/ |
Disallow | /abep/ |
Disallow | /cgi/ |
Disallow | /docs/Newsletters/BD |
Disallow | /es/docs/BooksellerCentral/informativos |
slurp
Rule | Path |
---|---|
Allow | /search/ |
Allow | /servlet/SearchResults |
Allow | /servlet/BookDetailsPL |
Allow | /servlet/BookstoreSearch |
Allow | /servlet/BrowseListingsResults |
Allow | /servlet/BrowseListingsEntry |
Allow | /servlet/ListingDetails |
Allow | /servlet/StoreFrontDisplay |
Allow | /abe/ListingDetails |
Allow | /servlet/DWRestService/ |
Allow | /collections/ |
Disallow | /servlet/ |
Disallow | /abe/ |
Disallow | /abep/ |
Disallow | /cgi/ |
Disallow | /docs/Newsletters/BD |
Disallow | /es/docs/BooksellerCentral/informativos |
rogerbot
Rule | Path |
---|---|
Disallow | /author/ |
Disallow | /blog/ |
Disallow | /search/ |
Disallow | /book-search/ |
Disallow | /abe/ListingDetails |
Disallow | /servlet/ |
Disallow | /abe/ |
Disallow | /abep/ |
Disallow | /cgi/ |
Disallow | /docs/Newsletters/BD |
Disallow | /site-map/ |
Disallow | /textbook/isbn |
Disallow | /textbook/top-selling-textbooks/ |
Disallow | /products/isbn/ |
Disallow | /top-authors/ |
Disallow | /titles/ |
Disallow | /docs/Newsletters/ |
Disallow | /*plp$ |
Disallow | /*sf$ |
Disallow | /*bd$ |
claritybot
Rule | Path |
---|---|
Allow | /search/ |
Allow | /servlet/SearchResults |
Allow | /servlet/BookDetailsPL |
Allow | /servlet/BookstoreSearch |
Allow | /servlet/BrowseListingsResults |
Allow | /servlet/BrowseListingsEntry |
Allow | /servlet/ListingDetails |
Allow | /servlet/StoreFrontDisplay |
Allow | /abe/ListingDetails |
Allow | /servlet/DWRestService/ |
Allow | /servlet/RecommendationsApi |
Allow | /servlet/HighlightInventory |
Allow | /collections/ |
Disallow | /servlet/ |
Other Records
Field | Value |
---|---|
sitemap | https://www.abebooks.com/sitemap.bdp_index.xml |
sitemap | https://www.abebooks.com/sitemap.plp_index.xml |
sitemap | https://www.abebooks.com/brightspot-sitemap.xml |
Comments