abebooks.com
robots.txt
Robots Exclusion Standard data for abebooks.com
Resource Scan
Scan Details
| Site Domain | abebooks.com |
| Base Domain | abebooks.com |
| Scan Status | Ok |
| Last Scan | 2024-05-03T13:49:30+00:00 |
| Next Scan | 2024-06-02T13:49:30+00:00 |
Last Scan
| Scanned | 2024-05-03T13:49:30+00:00 |
| URL | https://abebooks.com/robots.txt |
| Redirect | https://www.abebooks.com/robots.txt |
| Redirect Domain | www.abebooks.com |
| Redirect Base | abebooks.com |
| Domain IPs | 75.2.69.186, 99.83.223.161 |
| Redirect IPs | 13.225.4.113, 13.225.4.116, 13.225.4.49, 13.225.4.88 |
| Response IP | 13.225.4.49 |
| Found | Yes |
| Hash | af9db4a8c1c9a8d835085ae92cfc92725d24cd5c1d5b23888db7852aa61bda3d |
| SimHash | ba45f9454616 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /search/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /es/docs/BooksellerCentral/informativos |
| Disallow | /collections/ |
googlebot
| Rule | Path |
|---|---|
| Allow | /search/ |
| Allow | /servlet/SearchResults |
| Allow | /servlet/BookDetailsPL |
| Allow | /servlet/BookstoreSearch |
| Allow | /servlet/BrowseListingsResults |
| Allow | /servlet/BrowseListingsEntry |
| Allow | /servlet/ListingDetails |
| Allow | /servlet/StoreFrontDisplay |
| Allow | /abe/ListingDetails |
| Allow | /servlet/DWRestService/ |
| Allow | /servlet/RecommendationsApi |
| Allow | /servlet/HighlightInventory |
| Allow | /collections/ |
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /es/docs/BooksellerCentral/informativos |
| Disallow | /*IMAGE_URL$ |
msnbot
| Rule | Path |
|---|---|
| Allow | /search/ |
| Allow | /servlet/SearchResults |
| Allow | /servlet/BookDetailsPL |
| Allow | /servlet/BookstoreSearch |
| Allow | /servlet/BrowseListingsResults |
| Allow | /servlet/BrowseListingsEntry |
| Allow | /servlet/ListingDetails |
| Allow | /servlet/StoreFrontDisplay |
| Allow | /abe/ListingDetails |
| Allow | /servlet/DWRestService/ |
| Allow | /collections/ |
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /es/docs/BooksellerCentral/informativos |
bingbot
| Rule | Path |
|---|---|
| Allow | /search/ |
| Allow | /servlet/SearchResults |
| Allow | /servlet/BookDetailsPL |
| Allow | /servlet/BookstoreSearch |
| Allow | /servlet/BrowseListingsResults |
| Allow | /servlet/BrowseListingsEntry |
| Allow | /servlet/ListingDetails |
| Allow | /servlet/StoreFrontDisplay |
| Allow | /abe/ListingDetails |
| Allow | /servlet/DWRestService/ |
| Allow | /collections/ |
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /es/docs/BooksellerCentral/informativos |
slurp
| Rule | Path |
|---|---|
| Allow | /search/ |
| Allow | /servlet/SearchResults |
| Allow | /servlet/BookDetailsPL |
| Allow | /servlet/BookstoreSearch |
| Allow | /servlet/BrowseListingsResults |
| Allow | /servlet/BrowseListingsEntry |
| Allow | /servlet/ListingDetails |
| Allow | /servlet/StoreFrontDisplay |
| Allow | /abe/ListingDetails |
| Allow | /servlet/DWRestService/ |
| Allow | /collections/ |
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /es/docs/BooksellerCentral/informativos |
rogerbot
| Rule | Path |
|---|---|
| Disallow | /author/ |
| Disallow | /blog/ |
| Disallow | /search/ |
| Disallow | /book-search/ |
| Disallow | /abe/ListingDetails |
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /site-map/ |
| Disallow | /textbook/isbn |
| Disallow | /textbook/top-selling-textbooks/ |
| Disallow | /products/isbn/ |
| Disallow | /top-authors/ |
| Disallow | /titles/ |
| Disallow | /docs/Newsletters/ |
| Disallow | /*plp$ |
| Disallow | /*sf$ |
| Disallow | /*bd$ |
claritybot
| Rule | Path |
|---|---|
| Disallow | /author/ |
| Disallow | /search/ |
| Disallow | /book-search/ |
| Disallow | /abe/ListingDetails |
| Disallow | /servlet/ |
| Disallow | /abe/ |
| Disallow | /abep/ |
| Disallow | /cgi/ |
| Disallow | /docs/Newsletters/BD |
| Disallow | /site-map/ |
| Disallow | /textbook/isbn |
| Disallow | /textbook/top-selling-textbooks/ |
| Disallow | /products/isbn/ |
| Disallow | /top-authors/ |
| Disallow | /titles/ |
| Disallow | /docs/Newsletters/ |
| Disallow | /*plp$ |
| Disallow | /*sf$ |
| Disallow | /*bd$ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.abebooks.com/sitemap.bdp_index.xml |
| sitemap | https://www.abebooks.com/sitemap.plp_index.xml |
| sitemap | https://www.abebooks.com/brightspot-sitemap.xml |
Comments