1stdibs.com
robots.txt
Robots Exclusion Standard data for 1stdibs.com
Resource Scan
Scan Details
| Site Domain | 1stdibs.com |
| Base Domain | 1stdibs.com |
| Scan Status | Ok |
| Last Scan | 2024-10-05T15:19:44+00:00 |
| Next Scan | 2024-10-19T15:19:44+00:00 |
Last Scan
| Scanned | 2024-10-05T15:19:44+00:00 |
| URL | https://1stdibs.com/robots.txt |
| Redirect | https://www.1stdibs.com/robots.txt |
| Redirect Domain | www.1stdibs.com |
| Redirect Base | 1stdibs.com |
| Domain IPs | 151.101.1.205 |
| Redirect IPs | 151.101.1.205, 151.101.129.205, 151.101.193.205, 151.101.65.205 |
| Response IP | 199.232.45.205 |
| Found | Yes |
| Hash | a66b6b13584f8feb8affdde19cb20352e3199efbbd91581fe46e6c1edb70225b |
| SimHash | ea3050e0561b |
Groups
*
| Rule | Path |
|---|---|
| Allow | /soa/graphql/$ |
| Allow | /soa/graphql$ |
| Allow | /soa/graphql?* |
| Allow | /soa/graphql/?* |
| Disallow | /fr/search/ |
| Disallow | /de/search/ |
| Disallow | /search/ |
| Disallow | /fr/soa/ |
| Disallow | /de/soa/ |
| Disallow | /soa/ |
| Disallow | /fr/administration/ |
| Disallow | /de/administration/ |
| Disallow | /administration/ |
| Disallow | /fr/cgi-bin/ |
| Disallow | /de/cgi-bin/ |
| Disallow | /cgi-bin/ |
| Disallow | /fr/vpv/ |
| Disallow | /de/vpv/ |
| Disallow | /vpv/ |
| Disallow | /fr/citysearch-account/ |
| Disallow | /de/citysearch-account/ |
| Disallow | /citysearch-account/ |
| Disallow | /fr/citysearch-administration/ |
| Disallow | /de/citysearch-administration/ |
| Disallow | /citysearch-administration/ |
| Disallow | /fr/citysearch/ |
| Disallow | /de/citysearch/ |
| Disallow | /citysearch/ |
| Disallow | /fr/shared_libraries/ |
| Disallow | /de/shared_libraries/ |
| Disallow | /shared_libraries/ |
| Disallow | /item_mail.php |
| Disallow | /maintenance.php |
| Disallow | /fr/notfound.html |
| Disallow | /de/notfound.html |
| Disallow | /notfound.html |
| Disallow | /privacypolicy.php |
| Disallow | /useragreement.php |
| Disallow | /fr/my/ |
| Disallow | /de/my/ |
| Disallow | /my/ |
| Disallow | /fr/checkout/ |
| Disallow | /de/checkout/ |
| Disallow | /checkout/ |
| Disallow | /fr/print/ |
| Disallow | /de/print/ |
| Disallow | /print/ |
| Disallow | /fr/item/ |
| Disallow | /de/item/ |
| Disallow | /item/ |
| Disallow | /fr/favorites/ |
| Disallow | /de/favorites/ |
| Disallow | /favorites/ |
| Disallow | /fr/recognized-dealers/furniture/ |
| Disallow | /de/recognized-dealers/furniture/ |
| Disallow | /recognized-dealers/furniture/ |
| Disallow | /fr/recognized-dealers/art/ |
| Disallow | /de/recognized-dealers/art/ |
| Disallow | /recognized-dealers/art/ |
| Disallow | /fr/recognized-dealers/jewelry/ |
| Disallow | /de/recognized-dealers/jewelry/ |
| Disallow | /recognized-dealers/jewelry/ |
| Disallow | /fr/recognized-dealers/fashion/ |
| Disallow | /de/recognized-dealers/fashion/ |
| Disallow | /recognized-dealers/fashion/ |
| Disallow | /fr/more-from-seller/ |
| Disallow | /de/more-from-seller/ |
| Disallow | /more-from-seller/ |
| Disallow | /fr/mobile/ |
| Disallow | /de/mobile/ |
| Disallow | /mobile/ |
| Disallow | /fr/resp/ |
| Disallow | /de/resp/ |
| Disallow | /resp/ |
| Disallow | /fr/similar/ |
| Disallow | /fr/similar-sold/ |
| Disallow | /de/similar/ |
| Disallow | /de/similar-sold/ |
| Disallow | /similar/ |
| Disallow | /similar-sold/ |
| Disallow | /*gp%3D* |
| Disallow | /*uw%3D* |
| Disallow | /*al%3D* |
| Disallow | /*bl%3D* |
| Disallow | /*gender%3D* |
| Disallow | /*section%3D* |
| Disallow | /*modal%3D* |
| Disallow | /*measurements%3D* |
| Disallow | /*view%3D* |
| Disallow | /*origin%3D* |
| Disallow | /*rt%3D* |
| Disallow | /*ft%3D* |
| Disallow | /*ut%3D* |
| Disallow | /*pageSize%3D* |
| Disallow | /*q%3D* |
| Disallow | *?s=* |
| Disallow | /*room%3D* |
| Disallow | /*project%3D* |
| Disallow | /*dealer%3D* |
| Disallow | /*forceLocale%3D* |
| Disallow | /*topSeller%3D* |
| Disallow | /fr/cpc/ |
| Disallow | /de/cpc/ |
| Disallow | /cpc/ |
| Disallow | /*sale%3D* |
| Disallow | /fr/view/ |
| Disallow | /de/view/ |
| Disallow | /view/ |
| Disallow | /fr/info/forms2/ |
| Disallow | /de/info/forms2/ |
| Disallow | /info/forms2/ |
| Disallow | /fr/private-sale/ |
| Disallow | /de/private-sale/ |
| Disallow | /private-sale/ |
| Disallow | /fr/buy/*/*page%3D* |
| Disallow | /de/buy/*/*page%3D* |
| Disallow | /buy/*/*page%3D* |
| Disallow | /fr/buyer-base/ |
| Disallow | /de/buyer-base/ |
| Disallow | /buyer-base/ |
| Disallow | /*topic%3D* |
| Disallow | /*frame%3D* |
| Disallow | /it/administration/ |
| Disallow | /it/buy/*/*page%3D* |
| Disallow | /it/buyer-base/ |
| Disallow | /it/cgi-bin/ |
| Disallow | /it/checkout/ |
| Disallow | /it/citysearch-account/ |
| Disallow | /it/citysearch-administration/ |
| Disallow | /it/citysearch/ |
| Disallow | /it/cpc/ |
| Disallow | /it/favorites/ |
| Disallow | /it/info/forms2/ |
| Disallow | /it/item/ |
| Disallow | /it/mobile/ |
| Disallow | /it/more-from-seller/ |
| Disallow | /it/my/ |
| Disallow | /it/notfound.html |
| Disallow | /it/print/ |
| Disallow | /it/private-sale/ |
| Disallow | /it/recognized-dealers/art/ |
| Disallow | /it/recognized-dealers/fashion/ |
| Disallow | /it/recognized-dealers/furniture/ |
| Disallow | /it/recognized-dealers/jewelry/ |
| Disallow | /it/resp/ |
| Disallow | /it/search/ |
| Disallow | /it/shared_libraries/ |
| Disallow | /it/similar/ |
| Disallow | /it/similar-sold/ |
| Disallow | /it/soa/ |
| Disallow | /it/view/ |
| Disallow | /it/vpv/ |
| Disallow | /es/administration/ |
| Disallow | /es/buy/*/*page%3D* |
| Disallow | /es/buyer-base/ |
| Disallow | /es/cgi-bin/ |
| Disallow | /es/checkout/ |
| Disallow | /es/citysearch-account/ |
| Disallow | /es/citysearch-administration/ |
| Disallow | /es/citysearch/ |
| Disallow | /es/cpc/ |
| Disallow | /es/favorites/ |
| Disallow | /es/info/forms2/ |
| Disallow | /es/item/ |
| Disallow | /es/mobile/ |
| Disallow | /es/more-from-seller/ |
| Disallow | /es/my/ |
| Disallow | /es/notfound.html |
| Disallow | /es/print/ |
| Disallow | /es/private-sale/ |
| Disallow | /es/recognized-dealers/art/ |
| Disallow | /es/recognized-dealers/fashion/ |
| Disallow | /es/recognized-dealers/furniture/ |
| Disallow | /es/recognized-dealers/jewelry/ |
| Disallow | /es/resp/ |
| Disallow | /es/search/ |
| Disallow | /es/shared_libraries/ |
| Disallow | /es/similar/ |
| Disallow | /es/similar-sold/ |
| Disallow | /es/soa/ |
| Disallow | /es/view/ |
| Disallow | /es/vpv/ |
Comments