worthpoint.com
robots.txt
Robots Exclusion Standard data for worthpoint.com
Resource Scan
Scan Details
| Site Domain | worthpoint.com |
| Base Domain | worthpoint.com |
| Scan Status | Ok |
| Last Scan | 2024-11-15T21:27:59+00:00 |
| Next Scan | 2024-11-22T21:27:59+00:00 |
Last Scan
| Scanned | 2024-11-15T21:27:59+00:00 |
| URL | https://worthpoint.com/robots.txt |
| Redirect | https://www.worthpoint.com/robots.txt |
| Redirect Domain | www.worthpoint.com |
| Redirect Base | worthpoint.com |
| Domain IPs | 15.197.237.133, 3.33.241.121 |
| Redirect IPs | 54.209.14.98, 54.224.88.248 |
| Response IP | 54.209.14.98 |
| Found | Yes |
| Hash | 4a1ee413bb65ca5c104f6cfd8b0174a0c95bb8d3ca8bde5e978681099bb5fe16 |
| SimHash | 61fc65cbec78 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /profile |
| Disallow | /login |
| Disallow | /classfied |
| Disallow | /auctionhsad |
| Disallow | /mainCalendar |
| Disallow | /search |
| Disallow | /user |
| Disallow | /worthopedia/*/price |
| Disallow | /inventory/getPrice |
| Disallow | /inventory/loggedIn |
| Disallow | /pmimages |
| Disallow | /friends |
| Disallow | /templates |
| Disallow | /offsite-search.php |
| Disallow | /shoppingAssistant |
| Disallow | /about-item-review |
| Disallow | /about-item-removal |
| Disallow | /source/*/list |
| Disallow | /inventory/refineSearch |
| Disallow | /inventory/retrieveCategories |
| Disallow | /inventory/retrieveMaxAmount |
| Disallow | /inventory/search |
| Disallow | /spellCheck |
| Allow | /app/getAppSettings |
| Allow | /app/getLogin |
| Allow | /app/collection |
| Disallow | /app |
| Disallow | /marks/search |
| Disallow | /books/search |
| Disallow | /marks/searchDetails |
| Disallow | /books/searchDetails |
| Disallow | /lIUjcOwl |
| Disallow | /internalAd |
| Disallow | /liveLots |
ia_archiver
scoutjet
dotbot
spbot
netseer
downscout
seznam
exabot
bender
discobot
valueclick
lucidmedia
snapbot
sosospider
weesee
ahrefsbot
megaindex.ru
cliqzbot
semrushbot
mj12bot
blexbot
zumbot
sogou web spider
yandex
| Rule | Path |
|---|---|
| Disallow | /worthopedia |
| Disallow | /inventory |
| Disallow | /marks |
| Disallow | /books |
| Disallow | /source/*/list |
Comments