bestclassifiedsusa.com
robots.txt
Robots Exclusion Standard data for bestclassifiedsusa.com
Resource Scan
Scan Details
| Site Domain | bestclassifiedsusa.com |
| Base Domain | bestclassifiedsusa.com |
| Scan Status | Ok |
| Last Scan | 2024-05-27T23:15:30+00:00 |
| Next Scan | 2024-06-03T23:15:30+00:00 |
Last Scan
| Scanned | 2024-05-27T23:15:30+00:00 |
| URL | https://bestclassifiedsusa.com/robots.txt |
| Domain IPs | 104.21.53.238, 172.67.220.5, 2606:4700:3037::6815:35ee, 2606:4700:3037::ac43:dc05 |
| Response IP | 172.67.220.5 |
| Found | Yes |
| Hash | 878632ff83cf884921852020cb34e0b77cd910d27b453d40314799db58f5efac |
| SimHash | 65465143c012 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /cgi-sys/suspendedpage.cgi |
| Disallow | /cgi-bin/ |
| Disallow | /oc-admin/ |
| Disallow | /user/ |
| Disallow | /user/profile/ |
| Disallow | /cdn-cgi/ |
| Disallow | /search/ |
| Disallow | /osclasspay/ |
| Disallow | /item/send-friend/ |
| Disallow | /fi-favorite-items |
| Disallow | /company-filter/ |
| Disallow | /im-create-thread/ |
| Disallow | /*sPeriod |
| Disallow | /*sLocation |
| Disallow | /*iOrderType |
| Disallow | /*sOrder |
| Disallow | /*sPriceMax |
| Disallow | /*sPriceMin |
| Disallow | /*bPriceFree |
| Disallow | /*bPriceCheckWithSeller |
| Disallow | /*bPic |
| Disallow | /*bPremium |
| Disallow | /*bPhone |
| Disallow | /*tags |
| Disallow | /*pattern |
| Disallow | /*?action |
| Disallow | /*?page |
| Disallow | /*?type |
| Disallow | /*_*-r* |
| Disallow | /*_*-c* |
cutestat
ahrefsbot
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
splitsignalbot
semrushbot-coub
mj12bot
petalbot
rogerbot
dotbot
linguee bot
domain re-animator bot
mauibot (crawler.feedback+wc@gmail.com)
etaospider
chatgpt-user
chatgpt
gptbot
yandex
bingbot
ia_archiver
moget
ichiro
naverbot
yeti
baiduspider
baiduspider-video
baiduspider-image
sogou spider
youdaobot
djecobot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://bestclassifiedsusa.com/sitemap.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_category.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_region.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_item.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_item_1.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_item_2.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_item_3.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_business.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_faq.xml |
| sitemap | https://bestclassifiedsusa.com/sitemap_link.xml |
| sitemap | https://bestclassifiedsusa.com/blog/sitemap_index.xml |