cdandlp.com
robots.txt
Robots Exclusion Standard data for cdandlp.com
Resource Scan
Scan Details
| Site Domain | cdandlp.com |
| Base Domain | cdandlp.com |
| Scan Status | Ok |
| Last Scan | 2024-05-21T01:33:43+00:00 |
| Next Scan | 2024-06-20T01:33:43+00:00 |
Last Scan
| Scanned | 2024-05-21T01:33:43+00:00 |
| URL | https://cdandlp.com/robots.txt |
| Redirect | https://www.cdandlp.com/robots.txt |
| Redirect Domain | www.cdandlp.com |
| Redirect Base | cdandlp.com |
| Domain IPs | 195.167.226.21, 195.167.226.28 |
| Redirect IPs | 195.167.226.21, 195.167.226.28 |
| Response IP | 195.167.226.21 |
| Found | Yes |
| Hash | 4a2b59a0b18aea0b3a7e049e6eef6fb21c26f01eda7e989d4d8568e606f38102 |
| SimHash | 62774470c280 |
Groups
bingbot
adidxbot
bingpreview
microsoftpreview
gptbot
applebot
semrushbot
facebookexternalhit/1.1
facebookexternalhit/*
pinterestbot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
twiceler
pompos
semanticdiscovery
gaisbot
qihoobot
gigabot
shopwiki
ngbot/4.5
sosoimagespider
voilabot
httrack
httrack 3.0
winhttrack
voyager
dotbot
turnitinbot
turnitinbot/2.1
ahrefsbot
auto spider 1.0
exabot
blexbot
exabot-thumbnails
megaindex
petalbot
serpstatbot
linespider
ccbot/1.0
mj12bot
webalta crawler/2.0
boitho.com-dc
fatbot
discobot
sosospider
baiduspider
mlbot
bot/1.0
cazoodlebot
ia_archiver
seekbot
becomebot
asterias
snapbot
ichiro
irlbot
searchdaimon.com-dc
yanga worldsearch bot
sapphirewebcrawler/1.0
yeti/1.0
twengabot
speedy
yandex
dataforseobot
bytespider
bytedance
y!j-bru/vsidx
y!j-brw
y!j-asr
y!j-mmp/dscv
y!j-wsc
panscient.com
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /admin/ |
| Disallow | /CFIDE/ |
| Disallow | /caddie/ |
| Disallow | /commande/ |
| Disallow | /checkout/ |
| Disallow | /cart/ |
| Disallow | /gestion/ |
| Disallow | /moteur_de_rech/ |
| Disallow | /signup/ |
| Disallow | /login/ |
| Disallow | /vendre/ |
| Disallow | /*/get_picture_http_to_https.cfm |
| Disallow | /*/search_others_items_others_sellers_bottom_page.cfm |
| Disallow | /*/search_others_items_and_artists_same_seller_new.cfm |
| Disallow | /*/search_others_items_same_seller.cfm |
| Disallow | /*/search_others_items_others_sellers.cfm |
| Disallow | /*/search_other_picture_from_other_sellers.cfm |
| Disallow | /*/affiliatebanners.cfm |
| Disallow | /*?t= |
| Disallow | /*%26t%3D |
| Disallow | /*?l= |
| Disallow | /*%26l%3D |