drzewa.com.pl
robots.txt
Robots Exclusion Standard data for drzewa.com.pl
Resource Scan
Scan Details
| Site Domain | drzewa.com.pl |
| Base Domain | drzewa.com.pl |
| Scan Status | Ok |
| Last Scan | 2024-06-18T13:02:03+00:00 |
| Next Scan | 2024-07-18T13:02:03+00:00 |
Last Scan
| Scanned | 2024-06-18T13:02:03+00:00 |
| URL | https://drzewa.com.pl/robots.txt |
| Redirect | https://www.drzewa.com.pl/robots.txt |
| Redirect Domain | www.drzewa.com.pl |
| Redirect Base | drzewa.com.pl |
| Domain IPs | 162.159.136.54, 162.159.137.54 |
| Redirect IPs | 162.159.136.54, 162.159.137.54 |
| Response IP | 162.159.136.54 |
| Found | Yes |
| Hash | 35868f4cdf8e8bf5b92bab833c453080510f01e10a59124174029add06aea1c6 |
| SimHash | 6956794306a9 |
Groups
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
bingbot
| Rule | Path |
|---|---|
| Disallow | /katalog-atlas-roslin? |
| Disallow | /katalog-atlas-roslin/ |
| Disallow | /app/ |
| Disallow | /bin/ |
| Disallow | /dev/ |
| Disallow | /lib/ |
| Disallow | /phpserver/ |
| Disallow | /pkginfo/ |
| Disallow | /report/ |
| Disallow | /setup/ |
| Disallow | /update/ |
| Disallow | /var/ |
| Disallow | /vendor/ |
| Disallow | /tag/ |
| Disallow | /index.php/ |
| Disallow | /catalog/product_compare/ |
| Disallow | /control/ |
| Disallow | /contacts/ |
| Disallow | /customize/ |
| Disallow | /newsletter/ |
| Disallow | /review/ |
| Disallow | /sendfriend/ |
| Disallow | /wishlist/ |
| Disallow | /checkout/ |
| Disallow | /onestepcheckout/ |
| Disallow | /customer/ |
| Disallow | /customer/account/ |
| Disallow | /customer/account/login/ |
| Disallow | /composer.json |
| Disallow | /composer.lock |
| Disallow | /CONTRIBUTING.md |
| Disallow | /CONTRIBUTOR_LICENSE_AGREEMENT.html |
| Disallow | /COPYING.txt |
| Disallow | /Gruntfile.js |
| Disallow | /LICENSE.txt |
| Disallow | /LICENSE_AFL.txt |
| Disallow | /nginx.conf.sample |
| Disallow | /package.json |
| Disallow | /php.ini.sample |
| Disallow | /RELEASE_NOTES.txt |
| Disallow | /*?*product_list_mode= |
| Disallow | /*?*product_list_order= |
| Disallow | /*?*product_list_limit= |
| Disallow | /*?*product_list_dir= |
| Disallow | /*?dir* |
| Disallow | /*?dir=desc |
| Disallow | /*?dir=asc |
| Disallow | /*?limit=all |
| Disallow | /*?mode* |
| Disallow | /*?*price=* |
| Disallow | /*?SID= |
| Disallow | /*.CVS |
| Disallow | /*.Zip$ |
| Disallow | /*.Svn$ |
| Disallow | /*.Idea$ |
| Disallow | /*.Sql$ |
| Disallow | /*.Tgz$ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 30 |
zumbot
zmeu
zend_http_client
youdaobot
yodaobot
yisouspider
yamanalab-robot
xpymep\.exe
www\.integromedb\.org
wotbot
websitetheweb\.com
webindetail\.com
webcapture
wbsearchbot
visaduhoc\.info
turnitinbot
the\ incutio\ xml-rpc\ php\ library
surveybot
speedy
sosospider
solomonobot
sogou
socialsearcher
snoopy
sitebot
sistrix
shopwiki
seoengworldbot
semrushbot
searchmetrics
screenerbot
rojerbot
riddler
queryseekerspider
purebot
proximic
procogseobot
peoplepal
pagesinventory
openwebindex
ocelli
nextgensearchbot
netseer
netestate\ ne\ crawler
netcraftsurveyagent
ncbot
msie\ or\ firefox\ mutant
magpie\-crawler
ltbot
lipperhey
linkdex\.com
lindex\.com
libwww-perl
jikespider
jakarta\ commons-httpclient
ip\-web\-crawler\.com
indy\ library
gigabot
ftrf\:\ friendly
ezooms
ezinearticleslinkscanner
exabot
easouspider
dow\ jones\ searchbot
dotnetdotcom
dotbot
discoverybot
dinoping
dataprovider\.com
curious
compspybot
comodo-certificates-spider
comodo
clipish
charlotte
catchbot
butterfly
brandwatch\.net
baiduspider
backlinkcrawler
awcheckbot
aihitbot
add\ catalog
accelobot
aboundex
seo-crawling
blexbot
mj12bot
crawler
domaincrawler
spbot
scrapy
seokicks-robot
cliqzbot
linkdexbot
ucrawler
petalbot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.drzewa.com.pl/sitemap.xml |
Warnings
- 18 invalid lines.
Comments