emprivado.pt
robots.txt

Robots Exclusion Standard data for emprivado.pt

Resource Scan

Scan Details

Site Domain emprivado.pt
Base Domain emprivado.pt
Scan Status Ok
Last Scan2024-09-09T18:03:40+00:00
Next Scan 2024-10-09T18:03:40+00:00

Last Scan

Scanned2024-09-09T18:03:40+00:00
URL https://www.emprivado.pt/robots.txt
Domain IPs 51.77.159.177
Response IP 51.77.159.177
Found Yes
Hash c16c9dcf80cc656ea185c0a2166a86f9c5b5759edd280bbae5998155e7c09625
SimHash 2b3ef181439b

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 30

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

msnbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

slurp

Rule Path
Allow /

yahoo-blogs

Rule Path
Allow /

yahoo-mmcrawler

Rule Path
Allow /

bing

Rule Path
Allow /

bingbot

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /
Allow /*?p=
Allow /catalog/seo_sitemap/category/
Allow /catalogsearch/result/
Disallow /404/
Disallow /app/
Disallow /cgi-bin/
Disallow /downloader/
Disallow /lib/
Disallow /pkginfo/
Disallow /m2/
Disallow /report/
Disallow /stats/
Disallow /var/
Disallow /imagens/
Disallow /banner/
Disallow /teste/
Disallow /apoio/
Disallow /errors/
Disallow /scripts/
Disallow /shell/
Disallow /stats/
Disallow /catalog/product_compare/
Disallow /catalog/category/view/
Disallow /catalog/product/view/
Disallow /catalogsearch/
Disallow /catalogsearch/result/index/
Disallow /control/
Disallow /customer/
Disallow /customize/
Disallow /newsletter/
Disallow /poll/
Disallow /sendfriend/
Disallow /tag/
Disallow /wishlist/
Disallow /links/
Disallow /review/
Disallow /media/catalog/product/cache/
Disallow /catalog/product/gallery/
Disallow /cron.php
Disallow /cron.sh
Disallow /error_log
Disallow /install.php
Disallow /cleanup.php
Disallow /*?p=*&
Disallow /*?SID=

Other Records

Field Value
crawl-delay 600

Other Records

Field Value
sitemap https://www.emprivado.pt/smap/sitemap.xml
sitemap https://www.emprivado.pt/smap/sitemap-1-1.xml
sitemap https://www.emprivado.pt/smap/sitemap-1-2.xml
sitemap https://www.trukatruka.com/smap/sitemap6.xml
sitemap https://www.trukatruka.com/smap/sitemap6-8-1.xml
sitemap https://www.trukatruka.com/smap/sitemap6-8-2.xml
sitemap https://www.farmaciasexshop.com/smap/sitemap3.xml
sitemap https://www.farmaciaxxx/smap/sitemap5.xml
sitemap https://www.preliminaressexshop.com/smap/sitemap2.xml
sitemap https://www.preliminaressexshop.com/smap/sitemap2-2-1.xml
sitemap https://www.preliminaressexshop.com/smap/sitemap2-2-2.xml

Comments

  • Crawlers Setup
  • Allowable Index
  • Directories
  • Paths (clean URLs)
  • Sitemaps for this specific domain
  • Files
  • Paths (no clean URLs)