Log In Sign Up

ingredientesonline.com.br
robots.txt

Robots Exclusion Standard data for ingredientesonline.com.br

Resource Scan

Scan Details

Site Domain ingredientesonline.com.br
Base Domain ingredientesonline.com.br
Scan Status Ok
Last Scan2024-09-16T09:13:18+00:00
Next Scan 2024-10-16T09:13:18+00:00

Last Scan

Scanned2024-09-16T09:13:18+00:00
URL https://ingredientesonline.com.br/robots.txt
Redirect https://www.ingredientesonline.com.br/robots.txt
Redirect Domain www.ingredientesonline.com.br
Redirect Base ingredientesonline.com.br
Domain IPs 18.155.68.126, 18.155.68.3, 18.155.68.37, 18.155.68.77
Redirect IPs 3.165.102.104, 3.165.102.112, 3.165.102.45, 3.165.102.91
Response IP 3.165.102.91
Found Yes
Hash 04eb5a389f6a22ddaaa663c9e54b062161545bd684bf33283dd16035214ada3e
SimHash 3134f970069b

Groups

*

Rule Path
Disallow /admin/
Disallow /checkout/
Disallow /customer/
Disallow /catalogsearch/
Disallow /app/
Disallow /bin/
Disallow /dev/
Disallow /lib/
Disallow /phpserver/
Disallow /pub/
Disallow /setup/
Disallow /update/
Disallow /var/
Disallow /*.php$
Disallow /*.php/
Disallow /*.php%5C?
Disallow /*?SID=
Disallow /*?___store=
Disallow /*?___from_store=
Disallow /*?___SID=
Disallow /composer.json
Disallow /composer.lock
Disallow /Gruntfile.js
Disallow /htaccess.sample
Disallow /index.php.sample
Disallow /nginx.conf.sample
Disallow /php.ini.sample
Disallow /web.config.sample
Disallow /includes/
Disallow /lib/
Disallow /vendor/
Disallow /pub/static/
Disallow /pub/errors/
Disallow /pub/cron.php
Disallow /pub/cron.sh
Disallow /update/
Disallow /var/
Back to top

Other Records

Field Value
sitemap https://www.ingredientesonline.com.br/media/sitemap.xml
sitemap https://www.ingredientesonline.com.br/media/sitemap.xml
Back to top

Comments

  • Magento 2 Robots.txt File
  • Website Sitemap
  • Crawlers
  • Disallow Paths
  • Exclude System Paths
  • Exclude Extensions Paths
  • Exclude Paths with Query Parameters
  • Exclude Files
  • Exclude Directories
  • Crawl Delay (optional, if needed)
  • Crawl-delay: 5
  • Custom Directories or Files
  • Disallow: /custom-directory/
  • Disallow: /custom-file.html
Back to top