marset.com
robots.txt

Robots Exclusion Standard data for marset.com

Resource Scan

Scan Details

Site Domain marset.com
Base Domain marset.com
Scan Status Ok
Last Scan2025-06-27T15:08:21+00:00
Next Scan 2025-07-27T15:08:21+00:00

Last Scan

Scanned2025-06-27T15:08:21+00:00
URL https://marset.com/robots.txt
Redirect https://www.marset.com/robots.txt
Redirect Domain www.marset.com
Redirect Base marset.com
Domain IPs 172.66.40.141, 172.66.43.115, 2606:4700:3108::ac42:288d, 2606:4700:3108::ac42:2b73
Redirect IPs 172.66.40.141, 172.66.43.115, 2606:4700:3108::ac42:288d, 2606:4700:3108::ac42:2b73
Response IP 172.66.40.141
Found Yes
Hash 779a777047947c8f6ad360afbeeddc5305e76d5cc74be21acea396762da779e7
SimHash 6a4ddc22409b

Groups

googlebot

Rule Path
Allow *
Allow /*.css$
Allow /*.js$
Allow /wp-content/uploads/*
Allow /wp-content/*.js
Allow /wp-content/*.css
Allow /wp-includes/*.js
Allow /wp-includes/*.css
Allow /wp-admin/admin-ajax.php
Disallow /*/newsletter/*

*

Rule Path
Disallow /wp-admin/*
Disallow /cgi-bin
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-includes/
Disallow /*/attachment/
Disallow /tag/*
Disallow /*/login/*
Disallow /wp-content/cache/*
Disallow /wp-content/old/*
Disallow /page/
Disallow /comments/
Disallow /xmlrpc.php
Disallow *?attachment_id*
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow *cookies*
Disallow *cuenta/
Disallow *data*
Disallow *login*
Disallow /*/login*
Disallow /*/login/*
Disallow *contrasena*
Disallow /verificacion-de-e-mail/
Disallow *comments
Disallow /*/newsletter/*
Disallow /*/distributor*/
Disallow /*/profesional/*
Disallow /*/landings/*
Disallow /*/purchase-conditions/*
Disallow /*/aviso-legal/*
Disallow /*/proteccion-de-datos/*
Disallow /*/datenschutzbestimmungen/*
Disallow /*/privacy-policy/*
Disallow /*/politica-cookies/*
Disallow /*/cookie-policy/*
Disallow /*/cookie-ricthlinie/*
Allow /*?

*

Rule Path
Disallow /?s=
Disallow /search

*

Rule Path
Disallow /trackback
Disallow /*trackback
Disallow /*trackback*
Disallow /*/trackback

*

Rule Path
Disallow /feed/$
Disallow /feed/
Disallow /comments/feed/
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/trackback/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/trackback/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /*/*/*/trackback/$

semrushbot*
siteaudit*

Rule Path
Disallow /

noxtrumbot
msnbot
slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 80

Other Records

Field Value
sitemap https://www.marset.com/tipo-sitemap.xml
sitemap https://www.marset.com/exterior-sitemap.xml
sitemap https://www.marset.com/lamparas-sitemap.xml
sitemap https://www.marset.com/lamparas-exterior-sitemap.xml
sitemap https://www.marset.com/proyectos-sitemap.xml
sitemap https://www.marset.com/disenadores-sitemap.xml
sitemap https://www.marset.com/news-sitemap.xml
sitemap https://www.marset.com/stories-sitemap.xml

Comments

  • hi bot, may the light be with you. v21122022
  • evitem renderitzacions incomplertes
  • per tots els bots i crawlers
  • modificat a allow url dinamicas
  • bloqueig cerques
  • bloqueig trackbacks
  • bloqueig feeds
  • hello semrush
  • slow down rubbish