modena.com
robots.txt

Robots Exclusion Standard data for modena.com

Resource Scan

Scan Details

Site Domain modena.com
Base Domain modena.com
Scan Status Ok
Last Scan2025-11-25T22:08:52+00:00
Next Scan 2025-12-25T22:08:52+00:00

Last Scan

Scanned2025-11-25T22:08:52+00:00
URL https://modena.com/robots.txt
Redirect https://www.modena.com/robots.txt
Redirect Domain www.modena.com
Redirect Base modena.com
Domain IPs 104.20.32.76, 172.66.173.225, 2606:4700:10::6814:204c, 2606:4700:10::ac42:ade1
Redirect IPs 104.20.32.76, 172.66.173.225, 2606:4700:10::6814:204c, 2606:4700:10::ac42:ade1
Response IP 172.66.173.225
Found Yes
Hash 9fed130ca92fcce44f712520e7e1af15a27e3fd2762eafcc8ac12fa61d8e0e8a
SimHash 09164b7462cd

Groups

mj12bot
semrushbot
yandex
yandexbot
zoominfobot
um-ln
mtrobot
safednsbot
bytespider
bitlybot
barkrowle
coccocbot-image
petalbot
dotbot
blexbot
aspiegelbot
baiduspider
trovitbot
mail.ru_bot
seznambot
yisouspider
zh-cn
slackbot
slackbot-linkexpanding
claudebot
amazonbot
fidget-spinner-bot
pinterestbot
vebot
awariobot
geedoproductsearch
skypeuripreview
gptbot
chatgpt-user

Rule Path
Disallow /

*

Rule Path
Disallow /*/*/*/switch
Disallow /*/*/*/redirect
Disallow /*/elasticsuite
Disallow /*/catalogsearch
Disallow /*/*/*?srsltid*
Disallow /*/*/*/*/___store/*/___from_store
Disallow /*/*/*/switch/?*
Disallow /*/manual-book-download
Disallow /*/search/ajax/suggest/?*
Disallow /*/*/*/*?srsltid*

*

Rule Path
Allow /*?brand=*
Allow /*/*?brand=*
Disallow /api
Disallow /graphql*
Disallow /*?
Disallow /catalog
Disallow /catalogsearch
Disallow /wishlist
Disallow /admin
Disallow /checkout
Disallow /onestepcheckout
Disallow /customer
Disallow /review
Disallow /sendfriend
Disallow /enable-cookies
Disallow /LICENSE.txt
Disallow /LICENSE.html
Disallow /skin
Disallow /js
Disallow /directory
Disallow /404
Disallow /*?dir*
Disallow /*?dir*
Disallow /*?limit*
Disallow /*/*?mode*
Disallow /*/*?dir*
Disallow /*/*?limit*
Disallow /*/*?mode*
Disallow /catalogsearch/result*

*

Rule Path
Disallow /*?utm*
Disallow /*?*&utm*
Disallow /*/*?utm*
Disallow /*/*?*&utm*

*

Rule Path
Disallow /*.csv$
Disallow /*.xls$
Disallow /*.json$
Disallow /*.asa$
Disallow /*.asax$
Disallow /*.ascx$
Disallow /*.axd$
Disallow /*.backup$
Disallow /*.bak$
Disallow /*.bat$
Disallow /*.cdx$
Disallow /*.cer$
Disallow /*.cfg$
Disallow /*.cmd$
Disallow /*.com$
Disallow /*.config$
Disallow /*.conf$
Disallow /*.cs$
Disallow /*.csproj$
Disallow /*.csr$
Disallow /*.dat$
Disallow /*.db$
Disallow /*.dbf$
Disallow /*.dll$
Disallow /*.dos$
Disallow /*.htr$
Disallow /*.htw$
Disallow /*.ida$
Disallow /*.idc$
Disallow /*.idq$
Disallow /*.inc$
Disallow /*.ini$
Disallow /*.key$
Disallow /*.licx$
Disallow /*.lnk$
Disallow /*.log$
Disallow /*.mdb$
Disallow /*.old$
Disallow /*.pass$
Disallow /*.pdb$
Disallow /*.pol$
Disallow /*.printer$
Disallow /*.pwd$
Disallow /*.rdb$
Disallow /*.resources$
Disallow /*.resx$
Disallow /*.sql$
Disallow /*.swp$
Disallow /*.sys$
Disallow /*.vb$
Disallow /*.vbs$
Disallow /*.vbproj$
Disallow /*.vsdisco$
Disallow /*.webinfo$
Disallow /*.xsd$
Disallow /*.xsx$
Disallow /*.md$
Disallow /*.bzr$
Disallow /*.MD$
Disallow /*._darcs$
Disallow /*.git$
Disallow /*.ssh$
Disallow /*.svn$
Disallow /*.class$

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

instagram

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

meta-externalagent

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

facebookexternalhit

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

Other Records

Field Value
sitemap https://www.modena.com/media/sitemap/sitemap_id_en.xml
sitemap https://www.modena.com/media/sitemap/sitemap_id_id.xml

Comments

  • Block Bad Bot
  • Modena Rules
  • Default Rules
  • Blocking UTM
  • Blocking extension. Hardening security purpose for defend from unknow hack tools
  • Crawl Delay For Decrease High Load