Log In Sign Up

mera.bg
robots.txt

Robots Exclusion Standard data for mera.bg

Resource Scan

Scan Details

Site Domain mera.bg
Base Domain mera.bg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-02-11T20:36:59+00:00
Next Scan 2026-02-18T20:36:59+00:00

Last Successful Scan

Scanned2026-02-03T20:36:24+00:00
URL https://mera.bg/robots.txt
Redirect https://www.mera.bg/robots.txt
Redirect Domain www.mera.bg
Redirect Base mera.bg
Domain IPs 104.21.9.27, 172.67.141.46, 2606:4700:3034::6815:91b, 2606:4700:3034::ac43:8d2e
Redirect IPs 104.21.9.27, 172.67.141.46, 2606:4700:3034::6815:91b, 2606:4700:3034::ac43:8d2e
Response IP 172.67.141.46
Found Yes
Hash 2ec0c7fc2a3421fb0966448487f044c13c9183bb5d38c24d5df102c535cd168c
SimHash f90a59708e30

Groups

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

amazonbot
anthropic-ai
awariorssbot
awariosmartbot
blexbot
bytespider
ccbot
chatgpt-user
claude-web
claudebot
cohere-ai
dataforseobot
diffbot
dotbot
etaospider
facebot
google-extended
googleother
imagesiftbot
magpie-crawler
news-please
newsnow
omgili
omgilibot
petalbot
pinterestbot
scrapy
turnitinbot
twitterbot

Rule Path
Disallow /

*

Rule Path
Disallow /shop/filter
Disallow /track/
Disallow /offer/
Disallow /modal/
Disallow /close-fb-modal
Disallow /maintenance
Disallow /online
Back to top

Other Records

Field Value
sitemap https://www.mera.bg/sitemap.xml
Back to top

Comments

  • Sitemap
  • Baidu doesnt support Crawl-delay but added anyways in case they ever do
  • Disallowed Pinterestbot below
  • User-agent: Pinterestbot
  • Allow: /action/apiVisitRetailer
  • Allow: /interstitial/apiVisitRetailer
  • Crawl-delay: 1
Back to top