Log In Sign Up

boxalbums.com
robots.txt

Robots Exclusion Standard data for boxalbums.com

Resource Scan

Scan Details

Site Domain boxalbums.com
Base Domain boxalbums.com
Scan Status Ok
Last Scan2024-09-20T17:22:42+00:00
Next Scan 2024-09-27T17:22:42+00:00

Last Scan

Scanned2024-09-20T17:22:42+00:00
URL https://boxalbums.com/robots.txt
Domain IPs 91.219.194.2
Response IP 91.219.194.2
Found Yes
Hash c06f73ab2c383cc110ea8ed87b06835ce82705a748f368a9121c2735f5e7049e
SimHash 4674c4414296

Groups

*

Rule Path
Disallow /user
Disallow /page
Disallow /index.php?do=search
Disallow /forum
Disallow /19
Disallow /2

mail.ru_bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

Rule Path Comment
Disallow / áëîêèðóåò äîñòóï êî âñåìó ñàéòó

ahrefsbot

Rule Path Comment
Disallow / áëîêèðóåò äîñòóï êî âñåìó ñàéòó
Back to top

Comments

  • User-agent: Yandex
  • Host: boxalbums.com
  • Crawl-delay: 3
  • Disallow: /user
  • Disallow: /page
  • Disallow: /index.php?do=search
  • User-agent: Googlebot
  • Host: boxalbums.com
  • Crawl-delay: 3
  • Disallow: /user
  • Disallow: /page
  • Disallow: /index.php?do=search
  • User-agent: YandexImages
  • Crawl-delay: 10
Back to top

Warnings

  • `host` is not a known field.
Back to top