Log In Sign Up

aliani.hu
robots.txt

Robots Exclusion Standard data for aliani.hu

Resource Scan

Scan Details

Site Domain aliani.hu
Base Domain aliani.hu
Scan Status Ok
Last Scan2026-02-08T17:08:13+00:00
Next Scan 2026-02-15T17:08:13+00:00

Last Scan

Scanned2026-02-08T17:08:13+00:00
URL https://aliani.hu/robots.txt
Redirect https://www.aliani.hu/robots.txt
Redirect Domain www.aliani.hu
Redirect Base aliani.hu
Domain IPs 104.21.82.178, 172.67.161.52, 2606:4700:3033::6815:52b2, 2606:4700:3037::ac43:a134
Redirect IPs 104.21.82.178, 172.67.161.52, 2606:4700:3033::6815:52b2, 2606:4700:3037::ac43:a134
Response IP 172.67.161.52
Found Yes
Hash c3ccebb5acd0240231279087e285cdcb6a06690fac966ccda2d232b8e9060393
SimHash f90a59728e20

Groups

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

amazonbot
anthropic-ai
awariorssbot
awariosmartbot
blexbot
bytespider
ccbot
chatgpt-user
claude-web
claudebot
cohere-ai
dataforseobot
diffbot
dotbot
etaospider
facebot
google-extended
googleother
imagesiftbot
magpie-crawler
news-please
newsnow
omgili
omgilibot
petalbot
pinterestbot
scrapy
turnitinbot
twitterbot

Rule Path
Disallow /

*

Rule Path
Disallow /shop/filter
Disallow /track/
Disallow /offer/
Disallow /modal/
Disallow /close-fb-modal
Disallow /maintenance
Disallow /online
Back to top

Other Records

Field Value
sitemap https://www.aliani.hu/sitemap.xml
Back to top

Comments

  • Sitemap
  • Baidu doesnt support Crawl-delay but added anyways in case they ever do
  • Disallowed Pinterestbot below
  • User-agent: Pinterestbot
  • Allow: /action/apiVisitRetailer
  • Allow: /interstitial/apiVisitRetailer
  • Crawl-delay: 1
Back to top