Log In Sign Up

horecatiger.eu
robots.txt

Robots Exclusion Standard data for horecatiger.eu

Resource Scan

Scan Details

Site Domain horecatiger.eu
Base Domain horecatiger.eu
Scan Status Ok
Last Scan2025-12-13T04:33:18+00:00
Next Scan 2026-01-12T04:33:18+00:00

Last Scan

Scanned2025-12-13T04:33:18+00:00
URL https://horecatiger.eu/robots.txt
Domain IPs 104.18.24.221, 104.18.25.221, 2606:4700::6812:18dd, 2606:4700::6812:19dd
Response IP 104.18.25.221
Found Yes
Hash f18fb5d62b4c2092450fda51941dae9672ed69658b66637f5630973862f60049
SimHash 712e935934f6

Groups

*

Rule Path
Allow /
Disallow */checkout
Disallow */cart

mozilla

Rule Path
Allow /

chrome

Rule Path
Allow /

safari

Rule Path
Allow /

edge

Rule Path
Allow /

opera

Rule Path
Allow /

firefox

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

bingpreview

Rule Path
Allow /

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

slurp

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

yandexbot

Rule Path
Allow /

facebookbot

Rule Path
Allow /

applebot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

facebot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

snapchatbot

Rule Path
Allow /

tiktokbot

Rule Path
Allow /

redditbot

Rule Path
Allow /

discordbot

Rule Path
Allow /

telegrambot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot-google-mobile

Rule Path
Allow /

adsbot-google-mobile-apps

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

meta-adverts

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

twitterbot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

snapchatbot

Rule Path
Allow /

tiktokbot

Rule Path
Allow /

redditbot

Rule Path
Allow /

discordbot

Rule Path
Allow /

telegrambot

Rule Path
Allow /

amazonbot

Rule Path
Allow /

googlemerchantcenter

Rule Path
Allow /

amazonbot

Rule Path
Allow /

shopbot

Rule Path
Allow /

criteobot

Rule Path
Allow /

pricespider

Rule Path
Allow /

klarnabot

Rule Path
Allow /

google-extended

Rule Path
Allow /

openai

Rule Path
Allow /

claudebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

applebot-news

Rule Path
Allow /

neevabot

Rule Path
Allow /

cloudflare-alwaysonline

Rule Path
Allow /

fastly

Rule Path
Allow /

site24x7

Rule Path
Allow /

uptimerobot

Rule Path
Allow /

feedfetcher-google

Rule Path
Allow /

flipboardproxy

Rule Path
Allow /

applenewsbot

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

wpscan

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

wget

Rule Path
Disallow /

curl

Rule Path
Disallow /

google-site-verification

Rule Path
Allow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

pinterestbot

Rule Path
Allow /

whatsapp

Rule Path
Allow /

google-pagespeed

Rule Path
Allow /

google-favicon

Rule Path
Allow /

google-inspectiontool

Rule Path
Allow /

lighthouse

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

googlebot-discover

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

bingbot-news

Rule Path
Allow /

sqlmap

Rule Path
Disallow /

zgrab

Rule Path
Disallow /

masscan

Rule Path
Disallow /

curl

Rule Path
Disallow /

wget

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

java

Rule Path
Disallow /

brightdata

Rule Path
Disallow /

luminati

Rule Path
Disallow /

oxylabs

Rule Path
Disallow /

smartproxy

Rule Path
Disallow /

netnut

Rule Path
Disallow /

shifter

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://horecatiger.eu/en-eu/sitemap.xml
sitemap https://horecatiger.eu/fr-fr/sitemap.xml
sitemap https://horecatiger.eu/es-es/sitemap.xml
sitemap https://horecatiger.eu/it-it/sitemap.xml
sitemap https://horecatiger.eu/de-ch/sitemap.xml
sitemap https://horecatiger.eu/fr-ch/sitemap.xml
sitemap https://horecatiger.eu/it-ch/sitemap.xml
sitemap https://horecatiger.eu/nl-nl/sitemap.xml
Back to top

Comments

  • For all robots
  • Block access to specific groups of pages
  • Allow major web browsers (by common user-agent identifiers)
  • Allow Mobile-Specific Crawlers
  • Allow major search engine bots
  • Allow ad-related bots (Google Ads, Facebook Ads, etc.)
  • Allow Other Advertising & Social Crawlers
  • Allow Shopping & E-Commerce Bots (If Relevant)
  • Allow AI and Assistant Bots (If Relevant)
  • Allow CDN and Performance Bots
  • Allow Syndication and Aggregation Bots
  • Explicitly Block Bad Bots That Ignore Robots.txt
  • Allow Verification Bots
  • Allow for Page Speed & Core Web Vitals
  • Allow Google Discover & News Indexing (If Applicable)
  • Strengthen Security Against Credential Stuffing & Proxies
  • Block Known Proxy & VPN Crawlers (Advanced Security)
  • Temporarily disable so as to allow bots to crawl the new site ASAP
  • Request-rate: 1/10 # maximum rate is one page every 10 seconds
  • Crawl-delay: 5 # 10 seconds between page requests
  • Visit-time: 0400-0845 # only visit between 04:00 and 08:45 UTC
  • Allow search crawlers to discover the sitemap
Back to top