Log In Sign Up

beveragedaily.com
robots.txt

Robots Exclusion Standard data for beveragedaily.com

Resource Scan

Scan Details

Site Domain beveragedaily.com
Base Domain beveragedaily.com
Scan Status Ok
Last Scan2025-03-03T14:41:50+00:00
Next Scan 2025-04-02T14:41:50+00:00

Last Scan

Scanned2025-03-03T14:41:50+00:00
URL https://beveragedaily.com/robots.txt
Redirect https://www.beveragedaily.com:443/robots.txt
Redirect Domain www.beveragedaily.com
Redirect Base beveragedaily.com
Domain IPs 15.197.68.198, 15.197.82.42
Redirect IPs 23.209.46.11, 23.209.46.9, 2600:1413:b000:13::b857:c184, 2600:1413:b000:13::b857:c193
Response IP 42.99.140.184
Found Yes
Hash 00832835be6e192239f6878f918b6713cfdd616585a1da6e3f53586ca98e2d8f
SimHash f410e110a49f

Groups

*

Rule Path
Disallow /search/
Disallow /newsletter/
Disallow */pf/api/

mozilla/5.0 (x11; linux x86_64; rv:87.0) gecko/20100101 firefox/87.0;onetrustbot;

Rule Path
Disallow

anthropic-ai
bytespider
chatgpt
claudebot
gptbot
oai-searchbot

Rule Path
Disallow /search/
Disallow /newsletter/
Disallow */pf/api/

Other Records

Field Value
crawl-delay 1

ahrefsbot
ahrefssiteaudit
amazonbot
archive-it
audacy-podcast-scraper
discordbot
facebookexternalhit
feedburner
feedly
feedparser
feedvalidator
flipboard
gofeed
hubspot crawler
netnewswire
pinterestbot
twitterbot
whatsapp

Rule Path
Disallow /search/
Disallow /newsletter/
Disallow */pf/api/

Other Records

Field Value
crawl-delay 1

adsbot
aliyunsecbot
archivebot
awariobot
backlinksextendedbot
baiduspider
barkrowler
birdcrawlerbot
bitsightbot
blackboardally
blexbot
blogtrottr
bluechipbacklinks
bpimagewalker
bravebot
ccbot
cincraw
cis5550-crawler
cloudservermarketspider
coccocbot
coibotparser
crsspxlbot
cyberfindcrawler
cyotekwebcopy
darwin
dataforseobot
deepcrawl
diffbot
domainsbot
domainstatsbot
domcopbot
dotbot
dragonbot
dubbotbot
emailwolf
equellaurlbot
ev-crawler
exabot
facebot
faraday
feedbot
filemaker
glmslinkanalysis
grapeshotcrawler
harsilbot
iaskspider
imagesiftbot
inetdex-bot
informa java api
intently
io_bot
isscyberriskcrawler
istellabot
kazbtbot
linguee bot
linkanalyser
linkdexbot
linkfluence
livelapbot
magus bot
maxthon
mixrankbot
mj12bot
mj12bot
mojeekbot
monsidobot
moodlebot
netcraftsurveyagent
netestate ne crawler
neticle crawler
netpeak
newsify
nicecrawler
nodeping
online-webceo-bot
openindexspider
orbbot
paqlebot
petalbot
pocketparser
pubmatic crawler bot
rankurbot
scrapy
searchbot
sebot
seekport crawler
seekportbot
seekrbot
semanticscholarbot
sentibot
seo-audit-check-bot
seobilitybot
serpstatbot
seznambot
seznambot
simplepie
siteauditbot
sitebulb
siteone-crawler
smtbot
sprinklr
srcedamp
staffbase
startmebot
storebot
superbot
superpagesbot2
surdotlybot
tentacles
timpibot
trendictionbot0
turnitin
uk_lddc_bot
ultimate_sitemap_parser
vebidoobot
velenpublicwebcrawler
wallabyupbot
webwikibot
wellknownbot
wepchsearchengine
xbot
yeti
yisouspider
zoominfobot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.beveragedaily.com/arc/outboundfeeds/news-sitemap/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-section/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/videos/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/promotional-features/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/product-innovations/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/suppliers/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/events/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Retail-shopper-insights/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Manufacturers/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Ingredients/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Processing-packaging/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Markets/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/R-D/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Regulation-safety/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Editor-s-choice/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/News/Industry-voices/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Beer-cider/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Dairy-drinks/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Energy-sports-drinks/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Juice-drinks/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Soft-drinks/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Tea-coffee/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Spirits/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Water/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Sectors/Wine/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Beverage-entrepreneurs/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Craft/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Functional-beverages/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Future-flavors/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Health-wellness/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Low-no-alcohol-drinks/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Premium-indulgence/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Regulation/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Sustainability/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Trends/Sugar-reduction/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/AB-InBev/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Constellation-Brands/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Bacardi/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Brown-Forman/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Carlsberg/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Coca-Cola/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Danone/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Diageo/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Heineken/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Keurig-Dr-Pepper/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Molson-Coors/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Nestle/
sitemap https://www.beveragedaily.com/arc/outboundfeeds/sitemap-index/category/Big-brands/PepsiCo/
Back to top

Comments

  • Block all robots going to the following directories
  • OneTrustBot can go anywhere
  • AI bots specific crawl-delay
  • Other bots with specific crawl-delay
  • Unwanted bots
  • Sitemaps
Back to top

Warnings

  • 3 invalid lines.
Back to top