confectionerynews.com
robots.txt
Robots Exclusion Standard data for confectionerynews.com
Resource Scan
Scan Details
| Site Domain | confectionerynews.com |
| Base Domain | confectionerynews.com |
| Scan Status | Ok |
| Last Scan | 2025-03-03T01:50:12+00:00 |
| Next Scan | 2025-04-02T01:50:12+00:00 |
Last Scan
| Scanned | 2025-03-03T01:50:12+00:00 |
| URL | https://confectionerynews.com/robots.txt |
| Redirect | https://www.confectionerynews.com:443/robots.txt |
| Redirect Domain | www.confectionerynews.com |
| Redirect Base | confectionerynews.com |
| Domain IPs | 15.197.68.198, 15.197.82.42 |
| Redirect IPs | 125.56.219.33, 125.56.219.34, 2600:1413:b000:13::b857:c184, 2600:1413:b000:13::b857:c193 |
| Response IP | 23.52.171.122 |
| Found | Yes |
| Hash | a4a2892ad38c507711a0bceb7c00103238b03dd3fcaa87ed4caca7b8e558c0c8 |
| SimHash | fc1ee850e793 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /newsletter/ |
| Disallow | */pf/api/ |
mozilla/5.0 (x11; linux x86_64; rv:87.0) gecko/20100101 firefox/87.0;onetrustbot;
| Rule | Path |
|---|---|
| Disallow |
anthropic-ai
bytespider
chatgpt
claudebot
gptbot
oai-searchbot
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /newsletter/ |
| Disallow | */pf/api/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
ahrefsbot
ahrefssiteaudit
amazonbot
archive-it
audacy-podcast-scraper
discordbot
facebookexternalhit
feedburner
feedly
feedparser
feedvalidator
flipboard
gofeed
hubspot crawler
netnewswire
pinterestbot
twitterbot
whatsapp
| Rule | Path |
|---|---|
| Disallow | /search/ |
| Disallow | /newsletter/ |
| Disallow | */pf/api/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
adsbot
aliyunsecbot
archivebot
awariobot
backlinksextendedbot
baiduspider
barkrowler
birdcrawlerbot
bitsightbot
blackboardally
blexbot
blogtrottr
bluechipbacklinks
bpimagewalker
bravebot
ccbot
cincraw
cis5550-crawler
cloudservermarketspider
coccocbot
coibotparser
crsspxlbot
cyberfindcrawler
cyotekwebcopy
darwin
dataforseobot
deepcrawl
diffbot
domainsbot
domainstatsbot
domcopbot
dotbot
dragonbot
dubbotbot
emailwolf
equellaurlbot
ev-crawler
exabot
facebot
faraday
feedbot
filemaker
glmslinkanalysis
grapeshotcrawler
harsilbot
iaskspider
imagesiftbot
inetdex-bot
informa java api
intently
io_bot
isscyberriskcrawler
istellabot
kazbtbot
linguee bot
linkanalyser
linkdexbot
linkfluence
livelapbot
magus bot
maxthon
mixrankbot
mj12bot
mj12bot
mojeekbot
monsidobot
moodlebot
netcraftsurveyagent
netestate ne crawler
neticle crawler
netpeak
newsify
nicecrawler
nodeping
online-webceo-bot
openindexspider
orbbot
paqlebot
petalbot
pocketparser
pubmatic crawler bot
rankurbot
scrapy
searchbot
sebot
seekport crawler
seekportbot
seekrbot
semanticscholarbot
sentibot
seo-audit-check-bot
seobilitybot
serpstatbot
seznambot
seznambot
simplepie
siteauditbot
sitebulb
siteone-crawler
smtbot
sprinklr
srcedamp
staffbase
startmebot
storebot
superbot
superpagesbot2
surdotlybot
tentacles
timpibot
trendictionbot0
turnitin
uk_lddc_bot
ultimate_sitemap_parser
vebidoobot
velenpublicwebcrawler
wallabyupbot
webwikibot
wellknownbot
wepchsearchengine
xbot
yeti
yisouspider
zoominfobot
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/news-sitemap/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-section/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/videos/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/promotional-features/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/product-innovations/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/suppliers/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/events/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Commodities/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Ingredients/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Industry-voices/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Manufacturers/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Markets/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Processing-packaging/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/R-D/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Regulation-safety/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/News/Retail-shopper-insights/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/Biscuits/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/Candy/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/Chocolate/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/Cocoa/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/Gum/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Sectors/Sugar/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Health-functionality/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Natural-colors-flavors/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Premium/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Seasonal/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Smart-technology/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Start-ups/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Sugar-reduction/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Sustainability/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Trends/Worker-protection/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Ferrero/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Haribo/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Hershey/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Lindt-Sprungli/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Lotte/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Mars/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Meji/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Mondelez-International/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Nestle/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Perfetti-Van-Melle/ |
| sitemap | https://www.confectionerynews.com/arc/outboundfeeds/sitemap-index/category/Big-brands/Yildiz-Holding/ |
Warnings
- 3 invalid lines.
Comments