beadfx.com
robots.txt

Robots Exclusion Standard data for beadfx.com

Resource Scan

Scan Details

Site Domain beadfx.com
Base Domain beadfx.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-10-14T14:14:40+00:00
Next Scan 2025-10-21T14:14:40+00:00

Last Successful Scan

Scanned2025-09-13T14:13:42+00:00
URL https://beadfx.com/robots.txt
Domain IPs 69.27.107.24
Response IP 69.27.107.24
Found Yes
Hash 5c3d8a62e2278e1aa826e9a4f55b48efa3235dd3499a4551f260392d25fde218
SimHash 66d85b11c7c5

Groups

googlebot
googlebot-mobile

Rule Path
Disallow /cgi-bin/
Disallow /catalogue/cache/
Disallow /catalogue/bmz_cache/
Disallow /catalogue/index.php?main_page=product_reviews_write*
Disallow /cataloguenew/
Disallow /cataloguework/
Disallow /wordpress/wp-admin/
Disallow /classes/
Disallow /mobile/
Disallow /help/
Disallow /*flt*%3D*
Disallow /*sort%3D*
Disallow /*action%3D*
Disallow /*view%3D*

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Disallow /cgi-bin/
Disallow /catalogue/cache/
Disallow /catalogue/index.php?main_page=product_reviews_write*
Disallow /cataloguenew/
Disallow /cataloguework/
Disallow /mobile/
Disallow /help/
Disallow /*flt*%3D*
Disallow /*sort%3D*
Disallow /*action%3D*
Disallow /*view%3D*

Other Records

Field Value
crawl-delay 10

slurp
yahoobot

Rule Path
Disallow /cgi-bin/
Disallow /catalogue/cache/
Disallow /catalogue/bmz_cache/
Disallow /catalogue/index.php?main_page=product_reviews_write*
Disallow /cataloguenew/
Disallow /cataloguework/
Disallow /mobile/
Disallow /help/
Disallow /*flt*%3D*
Disallow /*sort%3D*
Disallow /*action%3D*
Disallow /*view%3D*

Other Records

Field Value
crawl-delay 10

duckduckbot
bingbot
bing
bingpreview
msnbot
msnbot-media
duckduckbot
facebookexternalhit
facebookbot
geedoproductsearch
twitterbot

Rule Path
Disallow /cgi-bin/
Disallow /catalogue/cache/
Disallow /catalogue/bmz_cache/
Disallow /catalogue/index.php?main_page=product_reviews_write*
Disallow /cataloguenew/
Disallow /cataloguework/
Disallow /mobile/
Disallow /help/
Disallow /*flt*%3D*
Disallow /*sort%3D*
Disallow /*action%3D*
Disallow /*view%3D*

Other Records

Field Value
crawl-delay 10

seo-powersuite-bot
chrome
safari

Rule Path
Disallow /cgi-bin/
Disallow /catalogue/cache/
Disallow /catalogue/bmz_cache/
Disallow /catalogue/index.php?main_page=product_reviews_write*
Disallow /cataloguenew/
Disallow /cataloguework/
Disallow /mobile/
Disallow /help/
Disallow /*flt*%3D*
Disallow /*sort%3D*
Disallow /*action%3D*
Disallow /*view%3D*

Other Records

Field Value
crawl-delay 10

ahrefsbot
ahrefssiteaudit

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

aiohttp

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /catalogue/cache/
Disallow /catalogue/bmz_cache/
Disallow /catalogue/index.php?main_page=product_reviews_write*
Disallow /cataloguenew/
Disallow /cataloguework/
Disallow /mobile/
Disallow /help/
Disallow /*flt*%3D*
Disallow /*sort%3D*
Disallow /*action%3D*
Disallow /*view%3D*

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

gptbot
claudebot
claude-user
claude-searchbot
ccbot
google-extended
applebot-extended
xxfacebookbot
meta-externalagent
meta-externalfetcher
diffbot
perplexitybot
perplexity‑user
omgili
omgilibot
webzio-extended
imagesiftbot
bytespider
tiktokspider
amazonbot
youbot
semrushbot-ocob
petalbot
velenpublicwebcrawler
turnitinbot
timpibot
oai-searchbot
icc-crawler
ai2bot
ai2bot-dolma
dataforseobot
awariobot
awariosmartbot
awariorssbot
google-cloudvertexbot
pangubot
kangaroo bot
sentibot
img2dataset
meltwater
seekr
peer39_crawler
cohere-ai
cohere-training-data-crawler
duckassistbot
scrapy
cotoyogi
aihitbot
factset_spyderbot
firecrawlagent

Rule Path
Disallow /

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.beadfx.com/catalogue/sitemap.xml
sitemap https://www.beadfx.com/catalogue/sitemap.xml
sitemap https://www.beadfx.com/catalogue/sitemap.xml
sitemap https://www.beadfx.com/catalogue/sitemap.xml

Comments

  • Block all known AI crawlers and assistants
  • from using content for training AI models.
  • Source: https://robotstxt.com/ai
  • Block any non-specified AI crawlers (e.g., new
  • or unknown bots) from using content for training
  • AI models, while allowing the website to be
  • indexed and accessed by bots. These directives
  • are still experimental and may not be supported
  • by all AI crawlers.

Warnings

  • `content-usage` is not a known field.
  • `disallowaitraining` is not a known field.