flux.ai
robots.txt

Robots Exclusion Standard data for flux.ai

Resource Scan

Scan Details

Site Domain flux.ai
Base Domain flux.ai
Scan Status Ok
Last Scan2025-10-12T23:03:09+00:00
Next Scan 2025-11-11T23:03:09+00:00

Last Scan

Scanned2025-10-12T23:03:09+00:00
URL https://flux.ai/robots.txt
Redirect https://www.flux.ai/robots.txt
Redirect Domain www.flux.ai
Redirect Base flux.ai
Domain IPs 34.49.169.58
Redirect IPs 104.18.20.80, 104.18.21.80, 2606:4700::6812:1450, 2606:4700::6812:1550
Response IP 104.18.21.80
Found Yes
Hash b3321307fd9938e68bed595df185585ca1f00640b6e5a8644aa64f8c1c322658
SimHash 787d9fedc7f7

Groups

*

Rule Path
Allow /p/*
Allow /blog/*
Allow /p/*test*
Allow /blog/*test*
Disallow /RF-user*
Disallow /rf-user*
Disallow /*test*

Other Records

Field Value
sitemap https://www.flux.ai/sitemap.xml

Comments

  • https://www.robotstxt.org/robotstxt.html
  • https://developers.google.com/search/docs/crawling-indexing/robots/robots_txt#url-matching-based-on-path-values
  • Applies to all user-agents
  • Allow (which override below disallow of /*test* because the rule path is longer/more specific)
  • Disallow test users