flexso.com
robots.txt

Robots Exclusion Standard data for flexso.com

Resource Scan

Scan Details

Site Domain flexso.com
Base Domain flexso.com
Scan Status Ok
Last Scan2025-08-27T08:25:05+00:00
Next Scan 2025-09-10T08:25:05+00:00

Last Scan

Scanned2025-08-27T08:25:05+00:00
URL https://flexso.com/robots.txt
Redirect https://www.flexso.com/robots.txt
Redirect Domain www.flexso.com
Redirect Base flexso.com
Domain IPs 94.237.108.238
Redirect IPs 94.237.108.238
Response IP 94.237.108.238
Found Yes
Hash 1baa896277dd6d1a515b034e84860e1eccbc6f50246c1ce57d82a99adb584ce5
SimHash 74501a5ab5b1

Groups

*

Rule Path
Disallow /cpresources/
Disallow /vendor/
Disallow /.env
Disallow /cache/
Disallow */autodiscover/
Disallow */assets/
Disallow */media/
Disallow */wp-content/
Disallow */wp-includes/
Disallow /uploads/files/marketing-pdf/
Disallow /web/uploads/files/marketing-pdf/
Disallow /cases/p2
Disallow /cases/p2?tag=2351
Disallow /cases/p2?tag=2379
Disallow /cases/p2?tag=3069
Disallow /cases/p3
Disallow /cases/p3?tag=2379
Disallow /cases/p3?tag=3069
Disallow /cases/p4
Disallow /cases/p4?tag=2351
Disallow /cases/p4?tag=3069
Disallow /cases/p5
Disallow /cases/p5?tag=2351
Disallow /cases/p5?tag=2379
Disallow /cases?tag=2351
Disallow /cases?tag=2379
Disallow /cases?tag=3069
Disallow /insights-blog/p2?cat=9
Disallow /insights-blog/p3
Disallow /insights-blog/p4?cat=9
Disallow /insights-blog/p5?cat=9
Disallow /insights-blog/p6?cat=9
Disallow /insights-blog/p7?cat=9
Disallow /insights-blog/p8
Disallow /insights-blog/p9?cat=9
Disallow /insights-blog?cat=9

Other Records

Field Value
crawl-delay 20

ahrefsbot
amazonbot
awariobot
barkrowler
blexbot
bw/1.1
bytespider
censysinspect
claudebot
coccocbot
dalvik/2.1.0
dataforseobot
dataprovider
dotbot
expanse
foregenix
geedoproductsearch
google-extended
gptbot
imagesiftbot
internet-measurement
ioncrawl
java
mj12bot
mozlila
orbbot
perplexitybot
petalbot
python-requests
scrapy
semrushbot
seznambot
wp_is_mobile
yandexbot
zoominfobot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.flexso.com/nl/sitemaps-1-sitemap.xml
sitemap https://www.flexso.com/fr/sitemaps-1-sitemap.xml
sitemap https://www.flexso.com/en/sitemaps-1-sitemap.xml
sitemap https://www.flexso.com/de/sitemaps-1-sitemap.xml

Comments

  • robots.txt for https://www.flexso.com/en/
  • live - don't allow web crawlers to index cpresources/ or vendor/
  • Disallow Bots