Log In Sign Up

featureos.app
robots.txt

Robots Exclusion Standard data for featureos.app

Resource Scan

Scan Details

Site Domain featureos.app
Base Domain featureos.app
Scan Status Ok
Last Scan2024-10-21T04:15:57+00:00
Next Scan 2024-11-20T04:15:57+00:00

Last Scan

Scanned2024-10-21T04:15:57+00:00
URL https://featureos.app/robots.txt
Domain IPs 104.26.0.153, 104.26.1.153, 172.67.75.197, 2606:4700:20::681a:199, 2606:4700:20::681a:99, 2606:4700:20::ac43:4bc5
Response IP 104.26.0.153
Found Yes
Hash 68d6d29221fc61fe304ee82e9bd70a0bc0cefc40e1044455aa14f486d7288b3b
SimHash a646c953d1b0

Groups

*

Rule Path Comment
Allow / -
Disallow /success-story/%5Bslug%5D -
Disallow /blog/%5Bslug%5D -
Disallow /glossary/%5BcategorySlug%5D This disallows any URL under /glossary/ that has a categorySlug and a slug
Disallow /glossary/%3AjvkHCRs4e -
Disallow /legal/%3AvZuYaDlBI -
Disallow /integrations/%3Abi_9crNAl -
Disallow /glossary/%5BcategorySlug%5D/%5Bslug%5D This disallows the direct access to /glossary/ with a colon (:) but might not be necessary if these are placeholders for actual slugs
Disallow /nav -

*
adsbot-google

Rule Path
Disallow /cdn-cgi/bm/cv/
Disallow /cdn-cgi/challenge-platform/
Disallow /cdn-cgi/images/trace/
Disallow /cdn-cgi/rum
Disallow /cdn-cgi/scripts/
Disallow /cdn-cgi/styles/
Disallow /cdn-fpw/sxg/

adsbot-google

Rule Path
Disallow /*.woff2

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler
mj12bot
ahrefsbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://featureos.app/sitemap-index.xml
Back to top

Comments

  • _ _ ___ _ _ ___ _____ ___ _____ _____
  • ( ) ( )( _`\ ( ) ( ) | _`\ ( _ )( _`\ ( _ )(_ _)
  • | |_| || (_(_)`\`\_/'/' | (_) )| ( ) || (_) )| ( ) | | |
  • | _ || _)_ `\ /' | , / | | | || _ <'| | | | | |
  • | | | || (_( ) | | | |\ \ | (_) || (_) )| (_) | | |
  • (_) (_)(____/' (_) (_) (_)(_____)(____/'(_____) (_)
  • ___ ___ _ _ _ ___ ___ _ _ ___ _____ _ _ _
  • ( _`\ ( _`\ ( ) ( )(_)( _`\ ( _`\ ( ) ( )( _`\ ( _ )( ) ( ) /'_`\
  • | (_) )| (_(_) | `\| || || ( (_)| (_(_) `\`\_/'/'| (_(_)| (_) || |_| |(_) ) |
  • | _ <'| _)_ | , ` || || | _ | _)_ `\ /' | _)_ | _ || _ | /'/'
  • | (_) )| (_( ) | |`\ || || (_( )| (_( ) _ | | | (_( )| | | || | | | |_|
  • (____/'(____/' (_) (_)(_)(____/'(____/'(_) (_) (____/'(_) (_)(_) (_) (_)
  • *
  • Host
  • ----
  • Sitemaps
  • --------
  • Directories
  • -----------
  • User-agent: GPTBot
  • Disallow: /
  • Block leaky Cloudflare endpoints
  • --------------------------------
  • Stop crawling fonts
  • -------------------
  • Ban noisy bots
  • --------------
Back to top

Warnings

  • `host` is not a known field.
Back to top