Log In Sign Up

fastslang.com
robots.txt

Robots Exclusion Standard data for fastslang.com

Resource Scan

Scan Details

Site Domain fastslang.com
Base Domain fastslang.com
Scan Status Ok
Last Scan2025-07-21T19:03:45+00:00
Next Scan 2025-08-20T19:03:45+00:00

Last Scan

Scanned2025-07-21T19:03:45+00:00
URL https://fastslang.com/robots.txt
Redirect https://www.fastslang.com/robots.txt
Redirect Domain www.fastslang.com
Redirect Base fastslang.com
Domain IPs 104.21.6.105, 172.67.134.185, 2606:4700:3031::6815:669, 2606:4700:3035::ac43:86b9
Redirect IPs 104.21.6.105, 172.67.134.185, 2606:4700:3031::6815:669, 2606:4700:3035::ac43:86b9
Response IP 104.21.6.105
Found Yes
Hash 0d2efbf27f9d702db633cbf9d1bed31df1e2c14d059823261da0f7b635cb4c43
SimHash 5804b95285af

Groups

*

Rule Path
Allow /

ccbot
gptbot
google-extended
applebot-extended
anthropic-ai
claudebot
cohere-ai
meta-externalagent
meta-externalfetcher

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.fastslang.com/main/sitemap_0.xml
sitemap https://www.fastslang.com/main/sitemap_1.xml
sitemap https://www.fastslang.com/main/sitemap_10.xml
sitemap https://www.fastslang.com/main/sitemap_11.xml
sitemap https://www.fastslang.com/main/sitemap_2.xml
sitemap https://www.fastslang.com/main/sitemap_3.xml
sitemap https://www.fastslang.com/main/sitemap_4.xml
sitemap https://www.fastslang.com/main/sitemap_5.xml
sitemap https://www.fastslang.com/main/sitemap_6.xml
sitemap https://www.fastslang.com/main/sitemap_7.xml
sitemap https://www.fastslang.com/main/sitemap_8.xml
sitemap https://www.fastslang.com/main/sitemap_9.xml
sitemap https://www.fastslang.com/search/sitemap_0.xml
sitemap https://www.fastslang.com/search/sitemap_1.xml
sitemap https://www.fastslang.com/search/sitemap_10.xml
sitemap https://www.fastslang.com/search/sitemap_11.xml
sitemap https://www.fastslang.com/search/sitemap_12.xml
sitemap https://www.fastslang.com/search/sitemap_13.xml
sitemap https://www.fastslang.com/search/sitemap_14.xml
sitemap https://www.fastslang.com/search/sitemap_15.xml
sitemap https://www.fastslang.com/search/sitemap_16.xml
sitemap https://www.fastslang.com/search/sitemap_2.xml
sitemap https://www.fastslang.com/search/sitemap_3.xml
sitemap https://www.fastslang.com/search/sitemap_4.xml
sitemap https://www.fastslang.com/search/sitemap_5.xml
sitemap https://www.fastslang.com/search/sitemap_6.xml
sitemap https://www.fastslang.com/search/sitemap_7.xml
sitemap https://www.fastslang.com/search/sitemap_8.xml
sitemap https://www.fastslang.com/search/sitemap_9.xml
Back to top

Comments

  • Allow search engines
  • Block AI crawlers
  • Optional blocks - commented out to preserve platform features
  • User-agent: YouBot # Blocks You.com search
  • User-agent: FacebookBot # Blocks FB previews
  • User-agent: Bytespider # Blocks TikTok previews
  • User-agent: Searchie # Blocks their search
  • User-agent: Omgilibot # Web intelligence
  • User-agent: Omgili # Web intelligence
  • User-agent: Diffbot # Web previews
  • User-agent: ImagesiftBot # Image indexing
  • User-agent: Timpibot # Web indexing
  • User-agent: Webzio-Extended # Web indexing
  • User-agent: Uniscan # Security scanning
  • Disallow: /
Back to top