Log In Sign Up

visimob.com
robots.txt

Robots Exclusion Standard data for visimob.com

Resource Scan

Scan Details

Site Domain visimob.com
Base Domain visimob.com
Scan Status Ok
Last Scan2025-10-22T19:46:34+00:00
Next Scan 2025-11-21T19:46:34+00:00

Last Scan

Scanned2025-10-22T19:46:34+00:00
URL https://visimob.com/robots.txt
Domain IPs 104.26.12.61, 104.26.13.61, 172.67.75.14, 2606:4700:20::681a:c3d, 2606:4700:20::681a:d3d, 2606:4700:20::ac43:4b0e
Response IP 104.26.13.61
Found Yes
Hash 3fbe85e57cd4fa544ae973d6606f64926ee69bae351d39e3e808b60d0c41f37d
SimHash 465dc8d2e5b2

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

gptbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

chatgpt-user

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

ccbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

claude-web

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

amazonbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

anthropic-ai

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

perplexitybot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 2

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 0

bingbot

Rule Path
Allow /
Disallow /admin/
Disallow /auth/
Disallow /api/
Allow /static/
Allow /uploads/

Other Records

Field Value
crawl-delay 0

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /
Allow /static/*
Allow /uploads/*
Back to top

Other Records

Field Value
sitemap https://visimob.com/sitemap.xml
sitemap https://visimob.com/sitemap-pages.xml
sitemap https://visimob.com/sitemap-projects.xml
sitemap https://visimob.com/sitemap-images.xml
sitemap https://visimob.com/sitemap-videos.xml
sitemap https://visimob.com/sitemap-news.xml
Back to top

Comments

  • AI and LLM Bots
  • Search Engine Bots
  • Sitemaps
  • Disallow admin areas
  • Allow crawling of CSS and JS
  • Block bad bots
  • Cache control hints
  • Static assets can be cached
Back to top