veo3.site
robots.txt

Robots Exclusion Standard data for veo3.site

Resource Scan

Scan Details

Site Domain veo3.site
Base Domain veo3.site
Scan Status Ok
Last Scan2025-10-06T01:11:28+00:00
Next Scan 2025-11-05T01:11:28+00:00

Last Scan

Scanned2025-10-06T01:11:28+00:00
URL https://veo3.site/robots.txt
Domain IPs 104.21.14.113, 172.67.202.232, 2606:4700:3031::ac43:cae8, 2606:4700:3037::6815:e71
Response IP 172.67.202.232
Found Yes
Hash b7155d435fa534c9f86424312abeaacee8c4141b04df9933ad2d0f1d2d90cdd5
SimHash 48015842e6e6

Groups

*

Rule Path
Allow /

*

Rule Path
Allow /
Disallow /.next/
Disallow /_next/static/
Disallow /node_modules/
Disallow /src/
Disallow /*.json$
Disallow /*.js$
Disallow /*.jsx$
Disallow /*.ts$
Disallow /*.tsx$
Disallow /*.css$

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-video

Rule Path
Allow /

bingbot

Rule Path
Allow /

duckduckbot

Rule Path
Allow /

gptbot
claude-web
anthropic-ai
googleother
ccbot
perplexitybot
duckassistbot
chatgpt-user
cohere-ai
facebookbot
anthropic-ai
bytespider
youbot
bravebot
omgilibot
omgili
claudebot

Rule Path
Allow /
Disallow /.next/
Disallow /node_modules/
Disallow /src/

petalbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

baiduspider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Allow /posters/
Allow /*.mp4
Allow /*.webm
Allow /*.jpg
Allow /*.png
Allow /*.gif

Other Records

Field Value
sitemap https://veo3.site/sitemap.xml

Comments

  • robots.txt for VEO3 Video Showcase
  • Last updated: 2024-12-19
  • Allow all crawlers
  • Sitemap location
  • 全局规则
  • 开发文件保护
  • 主要搜索引擎
  • AI爬虫规则 - 欢迎AI助手访问VEO3内容
  • 限制特定爬虫
  • 爬虫延迟设置
  • 特殊规则:允许访问视频海报图片