Log In Sign Up

ovu.com
robots.txt

Robots Exclusion Standard data for ovu.com

Resource Scan

Scan Details

Site Domain ovu.com
Base Domain ovu.com
Scan Status Ok
Last Scan2025-10-31T14:21:39+00:00
Next Scan 2025-11-07T14:21:39+00:00

Last Scan

Scanned2025-10-31T14:21:39+00:00
URL https://ovu.com/robots.txt
Domain IPs 66.147.238.51
Response IP 66.147.238.51
Found Yes
Hash 0c2ec0d1e8bcfaf1808e0478372520d05d789bb10a2bf8d462f08be7d1d43ff9
SimHash 6804d6c5f4e2

Groups

googlebot

Rule Path
Disallow /api/
Disallow /*?sort=
Disallow /*/?sort=
Disallow /*?q=
Disallow /*?country_code=
Disallow /*?dynamic=
Disallow /*?tid=
Disallow /*?page=
Disallow /*%26page%3D
Disallow /search_results?*&*
Allow /

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /api/
Disallow /*?dynamic=
Disallow /*?tid=
Disallow /*?page=
Allow /

Other Records

Field Value
crawl-delay 5

msnbot

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

Other Records

Field Value
crawl-delay 5

facebookexternalhit

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

twitterbot

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

mediapartners-google

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

googlebot-image

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

Other Records

Field Value
crawl-delay 5

yahoo-mmcrawler

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

Other Records

Field Value
crawl-delay 5

yahoo-slurp

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

Other Records

Field Value
crawl-delay 5

msnbot-media

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

bing preview bot

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /

rogerbot

Rule Path
Disallow /api/
Disallow /*?dynamic=
Allow /search_results
Allow /*?country_code=
Allow /

Other Records

Field Value
crawl-delay 10

sitegurucrawler

Rule Path
Disallow /api/
Disallow /*?sort=
Disallow /*/?sort=
Disallow /*?q=
Disallow /*?country_code=
Disallow /*?dynamic=
Disallow /search_results
Allow /

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /*?dynamic=
Disallow /*?tid=
Disallow /*?page=
Disallow /api/

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://ovu.com/filedata/cache/xml-sitemaps/sitemap.xml
Back to top

Comments

  • Emergency rate limiting - Add crawl delays
  • Temporarily block heavy SEO crawlers
  • Bing with rate limiting
  • Social media crawlers - keep lightweight
  • Google services
  • Yahoo crawlers
  • Microsoft crawlers
  • SEO/Analysis tools - keep but limit
  • Default for all other bots
  • Sitemap
Back to top