Log In Sign Up

dvbsat.org
robots.txt

Robots Exclusion Standard data for dvbsat.org

Resource Scan

Scan Details

Site Domain dvbsat.org
Base Domain dvbsat.org
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-29T06:47:49+00:00
Next Scan 2026-02-27T06:47:49+00:00

Last Successful Scan

Scanned2025-04-11T16:54:10+00:00
URL https://dvbsat.org/robots.txt
Domain IPs 104.21.37.237, 172.67.215.55, 2606:4700:3037::6815:25ed, 2606:4700:3037::ac43:d737
Response IP 172.67.215.55
Found Yes
Hash 087f3af80fbea9e5814f3e045e807b2eee011924eae133579a8e4729b323359b
SimHash 23943d214745

Groups

bingbot

Rule Path
Disallow /

*

Rule Path
Disallow /*-print/
Disallow /vbseocp.php
Disallow /login.php
Disallow /member.php
Disallow /memberlist.php
Disallow /newthread.php
Disallow /newreply.php
Disallow /printthread.php
Disallow /private.php
Disallow /profile.php
Disallow /register.php
Disallow /search.php
Disallow /sendmessage.php
Allow /
Back to top

Comments

  • ****************************************************************************
  • robots.txt
  • : Robots, spiders, and search engines use this file to detmine which
  • content they should *not* crawl while indexing your website.
  • : This system is called "The Robots Exclusion Standard."
  • : It is strongly encouraged to use a robots.txt validator to check
  • for valid syntax before any robots read it!
  • Examples:
  • Instruct all robots to stay out of the admin area.
  • : User-agent: *
  • : Disallow: /admin/
  • Restrict Google and MSN from indexing your images.
  • : User-agent: Googlebot
  • : Disallow: /images/
  • : User-agent: MSNBot
  • : Disallow: /images/
  • ****************************************************************************
  • Crawl-delay: 100
  • Sitemap: http://www.dvbsat.org/forum/sitemap/vbulletin_sitemap_index.xml.gz
Back to top