es.divxland.org
robots.txt

Robots Exclusion Standard data for es.divxland.org

Resource Scan

Scan Details

Site Domain es.divxland.org
Base Domain divxland.org
Scan Status Ok
Last Scan2026-02-28T11:36:36+00:00
Next Scan 2026-03-07T11:36:36+00:00

Last Scan

Scanned2026-02-28T11:36:36+00:00
URL https://es.divxland.org/robots.txt
Domain IPs 104.26.10.119, 104.26.11.119, 172.67.69.160, 2606:4700:20::681a:a77, 2606:4700:20::681a:b77, 2606:4700:20::ac43:45a0
Response IP 172.67.69.160
Found Yes
Hash 46273676c802c19b760d0bfe038e1ddffbbd40d883f09b01cd065d63ccd9a2e4
SimHash 0f0ed88087b0

Groups

mediapartners-google
adsbot-google

Rule Path
Disallow

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bytespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /*%5C*
Disallow /*?
Disallow /test
Disallow /wp-content/themes/apps/download.php
Disallow /files/apk/
Disallow /download/*/start/
Disallow /*download_expired*
Allow /

Other Records

Field Value
crawl-delay 3

huaweisymantecspider
clicksense
mj12bot
crawler4j
grapeshot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.divxland.org/wp-sitemap.xml
sitemap https://en.divxland.org/wp-sitemap.xml
sitemap https://es.divxland.org/wp-sitemap.xml
sitemap https://de.divxland.org/wp-sitemap.xml
sitemap https://fr.divxland.org/wp-sitemap.xml
sitemap https://br.divxland.org/wp-sitemap.xml
sitemap https://it.divxland.org/wp-sitemap.xml
sitemap https://tr.divxland.org/wp-sitemap.xml
sitemap https://ru.divxland.org/wp-sitemap.xml
sitemap https://id.divxland.org/wp-sitemap.xml
sitemap https://in.divxland.org/wp-sitemap.xml
sitemap https://vn.divxland.org/wp-sitemap.xml
sitemap https://cn.divxland.org/wp-sitemap.xml

Comments

  • Ads & Google Media
  • Yandex
  • Aggressive crawlers
  • Default rules
  • Block admin
  • Block junk / wildcards
  • Block internal / sensitive paths
  • Block bad bots completely
  • Sitemaps

Warnings

  • 2 invalid lines.