oneddl.org
robots.txt

Robots Exclusion Standard data for oneddl.org

Resource Scan

Scan Details

Site Domain oneddl.org
Base Domain oneddl.org
Scan Status Ok
Last Scan2025-08-21T23:21:57+00:00
Next Scan 2025-09-20T23:21:57+00:00

Last Scan

Scanned2025-08-21T23:21:57+00:00
URL https://oneddl.org/robots.txt
Domain IPs 104.21.13.205, 172.67.133.24, 2606:4700:3030::ac43:8518, 2606:4700:3035::6815:dcd
Response IP 172.67.133.24
Found Yes
Hash 2c6ca554577c29cf82a07d3f2965afc7a105045bb3b8e6c2c48d34565d7f7a0e
SimHash 510d9450e330

Groups

*

Rule Path
Disallow /engine/
Disallow /backup/
Disallow /language/
Disallow /upgrade/
Disallow /tags/
Disallow /addnews.html
Disallow /*subaction%3Duserinfo
Disallow /*subaction%3Dnewposts
Disallow /*do%3Dregister
Disallow /*do%3Dlostpassword
Disallow /*do%3Daddnews
Disallow /*do%3Dstats
Disallow /*do%3Dpm
Disallow /*do%3Dgo

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 5

yandex

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

pinterest

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

seokicks-robot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

uptimerobot/2.0

Rule Path
Disallow /

ezooms robot

Rule Path
Disallow /

perl lwp

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

wiseguys robot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://oneddl.org/sitemap.xml

Comments

  • ALL BOTS
  • Crawlers Setup
  • Block SEOkicks
  • Block SISTRIX
  • Block Uptime robot
  • Block Ezooms Robot
  • Block Perl LWP
  • Block netEstate NE Crawler
  • Block WiseGuys Robot

Warnings

  • 2 invalid lines.