Log In Sign Up

ddfreedth.com
robots.txt

Robots Exclusion Standard data for ddfreedth.com

Resource Scan

Scan Details

Site Domain ddfreedth.com
Base Domain ddfreedth.com
Scan Status Ok
Last Scan2025-10-26T22:15:42+00:00
Next Scan 2025-11-02T22:15:42+00:00

Last Scan

Scanned2025-10-26T22:15:42+00:00
URL https://www.ddfreedth.com/robots.txt
Domain IPs 142.250.4.121, 2404:6800:4003:c02::79
Response IP 142.251.12.121
Found Yes
Hash 67aa1ff47a31b0d67cebff1ab4571eb21bc30675a25e6fc5bcd2e16af49bd71a
SimHash 4265b20f0919

Groups

*

Rule Path Comment
Allow / -
Disallow /search?q= Prevent search results indexing

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

majestic12

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

exabot

Rule Path
Disallow /

spamrefresher

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

linkchecker

Rule Path
Disallow /

nmap

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

sqlmap

Rule Path
Disallow /

blackwidow

Rule Path
Disallow /

custo

Rule Path
Disallow /

disco

Rule Path
Disallow /

download demon

Rule Path
Disallow /

eirgrabber

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

extractorpro

Rule Path
Disallow /

eyenetie

Rule Path
Disallow /

flashget

Rule Path
Disallow /

getright

Rule Path
Disallow /

getweb!

Rule Path
Disallow /

go!zilla

Rule Path
Disallow /

go-ahead-got-it

Rule Path
Disallow /

grabnet

Rule Path
Disallow /

grafula

Rule Path
Disallow /

hmview

Rule Path
Disallow /

httrack

Rule Path
Disallow /

image stripper

Rule Path
Disallow /

image sucker

Rule Path
Disallow /

indy library

Rule Path
Disallow /

interget

Rule Path
Disallow /

internet ninja

Rule Path
Disallow /

jetcar

Rule Path
Disallow /

joc web spider

Rule Path
Disallow /

larbin

Rule Path
Disallow /

leechftp

Rule Path
Disallow /

mass downloader

Rule Path
Disallow /

midown tool

Rule Path
Disallow /

mister pix

Rule Path
Disallow /

navroad

Rule Path
Disallow /

nearsite

Rule Path
Disallow /

netants

Rule Path
Disallow /

netmechanic

Rule Path
Disallow /

netspider

Rule Path
Disallow /

net vampire

Rule Path
Disallow /

netzip

Rule Path
Disallow /

octopus

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

offline navigator

Rule Path
Disallow /

pagegrabber

Rule Path
Disallow /

papa foto

Rule Path
Disallow /

pavuk

Rule Path
Disallow /

pcbrowser

Rule Path
Disallow /

realdownload

Rule Path
Disallow /

reget

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

smartdownload

Rule Path
Disallow /

superbot

Rule Path
Disallow /

superhttp

Rule Path
Disallow /

surfbot

Rule Path
Disallow /

takeout

Rule Path
Disallow /

teleport pro

Rule Path
Disallow /

voideye

Rule Path
Disallow /

web image collector

Rule Path
Disallow /

web sucker

Rule Path
Disallow /

webauto

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webfetch

Rule Path
Disallow /

webgo is

Rule Path
Disallow /

webleacher

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

websauger

Rule Path
Disallow /

website extractor

Rule Path
Disallow /

website quester

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webwhacker

Rule Path
Disallow /

webzip

Rule Path
Disallow /

wget

Rule Path
Disallow /

widow

Rule Path
Disallow /

www-mechanize

Rule Path
Disallow /

wsb25

Rule Path
Disallow /

xaldon webspider

Rule Path
Disallow /

zeus

Rule Path
Disallow /

screamingfrogseospider

Rule Path
Disallow /

dynomapper

Rule Path
Disallow /

sitechecker

Rule Path
Disallow /

octoparse

Rule Path
Disallow /

parsehub

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

oncrawl

Rule Path
Disallow /

import.io

Rule Path
Disallow /

webharvy

Rule Path
Disallow /

zyte

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.ddfreedth.com/sitemap.xml
sitemap https://www.ddfreedth.com/sitemap-pages.xml
sitemap https://www.ddfreedth.com/feeds/posts/default?orderby=UPDATED
Back to top

Comments

  • Global crawl settings
  • Disallow sensitive paths
  • Block known aggressive crawlers
  • sitemap
Back to top

Warnings

  • 2 invalid lines.
Back to top