dormessentials.com
robots.txt

Robots Exclusion Standard data for dormessentials.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	dormessentials.com
Base Domain	dormessentials.com
Scan Status	Ok
Last Scan	2025-09-29T08:41:34+00:00
Next Scan	2025-10-06T08:41:34+00:00

Last Scan

Scanned	2025-09-29T08:41:34+00:00
URL	https://dormessentials.com/robots.txt
Domain IPs	192.124.249.164
Response IP	192.124.249.164
Found	Yes
Hash	5ad311d1c1765fc088a008d7041722e15a60e75678c96c179f249d936edaf603
SimHash	2a975a4363b5

Groups

*

Rule	Path
Disallow	/cgi-bin/
Disallow	/wp-admin/
Disallow	/wp-content/
Disallow	/wp-content/plugins/
Disallow	/wp-content/cache/
Disallow	/wp-content/themes/
Disallow	/wp-includes/
Disallow	/wp-json/
Disallow	/xmlrpc.php
Disallow	/readme.html

Rule

Path

Disallow

/cgi-bin/

Disallow

/wp-admin/

Disallow

/wp-content/

Disallow

/wp-content/plugins/

Disallow

/wp-content/cache/

Disallow

/wp-content/themes/

Disallow

/wp-includes/

Disallow

/wp-json/

Disallow

/xmlrpc.php

Disallow

/readme.html

googlebot

Rule	Path
Allow	/

Rule

Path

Allow

googlebot-image

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

mediapartners-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google

Rule	Path
Allow	/

Rule

Path

Allow

adsbot-google-mobile

Rule	Path
Allow	/

Rule

Path

Allow

bingbot

Rule	Path
Allow	/

Rule

Path

Allow

applebot

Rule	Path
Allow	/

Rule

Path

Allow

msnbot-media

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

yandex

Rule	Path
Allow	/

Rule

Path

Allow

yandeximages

Rule	Path
Allow	/wp-content/uploads/

Rule

Path

Allow

/wp-content/uploads/

slurp

Rule	Path
Allow	/

Rule

Path

Allow

abonti

Rule	Path
Disallow	/

Rule

Path

Disallow

bdcbot

Rule	Path
Disallow	/

Rule

Path

Disallow

blexbot

Rule	Path
Disallow	/

Rule

Path

Disallow

builtbottough

Rule	Path
Disallow	/

Rule

Path

Disallow

bullseye

Rule	Path
Disallow	/

Rule

Path

Disallow

bunnyslippers

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

cegbfeieh

Rule	Path
Disallow	/

Rule

Path

Disallow

cheesebot

Rule	Path
Disallow	/

Rule

Path

Disallow

cherrypicker

Rule	Path
Disallow	/

Rule

Path

Disallow

cherrypickerelite/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

cherrypickerse/1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

copyrightcheck

Rule	Path
Disallow	/

Rule

Path

Disallow

cosmos

Rule	Path
Disallow	/

Rule

Path

Disallow

crescent

Rule	Path
Disallow	/

Rule

Path

Disallow

crescent internet toolpak http ole control v.1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

doc

Rule	Path
Disallow	/

Rule

Path

Disallow

dittospyder

Rule	Path
Disallow	/

Rule

Path

Disallow

dotbot

Rule	Path
Disallow	/

Rule

Path

Disallow

download ninja

Rule	Path
Disallow	/

Rule

Path

Disallow

easouspider

Rule	Path
Disallow	/

Rule

Path

Disallow

emailcollector

Rule	Path
Disallow	/

Rule

Path

Disallow

emailsiphon

Rule	Path
Disallow	/

Rule

Path

Disallow

emailwolf

Rule	Path
Disallow	/

Rule

Path

Disallow

erocrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

exabot

Rule	Path
Disallow	/

Rule

Path

Disallow

extractorpro

Rule	Path
Disallow	/

Rule

Path

Disallow

fasterfox

Rule	Path
Disallow	/

Rule

Path

Disallow

feedbooster

Rule	Path
Disallow	/

Rule

Path

Disallow

foobot

Rule	Path
Disallow	/

Rule

Path

Disallow

genieo

Rule	Path
Disallow	/

Rule

Path

Disallow

httrack

Rule	Path
Disallow	/

Rule

Path

Disallow

harvest

Rule	Path
Disallow	/

Rule

Path

Disallow

incutio

Rule	Path
Disallow	/

Rule

Path

Disallow

infonavirobot

Rule	Path
Disallow	/

Rule

Path

Disallow

istellabot

Rule	Path
Disallow	/

Rule

Path

Disallow

java/1.

Rule

Path

Disallow

jennybot

Rule

Path

Disallow

kenjin spider

Rule

Path

Disallow

keyword density/0.9

Rule

Path

Disallow

lnspiderguy

Rule

Path

Disallow

lexibot

Rule

Path

Disallow

linkscan/8.1a unix

Rule

Path

Disallow

linkwalker

Rule

Path

Disallow

linkextractorpro

Rule

Path

Disallow

miixpc

Rule

Path

Disallow

mata hari

Rule

Path

Disallow

maxpointcrawler

Rule

Path

Disallow

megaindex

Rule

Path

Disallow

mj12bot

Rule

Path

Disallow

msie crawler

Rule

Path

Disallow

mozilla/4

Rule

Path

Disallow

mozilla/4.0 (compatible; bullseye; windows 95)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows 95)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows 98)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows nt)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows xp)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows 2000)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 4.0; windows me)

Rule

Path

Disallow

mozilla/4.0 (compatible; msie 7.0; windows nt 5.1; sv1)

Rule

Path

Disallow

mozilla/5

Rule

Path

Disallow

mozilla/5.0

Rule

Path

Disallow

mozilla/5.0 (compatible; ahrefsbot/7.0; +http://ahrefs.com/robot/)

Rule

Path

Disallow

mozilla/5.0 (compatible; googlebot/2.1; +http://www.google.com/bot.html)

Rule

Path

Disallow

mozilla/5.0 (compatible; yandexaccessibilitybot/3.0; +http://yandex.com/bots)

Rule

Path

Disallow

mozilla/5.0 (windows nt 6.1; wow64; rv:40.0) gecko/20100101 firefox/40.1

Rule

Path

Disallow

mozilla/5.0 (compatible; msie 10.0; windows nt 6.1; trident/6.0)

Rule

Path

Disallow

mozilla/5.0 (compatible; baiduspider/2.0; +http://www.baidu.com/search/spider.html)

Rule

Path

Disallow

microsoft url control

Rule

Path

Disallow

mippin

Rule

Path

Disallow

missigua locator

Rule

Path

Disallow

mister pix

Rule

Path

Disallow

nicerspro

Rule

Path

Disallow

npbot

Rule

Path

Disallow

netants

Rule

Path

Disallow

niki-bot

Rule

Path

Disallow

nutch

Rule

Path

Disallow

offline explorer

Rule

Path

Disallow

openfind

Rule

Path

Disallow

propowerbot/2.14

Rule

Path

Disallow

prowebwalker

Rule

Path

Disallow

python-urllib

Rule

Path

Disallow

queryn metasearch

Rule

Path

Disallow

rma

Rule

Path

Disallow

repomonkey

Rule

Path

Disallow

sistrix

Rule

Path

Disallow

seznambot

Rule

Path

Disallow

sitesnagger

Rule

Path

Disallow

snappreviewbot

Rule

Path

Disallow

sogou

Rule

Path

Disallow

spankbot

Rule

Path

Disallow

spinn3r

Rule

Path

Disallow

szukacz/1.4

Rule

Path

Disallow

teleport

Rule

Path

Disallow

telesoft

Rule

Path

Disallow

the intraformant

Rule

Path

Disallow

thenomad

Rule

Path

Disallow

tighttwatbot

Rule

Path

Disallow

titan

Rule

Path

Disallow

true_robot

Rule

Path

Disallow

turnitinbot

Rule

Path

Disallow

urly warning

Rule

Path

Disallow

ubicrawler

Rule

Path

Disallow

unisterbot

Rule

Path

Disallow

unknown

Rule

Path

Disallow

vci

Rule

Path

Disallow

wbsearchbot

Rule

Path

Disallow

www-collector-e

Rule

Path

Disallow

web downloader/6.9

Rule

Path

Disallow

web image collector

Rule

Path

Disallow

webauto

Rule

Path

Disallow

webbandit

Rule

Path

Disallow

webcopier

Rule

Path

Disallow

webenhancer

Rule

Path

Disallow

webreaper

Rule

Path

Disallow

websauger

Rule

Path

Disallow

webstripper

Rule

Path

Disallow

webzip

Rule

Path

Disallow

webmasterworldforumbot

Rule

Path

Disallow

website quester

Rule

Path

Disallow

webster pro

Rule

Path

Disallow

wotbox

Rule

Path

Disallow

xenu

Rule

Path

Disallow

zao

Rule

Path

Disallow

zeus

Rule

Path

Disallow

zyborg

Rule

Path

Disallow

Other Records

Field

Value

sitemap

https://dormessentials.com/sitemap.xml

sitemap

https://dormessentials.com/sitemap_images.xml

sitemap

https://dormessentials.com/sitemap_video.xml

Comments

****************************************************************************
robots.txt
: Robots, spiders, and search engines use this file to determine which
content they should *not* crawl while indexing your website.
: This system is called "The Standard Robots Exclusion."
****************************************************************************
Negative Robots Blocking

dormessentials.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

*

googlebot

googlebot-image

mediapartners-google

adsbot-google

adsbot-google-mobile

bingbot

applebot

msnbot-media

yandex

yandeximages

slurp

abonti

bdcbot

blexbot

builtbottough

bullseye

bunnyslippers

ccbot

cegbfeieh

cheesebot

cherrypicker

cherrypickerelite/1.0

cherrypickerse/1.0

copyrightcheck

cosmos

crescent

crescent internet toolpak http ole control v.1.0

doc

dittospyder

dotbot

download ninja

easouspider

emailcollector

emailsiphon

emailwolf

erocrawler

exabot

extractorpro

fasterfox

feedbooster

foobot

genieo

httrack

harvest

incutio

infonavirobot

istellabot

java/1.

jennybot

kenjin spider

keyword density/0.9

lnspiderguy

lexibot

linkscan/8.1a unix

linkwalker

linkextractorpro

miixpc

mata hari

maxpointcrawler

megaindex

mj12bot

msie crawler

mozilla/4

mozilla/4.0 (compatible; bullseye; windows 95)

mozilla/4.0 (compatible; msie 4.0; windows 95)

mozilla/4.0 (compatible; msie 4.0; windows 98)

mozilla/4.0 (compatible; msie 4.0; windows nt)

mozilla/4.0 (compatible; msie 4.0; windows xp)

mozilla/4.0 (compatible; msie 4.0; windows 2000)

mozilla/4.0 (compatible; msie 4.0; windows me)

mozilla/4.0 (compatible; msie 7.0; windows nt 5.1; sv1)

mozilla/5

mozilla/5.0

mozilla/5.0 (compatible; ahrefsbot/7.0; +http://ahrefs.com/robot/)

dormessentials.com
robots.txt