Log In Sign Up

catholicdoors.com
robots.txt

Robots Exclusion Standard data for catholicdoors.com

Resource Scan

Scan Details

Site Domain catholicdoors.com
Base Domain catholicdoors.com
Scan Status Ok
Last Scan2024-10-04T06:49:37+00:00
Next Scan 2024-10-11T06:49:37+00:00

Last Scan

Scanned2024-10-04T06:49:37+00:00
URL https://catholicdoors.com/robots.txt
Redirect https://www.catholicdoors.com/robots.txt
Redirect Domain www.catholicdoors.com
Redirect Base catholicdoors.com
Domain IPs 216.38.10.129
Redirect IPs 216.38.10.129
Response IP 216.38.10.129
Found Yes
Hash 4c580805e74c81a51e30d4864232237867b4500caff0482adf897bbf5ff27d69
SimHash a3450a03ddf1

Groups

*

Rule Path
Disallow /cgi-bin/

googlebot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

bingbot

Rule Path
Allow /

altavista

Rule Path
Allow /

euroseek

Rule Path
Allow /

excite bot

Rule Path
Allow /

exactseek

Rule Path
Allow /

infoseek

Rule Path
Allow /

metasearch

Rule Path
Allow /

web fetch

Rule Path
Allow /

msnbot

Rule Path
Allow /

yandex

Rule Path
Allow /

yahoo slurp

Rule Path
Allow /

accelovation

Rule Path
Disallow /

amazonaws.com

Rule Path
Disallow /

attrakt

Rule Path
Disallow /

attributor

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

blexbot/1.0

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

cloudacl

Rule Path
Disallow /

creative autoupdate

Rule Path
Disallow /

charlotte.betaspider

Rule Path
Disallow /

dropdowndeals

Rule Path
Disallow /

femtosearchbot

Rule Path
Disallow *

findfiles.net/0.98

Rule Path
Disallow /

getleft 1.2

Rule Path
Disallow /

gootkit auto-rooter scanner

Rule Path
Disallow /

httrack

Rule Path
Disallow /

httrack 3.0x

Rule Path
Disallow /

ichiro/2.0

Rule Path
Disallow /

ilial

Rule Path
Disallow /

influencebot/0.9

Rule Path
Disallow /

itsapic.com_crawler

Rule Path
Disallow /

kimsufi.com

Rule Path
Disallow /

lemurproject nutch spider

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

litefinder

Rule Path
Disallow /

lssbot.com

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

msr-isrccrawler

Rule Path
Disallow /

msrputnik

Rule Path
Disallow /

mvk-it.com

Rule Path
Disallow /

nutch

Rule Path
Disallow /

pagenest

Rule Path
Disallow /

proxyway.com

Rule Path
Disallow /

ptd-crawler

Rule Path
Disallow /

qbikspider

Rule Path
Disallow /

ripper

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sixtrix

Rule Path
Disallow /

softbytelabs.com

Rule Path
Disallow /

squid-prefetch

Rule Path
Disallow /

swish

Rule Path
Disallow /

swish-e

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twtelecom.net

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

xenu link sleuth/1.3.8

Rule Path
Disallow /

webpix

Rule Path
Disallow /

webster pro v3.4

Rule Path
Disallow /

blexhn15.webmeup

Rule Path
Disallow /

wells search ii

Rule Path
Disallow /

wep

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

zermelo

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.catholicdoors.com/sitemap_location.xml
sitemap https://www.catholicdoors.com/sitemap_location.txt
Back to top

Warnings

  • 2 invalid lines.
Back to top