Log In Sign Up

steve-parker.org
robots.txt

Robots Exclusion Standard data for steve-parker.org

Resource Scan

Scan Details

Site Domain steve-parker.org
Base Domain steve-parker.org
Scan Status Ok
Last Scan2025-06-04T03:55:21+00:00
Next Scan 2025-06-11T03:55:21+00:00

Last Scan

Scanned2025-06-04T03:55:21+00:00
URL https://steve-parker.org/robots.txt
Domain IPs 66.175.210.189
Response IP 66.175.210.189
Found Yes
Hash eaaca271986836cbf7bbfa9bd185a50620218dbd43176cbba37749eab3677d6c
SimHash 51caf2284267

Groups

*

Rule Path
Disallow /articles/funny/misc/
Disallow /ads/advertisers/

webcopier

Rule Path
Disallow /

npbot

Rule Path
Disallow /

advanced email extractor

Rule Path
Disallow /

e-collector

Rule Path
Disallow /

big brother

Rule Path
Disallow /

getright

Rule Path
Disallow /

homepagesearch

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

nationaldirectory-webspider/1.3

Rule Path
Disallow /

netants

Rule Path
Disallow /

netants/1.21

Rule Path
Disallow /

netants/1.24

Rule Path
Disallow /

netants/1.25

Rule Path
Disallow /

offlineexplorer

Rule Path
Disallow /

openfind

Rule Path
Disallow /

ucmore

Rule Path
Disallow /

webdownloader

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

da

Rule Path
Disallow /

fast-webcrawler

Rule Path
Disallow /

getright

Rule Path
Disallow /

webcollage

Rule Path
Disallow /

vspider

Rule Path
Disallow /

bumblebee

Rule Path
Disallow /

wget

Rule Path
Disallow /sh

wget

Rule Path
Disallow /

httrack

Rule Path
Disallow /
Back to top