marcelopedra.com.ar
robots.txt
Robots Exclusion Standard data for marcelopedra.com.ar
Resource Scan
Scan Details
| Site Domain | marcelopedra.com.ar |
| Base Domain | marcelopedra.com.ar |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-03-25T14:48:09+00:00 |
| Next Scan | 2025-06-23T14:48:09+00:00 |
Last Successful Scan
| Scanned | 2024-05-30T14:34:06+00:00 |
| URL | https://marcelopedra.com.ar/robots.txt |
| Response IP | 104.21.9.129 |
| Found | Yes |
| Hash | bdf42da3f3b22fe94049fcbb41474660cce64dbd527d706935c4875676fd56a1 |
| SimHash | 6537f0604955 |
Groups
almaden
aspseek
asterias
baiduspider
bbot
bravobrian
becomebot
cherrypicker
cherrypickerse/1.0
cherrypickerelite/1.0
copyrightcheck
cosmos
crescent
crescent internet toolpak http ole control v.1.0
dittospyder
dloader(naverrobot)
dumbbot
dumbot
egotobot
emailcollector
emailsiphon
emailwolf
extractorpro
gaisbot
generic
getright/4.2
geturl
grub-client
httrack
infonavirobot
jyxobot
larbin
linkscan/8.1a unix
linkwalker
linkextractorpro
mata hari
microsoft url control - 5.01.4511
microsoft url control - 6.00.8169
microsoft url control
mj12bot
moget
moget/2.1
naverrobot
netants
nexabot
nexabot
npbot
nutchorg
nutch
obot
omniexplorer_bot
openbot
openfind
owr_crawler
psbot
puf
quepasacreep
rabaz
rpt-httpclient
scoutabout
semanticdiscovery
steeler
shim-crawler
teleport
teleportpro
telesoft
turnitinbot
tutorgig
url control
voyager
vsecrawler
webbandit
webbandit/3.50
webcopier
webcopy
webfetcher
webminer
webreaper
websauger
webstripper
webzip/4.0
wget/1.6
wget/1.5.3
wget
http://www.almaden.ibm.com/cs/crawler
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /cgi-bin/ |
| Disallow | /temp/ |
| Disallow | /soft/ |
| Disallow | /pics/ |