Log In Sign Up

benditofutbol.com
robots.txt

Robots Exclusion Standard data for benditofutbol.com

Resource Scan

Scan Details

Site Domain benditofutbol.com
Base Domain benditofutbol.com
Scan Status Ok
Last Scan2024-06-13T21:40:52+00:00
Next Scan 2024-06-20T21:40:52+00:00

Last Scan

Scanned2024-06-13T21:40:52+00:00
URL https://benditofutbol.com/robots.txt
Redirect https://www.benditofutbol.com/robots.txt
Redirect Domain www.benditofutbol.com
Redirect Base benditofutbol.com
Domain IPs 3.131.118.244
Redirect IPs 18.154.206.17, 18.154.206.29, 18.154.206.4, 18.154.206.47, 2600:9000:249b:3800:d:3322:7140:93a1, 2600:9000:249b:4400:d:3322:7140:93a1, 2600:9000:249b:5000:d:3322:7140:93a1, 2600:9000:249b:5400:d:3322:7140:93a1, 2600:9000:249b:8000:d:3322:7140:93a1, 2600:9000:249b:c800:d:3322:7140:93a1, 2600:9000:249b:d400:d:3322:7140:93a1, 2600:9000:249b:f800:d:3322:7140:93a1
Response IP 18.165.171.124
Found Yes
Hash 934389aeaaafa2be72b7edb0d9a0d6cbb104a7bff5719272720fa724316c7b93
SimHash 4a349f704ad3

Groups

blp_bbot/0.1

Rule Path
Disallow /

libreprensabot/1.0

Rule Path
Disallow /

baiduspider/2.0

Rule Path
Disallow /

xenu link sleuth/1.3.8

Rule Path
Disallow /

owlin bot

Rule Path
Disallow /

superfeedr bot/2.0

Rule Path
Disallow /

flipboard/3.2.6 cfnetwork/711.0.6 darwin/14.0.0

Rule Path
Disallow /

simplepie/1.1.3

Rule Path
Disallow /

rome client

Rule Path
Disallow /

genieo/1.0

Rule Path
Disallow /

snarfer/1.0.2

Rule Path
Disallow /

moodlebot/1.0

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

bbot

Rule Path
Disallow /

blp_bbot/0.1

Rule Path
Disallow /

blp_bbot

Rule Path
Disallow /

psbot/0.1

Rule Path
Disallow /

psbot

Rule Path
Disallow /

exabot/3.0

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

yeti

Rule Path
Disallow /

snapbot/1.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

superfeedr

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

ezooms/1.0

Rule Path
Disallow /

netestate ne crawler

Rule Path
Disallow /

funwebproducts

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

catchbot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

trustpilotcrawler

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

blogsearch

Rule Path
Disallow /

powermarks

Rule Path
Disallow /

blogsearch/2 +http://www.icerocket.com/

Rule Path
Disallow /

jakarta commons-httpclient/3.1

Rule Path
Disallow /

rome client (http://tinyurl.com/64t5n) ver: 0.9

Rule Path
Disallow /

rome client

Rule Path
Disallow /

*

Rule Path
Disallow /author/*
Back to top

Other Records

Field Value
sitemap http://www.elcomercio.com/sitemap-index.xml
sitemap http://www.elcomercio.com/sitemap-google-news.xml
Back to top