greyhound-data.com
robots.txt

Robots Exclusion Standard data for greyhound-data.com

Resource Scan

Scan Details

Site Domain greyhound-data.com
Base Domain greyhound-data.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan4/11/2025, 5:05:54 AM
Next Scan 4/18/2025, 5:05:54 AM

Last Successful Scan

Scanned4/22/2024, 10:35:30 PM
URL https://greyhound-data.com/robots.txt
Domain IPs 104.26.14.16, 104.26.15.16, 172.67.69.236, 2606:4700:20::681a:e10, 2606:4700:20::681a:f10, 2606:4700:20::ac43:45ec
Response IP 104.26.14.16
Found Yes
Hash fdf0d7712a22c346b7be33ce192f2d055291081e224984d6bc1d40117edfad40
SimHash 025ad290f639

Groups

mediapartners-google

Rule Path
Allow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

seokicks-robot

Rule Path
Disallow /

bingbot/2.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mj12bot

Rule Path
Disallow /

dotnetdotcom

Rule Path
Disallow /

yandex

Rule Path
Disallow /

integromedb

Rule Path
Disallow /

seznambot/3.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

scoutjet

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

asterias

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

charlotte

Rule Path
Disallow /

sogou+spider

Rule Path
Disallow /

sogou web spider/4.0

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm

Product Comment
sogou web spider/4.0(+http://www.sogou.com/docs/help/webmasters.htm 07)
Rule Path
Disallow /

jikespider

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot/6~bl

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

applebot/0.1

Rule Path
Disallow /

mail.ru_bot/2.0

Rule Path
Disallow /

scrapy/1.8.0

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

*

Rule Path
Disallow /a
Disallow /gfxupload.php
Disallow /picture.php
Disallow /dbdogaddsibling.php
Disallow /dbdoginsert.php
Disallow /dbdogreporterror.php
Disallow /dbdogupdate.php
Disallow /dbraceinsert.php
Disallow /dbraceupdate.php
Disallow /dbtestmating.php
Disallow /edit_profile.php
Disallow /profile.php