gundem.milliyet.com.tr
robots.txt

Robots Exclusion Standard data for gundem.milliyet.com.tr

Resource Scan

Scan Details

Site Domain gundem.milliyet.com.tr
Base Domain milliyet.com.tr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-03-15T20:16:13+00:00
Next Scan 2024-05-14T20:16:13+00:00

Last Successful Scan

Scanned2024-01-16T20:14:40+00:00
URL http://gundem.milliyet.com.tr/robots.txt
Redirect https://www.milliyet.com.tr/robots.txt
Redirect Domain www.milliyet.com.tr
Redirect Base milliyet.com.tr
Domain IPs 83.66.162.166
Redirect IPs 51.161.197.225, 51.79.231.108
Response IP 51.161.197.225
Found Yes
Hash 9dacae1e832189946994b531a39bf26936e61b24bf014ab52ac38a6c82b821f4
SimHash 94041a406f12

Groups

*

Rule Path
Disallow /root/*
Disallow /arama/
Disallow /Milliyet.aspx