zekihaber.com
robots.txt

Robots Exclusion Standard data for zekihaber.com

Resource Scan

Scan Details

Site Domain zekihaber.com
Base Domain zekihaber.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-06-27T19:58:45+00:00
Next Scan 2024-09-25T19:58:45+00:00

Last Successful Scan

Scanned2024-02-29T19:50:45+00:00
URL https://zekihaber.com/robots.txt
Redirect https://www.zekihaber.com/robots.txt
Redirect Domain www.zekihaber.com
Redirect Base zekihaber.com
Domain IPs 104.21.55.213, 172.67.173.108, 2606:4700:3031::ac43:ad6c, 2606:4700:3032::6815:37d5
Redirect IPs 104.21.55.213, 172.67.173.108, 2606:4700:3031::ac43:ad6c, 2606:4700:3032::6815:37d5
Response IP 104.21.55.213
Found Yes
Hash aa0b553c9987f0049c5b1697c456a9121d4cf5953cd502851cb5c5eed1051335
SimHash 6d301e24c412

Groups

*

Rule Path
Disallow /public
Disallow /public/*
Disallow /public/index.php
Disallow /public/index.php/*
Disallow /service*
Disallow /share*
Disallow /tr/*
Disallow /*?ref=
Disallow /*?q=
Disallow /*?preview=
Disallow /*?utm_source=
Disallow /*?ref=
Allow /

adsbot-google

Rule Path
Disallow /advert/*
Allow /

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://www.zekihaber.com/sitemap.xml