mnovine.hr
robots.txt
Robots Exclusion Standard data for mnovine.hr
Resource Scan
Scan Details
| Site Domain | mnovine.hr |
| Base Domain | mnovine.hr |
| Scan Status | Ok |
| Last Scan | 2024-11-15T06:37:54+00:00 |
| Next Scan | 2024-11-22T06:37:54+00:00 |
Last Scan
| Scanned | 2024-11-15T06:37:54+00:00 |
| URL | https://mnovine.hr/robots.txt |
| Redirect | https://www.mnovine.hr/robots.txt |
| Redirect Domain | www.mnovine.hr |
| Redirect Base | mnovine.hr |
| Domain IPs | 104.26.6.206, 104.26.7.206, 172.67.68.116, 2606:4700:20::681a:6ce, 2606:4700:20::681a:7ce, 2606:4700:20::ac43:4474 |
| Redirect IPs | 104.26.6.206, 104.26.7.206, 172.67.68.116, 2606:4700:20::681a:6ce, 2606:4700:20::681a:7ce, 2606:4700:20::ac43:4474 |
| Response IP | 104.26.7.206 |
| Found | Yes |
| Hash | df3653af829a7eb606db3b81a74d2dbab7aac0d2ad5cc1a710159ec1a543bf54 |
| SimHash | 341edf606e9a |
Groups
blexbot
mj12bot
ahrefsbot
semrushbot
seekport crawler
pricebot
dotbot
charlotte
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 60 |
bingbot
pinterestbot
facebook
facebot
facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 15 |