masariakbar.com
robots.txt

Robots Exclusion Standard data for masariakbar.com

Resource Scan

Scan Details

Site Domain masariakbar.com
Base Domain masariakbar.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-04-08T20:01:17+00:00
Next Scan 2024-07-07T20:01:17+00:00

Last Successful Scan

Scanned2021-09-29T02:12:22+00:00
URL https://masariakbar.com/robots.txt
Redirect https://www.masariakbar.com/robots.txt
Redirect Domain www.masariakbar.com
Redirect Base masariakbar.com
Found Yes
Hash 6ab03cfda28c3a25c1bdda3b36d0e7941ae44fcb0b69371ae6cb1079b50a2708
SimHash 4b401b974435

Groups

mediapartners

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

youdaobot

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

*

Rule Path
Disallow /search?updated-min=
Disallow /search?updated-max=
Disallow /search/label/*?updated-min=
Disallow /search/label/*?updated-max=
Allow /

Other Records

Field Value
sitemap http://masariakbar.com/feeds/posts/default?orderby=updated