belibispustaka.com
robots.txt

Robots Exclusion Standard data for belibispustaka.com

Resource Scan

Scan Details

Site Domain belibispustaka.com
Base Domain belibispustaka.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-07T13:15:07+00:00
Next Scan 2024-11-06T13:15:07+00:00

Last Successful Scan

Scanned2024-07-10T13:13:08+00:00
URL https://www.belibispustaka.com/robots.txt
Domain IPs 172.217.194.121, 2404:6800:4003:c1a::79
Response IP 74.125.130.121
Found Yes
Hash c966c438020c4035d2a576a905a8627b3d1bf17ef459133ac1246558030c2efc
SimHash 4b4c5b954415

Groups

mediapartners

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

mediapartners-google

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

googlebot

Rule Path
Allow /

slurp

Rule Path
Allow /

bingbot

Rule Path
Allow /

msnbot

Rule Path
Allow /

baiduspider

Rule Path
Allow /

baiduspider-image

Rule Path
Allow /

youdaobot

Rule Path
Allow /

sogou web spider

Rule Path
Allow /

*

Rule Path
Disallow /search?updated-min=
Disallow /search?updated-max=
Disallow /search/label/*?updated-min=
Disallow /search/label/*?updated-max=
Allow /

Other Records

Field Value
sitemap https://www.belibispustaka.com/feeds/posts/default?orderby=updated