brta.gov.bd
robots.txt

Robots Exclusion Standard data for brta.gov.bd

Resource Scan

Scan Details

Site Domain brta.gov.bd
Base Domain brta.gov.bd
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't establish SSL connection.
Last Scan2024-10-30T00:54:25+00:00
Next Scan 2025-01-28T00:54:25+00:00

Last Successful Scan

Scanned2024-03-11T23:43:18+00:00
URL http://brta.gov.bd/robots.txt
Domain IPs 103.163.210.129
Response IP 103.163.210.129
Found Yes
Hash 862fe6fbfc9f8046f4ce1eba3385e5ab978feb0112eacb5aa7fc96959b3d9c1c
SimHash 245454825511

Groups

mediapartners-google

Rule Path
Disallow

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

slurp

Rule Path
Disallow

teoma

Rule Path
Disallow

twiceler

Rule Path
Disallow

gigabot

Rule Path
Disallow

scrubby

Rule Path
Disallow

robozilla

Rule Path
Disallow

nutch

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

baiduspider

Rule Path
Disallow

naverbot

Rule Path
Disallow

yeti

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

asterias

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

*

Rule Path
Disallow
Disallow /cgi-bin/