Log In Sign Up

botlab.biz
robots.txt

Robots Exclusion Standard data for botlab.biz

Resource Scan

Scan Details

Site Domain botlab.biz
Base Domain botlab.biz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan3/30/2025, 6:52:47 AM
Next Scan 6/28/2025, 6:52:47 AM

Last Successful Scan

Scanned2/17/2023, 5:57:53 AM
URL https://botlab.biz/robots.txt
Domain IPs 104.21.81.169, 172.67.162.226, 2606:4700:3031::6815:51a9, 2606:4700:3036::ac43:a2e2
Response IP 172.67.162.226
Found Yes
Hash 23914a0a3f6c7eb3d36d033434ee2793a7657a13f0fd9e0968709c25bdc92d88
SimHash 4e3c8166ee90

Groups

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://botlab.biz/index_sitemap_1.xml
sitemap https://botlab.biz/index_sitemap_2.xml
sitemap https://botlab.biz/index_sitemap_3.xml
sitemap https://botlab.biz/index_sitemap_4.xml
sitemap https://botlab.biz/index_sitemap_5.xml
sitemap https://botlab.biz/index_sitemap_6.xml
sitemap https://botlab.biz/article_sitemap_2212.xml
sitemap https://botlab.biz/article_sitemap_125.xml
sitemap https://botlab.biz/article_sitemap_183.xml
sitemap https://botlab.biz/article_sitemap_1024.xml
sitemap https://botlab.biz/article_sitemap_116.xml
sitemap https://botlab.biz/article_sitemap_176.xml
sitemap https://botlab.biz/article_sitemap_2223.xml
sitemap https://botlab.biz/article_sitemap_00.xml
sitemap https://botlab.biz/article_sitemap_141.xml
sitemap https://botlab.biz/article_sitemap_233.xml
sitemap https://botlab.biz/article_sitemap_2519.xml
sitemap https://botlab.biz/article_sitemap_61.xml
Back to top