24chasa.bg
robots.txt

Robots Exclusion Standard data for 24chasa.bg

Resource Scan

Scan Details

Site Domain 24chasa.bg
Base Domain 24chasa.bg
Scan Status Ok
Last Scan2024-05-15T11:52:32+00:00
Next Scan 2024-05-22T11:52:32+00:00

Last Scan

Scanned2024-05-15T11:52:32+00:00
URL https://24chasa.bg/robots.txt
Redirect https://www.24chasa.bg/robots.txt
Redirect Domain www.24chasa.bg
Redirect Base 24chasa.bg
Domain IPs 104.21.86.104, 172.67.217.215, 2606:4700:3034::6815:5668, 2606:4700:3034::ac43:d9d7
Redirect IPs 104.21.86.104, 172.67.217.215, 2606:4700:3034::6815:5668, 2606:4700:3034::ac43:d9d7
Response IP 104.21.86.104
Found Yes
Hash 8682ed65ed9cafe60703f7f84d7dfd72e22c5440bd9d3f2cf60b1bf952e7fd64
SimHash 4d101b75e713

Groups

scrapy

Rule Path
Disallow /

*

Rule Path
Disallow /app/
Disallow /Shared/
Disallow /shared/
Disallow /igra/
Disallow /Search
Disallow /Mobile/Index/*
Disallow /Article/RecommendArticle/
Allow /tv/$
Disallow /tv/*

Other Records

Field Value
sitemap https://www.24chasa.bg/sitemap.xml
sitemap https://www.24chasa.bg/sitemap-rubrics.xml
sitemap https://www.24chasa.bg/sitemap-static.xml