24plovdiv.bg
robots.txt

Robots Exclusion Standard data for 24plovdiv.bg

Resource Scan

Scan Details

Site Domain 24plovdiv.bg
Base Domain 24plovdiv.bg
Scan Status Ok
Last Scan2024-11-12T09:05:50+00:00
Next Scan 2024-11-19T09:05:50+00:00

Last Scan

Scanned2024-11-12T09:05:50+00:00
URL https://24plovdiv.bg/robots.txt
Redirect https://www.24plovdiv.bg/robots.txt
Redirect Domain www.24plovdiv.bg
Redirect Base 24plovdiv.bg
Domain IPs 104.21.74.48, 172.67.155.61, 2606:4700:3031::ac43:9b3d, 2606:4700:3035::6815:4a30
Redirect IPs 104.21.74.48, 172.67.155.61, 2606:4700:3031::ac43:9b3d, 2606:4700:3035::6815:4a30
Response IP 104.21.74.48
Found Yes
Hash 449f67b3bee676cd922d23c1b86b09fab9fd1b079a87d86cb612cd9182586afb
SimHash 55000b7d771b

Groups

scrapy

Rule Path
Disallow /

*

Rule Path
Disallow /app/
Disallow /Shared/
Disallow /shared/
Disallow /igra/
Disallow /Search
Disallow /Mobile/Index/*
Disallow /Article/RecommendArticle/
Allow /tv/$
Disallow /tv/*

Other Records

Field Value
sitemap https://www.24plovdiv.bg/sitemap.xml
sitemap https://www.24plovdiv.bg/sitemap-rubrics.xml
sitemap https://www.24plovdiv.bg/sitemap-static.xml