thenews11.com
robots.txt

Robots Exclusion Standard data for thenews11.com

Resource Scan

Scan Details

Site Domain thenews11.com
Base Domain thenews11.com
Scan Status Ok
Last Scan2026-02-13T18:56:13+00:00
Next Scan 2026-02-20T18:56:13+00:00

Last Scan

Scanned2026-02-13T18:56:13+00:00
URL https://thenews11.com/robots.txt
Domain IPs 104.21.91.140, 172.67.222.19, 2606:4700:3030::6815:5b8c, 2606:4700:3032::ac43:de13
Response IP 104.21.91.140
Found Yes
Hash 6d6176139ca1969e65307e44f23d11fdeccfe5377dd9eecc9e9102a2b5594c05
SimHash a1835ce66236

Groups

*

Rule Path
Disallow /admin/
Disallow /core/
Disallow /includes/
Disallow /temp/
Disallow /private/
Disallow /*?sessionid=
Disallow /*?track=
Allow /assets/css/
Allow /assets/js/
Allow /images/

Other Records

Field Value
sitemap https://thenews11.com/sitemap.xml
sitemap https://thenews11.com/imagesitemap.xml
sitemap https://thenews11.com/staticpages.xml
sitemap https://thenews11.com/category.xml
sitemap https://thenews11.com/storysitemap.xml

Comments

  • Allow assets for rendering pages
  • Sitemap locations for better crawling