newscityhub.com
robots.txt

Robots Exclusion Standard data for newscityhub.com

Resource Scan

Scan Details

Site Domain newscityhub.com
Base Domain newscityhub.com
Scan Status Ok
Last Scan2024-09-22T14:02:24+00:00
Next Scan 2024-09-29T14:02:24+00:00

Last Scan

Scanned2024-09-22T14:02:24+00:00
URL https://newscityhub.com/robots.txt
Domain IPs 104.26.0.148, 104.26.1.148, 172.67.72.46, 2606:4700:20::681a:194, 2606:4700:20::681a:94, 2606:4700:20::ac43:482e
Response IP 172.67.72.46
Found Yes
Hash 8a7d25943a2ee0ce3a69251c3d04dc9b8db8a3c493d534f28110ffe170aabd26
SimHash 6a159c228713

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.newscityhub.com/sitemap_index_1.xml
sitemap https://www.newscityhub.com/sitemap_index_2.xml
sitemap https://www.newscityhub.com/sitemap_index_3.xml
sitemap https://www.newscityhub.com/sitemap_index_4.xml
sitemap https://www.newscityhub.com/sitemap_index_5.xml
sitemap https://www.newscityhub.com/sitemap_index_6.xml
sitemap https://www.newscityhub.com/sitemap_index_7.xml
sitemap https://www.newscityhub.com/sitemap_index_8.xml