cn.ntdtv.com
robots.txt

Robots Exclusion Standard data for cn.ntdtv.com

Resource Scan

Scan Details

Site Domain cn.ntdtv.com
Base Domain ntdtv.com
Scan Status Ok
Last Scan2024-09-25T18:59:40+00:00
Next Scan 2024-10-02T18:59:40+00:00

Last Scan

Scanned2024-09-25T18:59:40+00:00
URL https://cn.ntdtv.com/robots.txt
Domain IPs 104.18.28.4, 104.18.29.4, 2606:4700::6812:1c04, 2606:4700::6812:1d04
Response IP 104.18.29.4
Found Yes
Hash 285073247412893c143407952abbc151484a8d7ccc59a0f14d4f0c648ed99297
SimHash bc14d314eba2

Groups

*

Rule Path
Disallow */uncategorized/*
Disallow /wp-includes/*
Disallow /wp-admin/*
Disallow /wp-content/plugins/*
Disallow /assets/plugins/*
Disallow /feedback/*
Disallow /*?q=*
Disallow /*?ref

twitterbot

Rule Path
Allow /*?*utm_

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

polecatbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap.xml.gz
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap_pages_gb.xml.gz
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap_pages_b5.xml.gz
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap_news_gb.xml.gz
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap_news_b5.xml.gz
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap_terms_gb.xml.gz
sitemap https://www.ntdtv.com/assets/uploads/sitemap/sitemap_terms_b5.xml.gz