idg.tv
robots.txt

Robots Exclusion Standard data for idg.tv

Resource Scan

Scan Details

Site Domain idg.tv
Base Domain idg.tv
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-06-10T11:46:36+00:00
Next Scan 2024-09-08T11:46:36+00:00

Last Successful Scan

Scanned2023-11-14T02:22:49+00:00
URL https://idg.tv/robots.txt
Redirect https://www.idg.tv/robots.txt
Redirect Domain www.idg.tv
Redirect Base idg.tv
Domain IPs 151.101.130.165, 151.101.194.165, 151.101.2.165, 151.101.66.165
Redirect IPs 151.101.130.165, 151.101.194.165, 151.101.2.165, 151.101.66.165
Response IP 199.232.46.165
Found Yes
Hash 40cf30f0894ac771d55180461f0afccf23a0aa9967b1758ec3ae647ea94ac00c
SimHash e8ad6a482fb2

Groups

proximic

Rule Path
Disallow

*

Rule Path
Disallow /search
Disallow /keywords
Disallow /news-keywords

Other Records

Field Value
sitemap https://www.idg.tv/sitemap.xml