agentnews.my.id
robots.txt

Robots Exclusion Standard data for agentnews.my.id

Resource Scan

Scan Details

Site Domain agentnews.my.id
Base Domain agentnews.my.id
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-09-29T12:22:39+00:00
Next Scan 2024-12-28T12:22:39+00:00

Last Successful Scan

Scanned2023-08-13T11:27:56+00:00
URL https://agentnews.my.id/robots.txt
Redirect https://www.agentnews.my.id/robots.txt
Redirect Domain www.agentnews.my.id
Redirect Base agentnews.my.id
Domain IPs 216.239.32.21, 216.239.34.21, 216.239.36.21, 216.239.38.21
Redirect IPs 142.251.175.121, 2404:6800:4003:c05::79
Response IP 142.250.4.121
Found Yes
Hash 5314112157bb764203486de22cf03bf2c34635c202fc5cf9ba55aead1cdd28a0
SimHash 69049a704792

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /search
Allow /

Other Records

Field Value
sitemap https://www.agentnews.my.id/sitemap.xml