about.smartnews.com
robots.txt

Robots Exclusion Standard data for about.smartnews.com

Resource Scan

Scan Details

Site Domain about.smartnews.com
Base Domain smartnews.com
Scan Status Ok
Last Scan2024-06-02T05:47:21+00:00
Next Scan 2024-07-02T05:47:21+00:00

Last Scan

Scanned2024-06-02T05:47:21+00:00
URL https://about.smartnews.com/robots.txt
Domain IPs 13.114.39.27, 18.182.68.89, 54.178.235.104
Response IP 13.114.39.27
Found Yes
Hash 3145fd3d9c69a2009d594144c4088cd8c32a4c6c1bb9188806adfebd51c7a3e6
SimHash 98055000c711

Groups

*

Rule Path
Disallow /pdf/
Disallow /ja/pdf/

twitterbot

Rule Path
Disallow