gateway.reddit.com
robots.txt

Robots Exclusion Standard data for gateway.reddit.com

Resource Scan

Scan Details

Site Domain gateway.reddit.com
Base Domain reddit.com
Scan Status Ok
Last Scan2024-06-01T02:25:17+00:00
Next Scan 2024-07-01T02:25:17+00:00

Last Scan

Scanned2024-06-01T02:25:17+00:00
URL https://gateway.reddit.com/robots.txt
Domain IPs 151.101.1.140, 151.101.129.140, 151.101.193.140, 151.101.65.140
Response IP 151.101.109.140
Found Yes
Hash 85c4cee2f8166902b3aa71d30f2f49c8fbc2fca16c36ef902ca66ef1f7acd401
SimHash 6d575bc0c3d0

Groups

*

Rule Path
Disallow /

googlebot
adsbot-google
adsbot-google-mobile
adsbot-google-mobile-apps
googlebot-image
googlebot-video
bingbot
msnbot-media
applebot
claritybot
neevabot

Rule Path
Disallow /desktopapi/v1/sidebar_ads*
Disallow /desktopapi/v1/comments_page_ads*