blog.media.mit.edu
robots.txt

Robots Exclusion Standard data for blog.media.mit.edu

Resource Scan

Scan Details

Site Domain blog.media.mit.edu
Base Domain mit.edu
Scan Status Ok
Last Scan2024-05-08T16:34:15+00:00
Next Scan 2024-06-07T16:34:15+00:00

Last Scan

Scanned2024-05-08T16:34:15+00:00
URL http://blog.media.mit.edu/robots.txt
Domain IPs 2404:6800:4003:c06::79, 74.125.200.121
Response IP 64.233.170.121
Found Yes
Hash 8631034fa738200edcfc3c3ecf493cab395f3fcb1ef21615e0158adfd7c5e519
SimHash 2b04d660c693

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /search
Allow /

Other Records

Field Value
sitemap http://blog.media.mit.edu/sitemap.xml