Log In Sign Up

moma.org
robots.txt

Robots Exclusion Standard data for moma.org

Resource Scan

Scan Details

Site Domain moma.org
Base Domain moma.org
Scan Status Ok
Last Scan2024-10-29T07:00:47+00:00
Next Scan 2024-11-28T07:00:47+00:00

Last Scan

Scanned2024-10-29T07:00:47+00:00
URL https://moma.org/robots.txt
Redirect https://www.moma.org/robots.txt
Redirect Domain www.moma.org
Redirect Base moma.org
Domain IPs 104.18.8.51, 104.18.9.51, 2606:4700::6812:833, 2606:4700::6812:933
Redirect IPs 104.18.8.51, 104.18.9.51, 2606:4700::6812:833, 2606:4700::6812:933
Response IP 104.18.8.51
Found Yes
Hash 56e6536f5f88a30c70c20c680b996679431fe311898b5523f30a77267f6f4b77
SimHash baac09c4e674

Groups

*

Rule Path
Disallow /dist/robots.*.js
Disallow /assets/robots-*.js
Disallow /calendar/events/9322
Disallow /calendar/programs/46
Disallow /calendar/programs/50
Disallow /calendar/programs/9
Disallow /calendar/exhibitions/3223
Disallow /collection/browse_results.php
Disallow /collection/object.php
Disallow /collection/artist.php
Disallow /collection/theme.php
Disallow /collection_lb/browse_results.php
Disallow /collection_ge/browse_results.php
Disallow /collection/search.php
Disallow /visit/calendar/search
Disallow /visit/calendar/filters
Disallow /visit/calendar/films?day*
Disallow /media/*
Disallow /screens/*
Disallow /visit/wifi/
Disallow /cdn-cgi/

twitterbot

Rule Path
Allow /media/*
Back to top

Comments

  • see http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • to ban all spiders from the entire site uncomment the next two lines:
  • user-agent: *
  • disallow: /
Back to top