Log In Sign Up

macaulay.cuny.edu
robots.txt

Robots Exclusion Standard data for macaulay.cuny.edu

Resource Scan

Scan Details

Site Domain macaulay.cuny.edu
Base Domain cuny.edu
Scan Status Ok
Last Scan2025-08-21T13:31:31+00:00
Next Scan 2025-09-20T13:31:31+00:00

Last Scan

Scanned2025-08-21T13:31:31+00:00
URL https://macaulay.cuny.edu/robots.txt
Domain IPs 128.228.0.107
Response IP 128.228.0.107
Found Yes
Hash 70a3b1897758260c773604a35c30b4aaca4768c9dc87cde370149687bcaa35c8
SimHash 6a09d800c5b5

Groups

googlebot
bingbot
msnbot
twitterbot
slurp
duckduckbot

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 30

*

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://macaulay.cuny.edu/wp-sitemap.xml
Back to top