Log In Sign Up

ora.ox.ac.uk
robots.txt

Robots Exclusion Standard data for ora.ox.ac.uk

Resource Scan

Scan Details

Site Domain ora.ox.ac.uk
Base Domain ox.ac.uk
Scan Status Ok
Last Scan3/3/2025, 8:58:29 PM
Next Scan 4/2/2025, 8:58:29 PM

Last Scan

Scanned3/3/2025, 8:58:29 PM
URL https://ora.ox.ac.uk/robots.txt
Domain IPs 129.67.246.216
Response IP 129.67.246.216
Found Yes
Hash 0a4a967a306f3b8fc1dad6a724966553e691827e721195b5ae7dc7953c6c2719
SimHash 0d35d231e5f9

Groups

*

Rule Path
Disallow /

core
googlebot
adsbot-google
bingbot
slurp
duckduckbot
baiduspider
yandexbot
mail.ru
exabot
ia_archiver
twitterbot
facebot
facebookexternalhit

Rule Path
Allow /
Disallow /export_csv_search_results
Disallow /stats
Disallow /*query_analytics
Back to top

Other Records

Field Value
sitemap https://ora.ox.ac.uk/sitemaps/sitemap1.txt
sitemap https://ora.ox.ac.uk/sitemaps/sitemap2.txt
sitemap https://ora.ox.ac.uk/sitemaps/sitemap3.txt
sitemap https://ora.ox.ac.uk/sitemaps/sitemap4.txt
sitemap https://ora.ox.ac.uk/sitemaps/sitemap5.txt
sitemap https://ora.ox.ac.uk/sitemaps/sitemap6.txt
sitemap https://ora.ox.ac.uk/sitemaps/sitemap.txt
Back to top

Comments

  • Global disallow rule
  • Bot specific settings, in case we need them.
  • SITEMAPS are appended automatically after this line
Back to top

Warnings

  • 1 invalid line.
Back to top