umaine.edu
robots.txt

Robots Exclusion Standard data for umaine.edu

Resource Scan

Scan Details

Site Domain umaine.edu
Base Domain umaine.edu
Scan Status Ok
Last Scan2025-02-25T09:10:46+00:00
Next Scan 2025-03-27T09:10:46+00:00

Last Scan

Scanned2025-02-25T09:10:46+00:00
URL https://umaine.edu/robots.txt
Domain IPs 130.111.28.163
Response IP 130.111.28.163
Found Yes
Hash 78056be1ba9dcdee28b98c69e42930d9a98d851b7dcec3f0957ffeafd2f0d26a
SimHash 49a01f526616

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

turnitinbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

semanticscholarbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /?s=
Disallow /page/*/?s=
Disallow /search
Disallow *post_type%3Dtribe_events*
Disallow *hide_subsequent_recurrences%3D*
Disallow *tribe-bar-date%3D*
Disallow *tribe-venue%3D*
Disallow *eventDisplay%3D*
Disallow *eventDate%3D*
Disallow *paged%3D*
Disallow *pagename%3D*
Disallow *shortcode%3D*
Disallow *ical%3D*
Disallow *outlook-ical%3D*
Disallow *related_series%3D*
Disallow *tribe_geofence%3D*
Allow /events/*
Allow /event/*

Other Records

Field Value
sitemap https://umaine.edu/sitemaps.xml

Comments

  • Block search results
  • For Events Calendar