maclarenart.com
robots.txt

Robots Exclusion Standard data for maclarenart.com

Resource Scan

Scan Details

Site Domain maclarenart.com
Base Domain maclarenart.com
Scan Status Ok
Last Scan2026-02-01T01:28:56+00:00
Next Scan 2026-03-03T01:28:56+00:00

Last Scan

Scanned2026-02-01T01:28:56+00:00
URL https://maclarenart.com/robots.txt
Domain IPs 51.79.79.95
Response IP 51.79.79.95
Found Yes
Hash b9084c01602aec2d4d9cc4e8840d0bace22030b3d028666be871375a21d679c4
SimHash 2006ca3266b3

Groups

barkrowler

Rule Path
Disallow /

geedoproductsearch

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-json/
Disallow /xmlrpc.php
Disallow /wp-cron.php
Disallow /cart/
Disallow /checkout/
Disallow /my-account/
Disallow /*add-to-cart%3D*
Disallow /?s=
Disallow /*?s=
Disallow /events/category/
Disallow /events/tag/
Disallow /events/list/
Disallow /events/month/
Disallow /events/week/
Disallow /events/day/
Disallow /events/*/?tribe-bar-date=*
Disallow /events/*/?eventDisplay=past*
Disallow /events/*/?eventDisplay=future*
Disallow /events/*/page/
Disallow /events/*/?outlook-ical=*
Disallow /events/*?ical=*
Disallow /?s=
Disallow /wp-cron.php

googlebot

Rule Path
Allow /

googlebot-news

Rule Path
Allow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

*

Rule Path
Disallow /calendar-2/action~posterboard/
Disallow /calendar-2/action~agenda/
Disallow /calendar-2/action~oneday/
Disallow /calendar-2/action~month/
Disallow /calendar-2/action~week/
Disallow /calendar-2/action~stream/
Disallow /calendar-2/action~undefined/
Disallow /calendar-2/action~http%3A/
Disallow /calendar-2/action~default/
Disallow /calendar-2/action~poster/
Disallow /calendar-2/action~*/
Disallow /*controller%3Dai1ec_exporter_controller*
Disallow /*/action~*/

Other Records

Field Value
sitemap https://maclarenart.com/sitemap.xml

Comments

  • Block excessive events calendar crawling
  • Block search results pages
  • Block wp-cron from being crawled
  • Allow specific bots if needed
  • Set crawl delay
  • Sitemap location for all bots