warmuseum.ca
robots.txt

Robots Exclusion Standard data for warmuseum.ca

Resource Scan

Scan Details

Site Domain warmuseum.ca
Base Domain warmuseum.ca
Scan Status Ok
Last Scan2024-06-30T11:22:49+00:00
Next Scan 2024-07-30T11:22:49+00:00

Last Scan

Scanned2024-06-30T11:22:49+00:00
URL https://www.warmuseum.ca/robots.txt
Domain IPs 13.226.2.103, 13.226.2.115, 13.226.2.14, 13.226.2.7, 2600:9000:21f8:2a00:10:a1a7:7d00:93a1, 2600:9000:21f8:3400:10:a1a7:7d00:93a1, 2600:9000:21f8:3e00:10:a1a7:7d00:93a1, 2600:9000:21f8:7200:10:a1a7:7d00:93a1, 2600:9000:21f8:8a00:10:a1a7:7d00:93a1, 2600:9000:21f8:8c00:10:a1a7:7d00:93a1, 2600:9000:21f8:9200:10:a1a7:7d00:93a1, 2600:9000:21f8:b000:10:a1a7:7d00:93a1
Response IP 18.165.171.82
Found Yes
Hash 8d614a3ae05069a368ea167bd246ac30bad0d12f1e63fb3dfbd944689ebea73c
SimHash 5622fa6049b7

Groups

voltron

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

*

Rule Path
Disallow /*.json
Disallow /*.xml
Disallow /*.embed
Disallow /collections/gallery
Disallow /collections/galerie
Disallow /wp-admin/
Disallow /*/wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /*/wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 4

Comments

  • 80legs
  • 80legs' new crawler

Warnings

  • 2 invalid lines.