Log In Sign Up

web.aiu.ac.jp
robots.txt

Robots Exclusion Standard data for web.aiu.ac.jp

Resource Scan

Scan Details

Site Domain web.aiu.ac.jp
Base Domain aiu.ac.jp
Scan Status Ok
Last Scan2025-08-31T04:38:36+00:00
Next Scan 2025-09-30T04:38:36+00:00

Last Scan

Scanned2025-08-31T04:38:36+00:00
URL https://web.aiu.ac.jp/robots.txt
Domain IPs 13.35.37.16, 13.35.37.25, 13.35.37.26, 13.35.37.43, 2600:9000:213e:1200:1e:edb:5280:93a1, 2600:9000:213e:3400:1e:edb:5280:93a1, 2600:9000:213e:3800:1e:edb:5280:93a1, 2600:9000:213e:600:1e:edb:5280:93a1, 2600:9000:213e:6e00:1e:edb:5280:93a1, 2600:9000:213e:7c00:1e:edb:5280:93a1, 2600:9000:213e:a600:1e:edb:5280:93a1, 2600:9000:213e:f000:1e:edb:5280:93a1
Response IP 13.35.37.16
Found Yes
Hash bba37aaf438da15271e066e04e05584b86883a788bde3fae9834bc0d9a0105ef
SimHash 056c5ac4802b

Groups

*

Rule Path
Disallow /wp-content/uploads/wpo/wpo-plugins-tables-list.json

*

Rule Path
Disallow

facebookexternalhit
facebookexternalhit/1.1
facebookexternalhit/1.1 (+h^ttp://www.facebook.com/externalhit_uatext.php)
facebookcatalog/1.0
baiduspider
baiduspider+
baidumobaider
baiduimagespider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Other Records

Field Value
sitemap https://web.aiu.ac.jp/sitemap_index.xml
Back to top

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK
  • Facebook Bot Crawl Rate Limit
Back to top