windowsfiles.jp
robots.txt

Robots Exclusion Standard data for windowsfiles.jp

Resource Scan

Scan Details

Site Domain windowsfiles.jp
Base Domain windowsfiles.jp
Scan Status Ok
Last Scan2024-06-27T16:41:32+00:00
Next Scan 2024-07-04T16:41:32+00:00

Last Scan

Scanned2024-06-27T16:41:32+00:00
URL https://windowsfiles.jp/robots.txt
Domain IPs 104.21.55.141, 172.67.148.242, 2606:4700:3030::6815:378d, 2606:4700:3034::ac43:94f2
Response IP 104.21.55.141
Found Yes
Hash ba3df0fc29e023d30092bc2bbde239a0c95d69c99ff7cc2d6aa4b18be0aa71e6
SimHash 531c98604ab8

Groups

*

Rule Path
Disallow /cgi-bin
Disallow /cgi-sys
Allow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mediatoolkit

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.windowsfiles.jp/google_sitemap.gz

Comments

  • Crawl-Delay: 20