tech-review.jp
robots.txt

Robots Exclusion Standard data for tech-review.jp

Resource Scan

Scan Details

Site Domain tech-review.jp
Base Domain tech-review.jp
Scan Status Ok
Last Scan2024-11-03T12:01:23+00:00
Next Scan 2024-12-03T12:01:23+00:00

Last Scan

Scanned2024-11-03T12:01:23+00:00
URL https://tech-review.jp/robots.txt
Domain IPs 162.43.96.30
Response IP 162.43.96.30
Found Yes
Hash 691d728c69c00e3113b8b76c4986e4e0973e308a7465ebb1a446e900c89852cf
SimHash dbb2e74a9fb0

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/themes/swell_child/assets/js/
Disallow /cheapest/wp-admin/
Allow /cheapest/wp-admin/admin-ajax.php
Disallow /iosys*/
Disallow /yahoo*/
Disallow /qoo10*/
Disallow /ymobile*/
Disallow /amazon-*/
Disallow /rakuten-*/
Disallow /cheapest/amazon-*/
Disallow /cheapest/rakuten-*/
Disallow /*pergear*/
Disallow /*sk-*/
Disallow /*vector-*/
Disallow /*sunstella-*/
Disallow /*kojima-*/
Disallow /*sofmap-*/
Disallow /*biccamera-*/
Disallow /*au-*/
Disallow /*docomo-*/
Disallow /*softbank-*/
Disallow /*rakutenmb-*/
Disallow /*yamada-*/
Disallow /*microsoft-*/
Disallow /*gmo-*/
Disallow /*nicosuma-*/
Disallow /*2ndstreet-*/
Disallow /*nojima-*/
Disallow /cheapest/wp-content/themes/swell_child/assets/js/
Disallow /*iosys-*/
Disallow /*yahoo-*/
Disallow /*qoo10-*/
Disallow /*ymobile-*/
Disallow /*pergear-*/
Disallow /*?PageSpeed=noscript

ccbot

Rule Path
Disallow /

voltron

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

python

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

tweetmemebot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

admantx-apacas

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mappy

Rule Path
Disallow /

yak

Rule Path
Disallow /

pockey

Rule Path
Disallow /

psbot

Rule Path
Disallow /

superfeedr

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://tech-review.jp/news-sitemap.xml
sitemap https://tech-review.jp/video-sitemap.xml
sitemap https://tech-review.jp/sitemap.xml

Warnings

  • 2 invalid lines.