avdouga-correct.net
robots.txt

Robots Exclusion Standard data for avdouga-correct.net

Resource Scan

Scan Details

Site Domain avdouga-correct.net
Base Domain avdouga-correct.net
Scan Status Ok
Last Scan2026-02-03T08:13:24+00:00
Next Scan 2026-02-10T08:13:24+00:00

Last Scan

Scanned2026-02-03T08:13:24+00:00
URL https://avdouga-correct.net/robots.txt
Domain IPs 150.95.59.40
Response IP 150.95.59.40
Found Yes
Hash f0fe3081cebf591887fbf802c2f21e2b0b182f7c1fe24b8b53b90ff686f672f6
SimHash 6b8458b52bd3

Groups

*

Rule Path
Allow /
Disallow /*?page=1%5B0-9%5D
Disallow /*?page=2%5B0-9%5D
Disallow /*?page=3%5B0-9%5D
Disallow /*?page=%5B4-9%5D%5B0-9%5D
Disallow /*?page=%5B1-9%5D%5B0-9%5D%5B0-9%5D
Disallow /actress/*/page/
Disallow /genre/*/page/
Disallow /maker/*/page/
Disallow /label/*/page/
Disallow /series/*/page/
Disallow /*?page=*%2F
Disallow /*?page=*%2F*
Disallow /?page=*%2F
Disallow /*?*&page=
Disallow /*?*page=*&
Disallow /*?page=*&*
Disallow /*?*sort=
Disallow /*?*order=
Disallow /cron/
Disallow /lib/
Disallow /tag/

googlebot

Rule Path
Allow /
Disallow /actress/*/page/
Disallow /genre/*/page/
Disallow /maker/*/page/
Disallow /label/*/page/
Disallow /series/*/page/
Disallow /*?*&page=
Disallow /*?*page=*&

Other Records

Field Value
crawl-delay 0

Other Records

Field Value
sitemap https://avdouga-correct.net/sitemap.xml

Comments

  • サイトマップ
  • クロール頻度の制御(深いページネーション)
  • 10ページ以降はクロールを制限
  • 不正なURLパターンのクロールを禁止
  • pageパラメータに不正な値が含まれるパターンを禁止
  • 重複コンテンツを防ぐ
  • 検索結果ページのパラメータ制限
  • 管理ディレクトリ
  • Googlebot専用設定
  • 不正なURLパターンのクロールを禁止
  • 重複コンテンツを防ぐ