Log In Sign Up

moegirl.icu
robots.txt

Robots Exclusion Standard data for moegirl.icu

Resource Scan

Scan Details

Site Domain moegirl.icu
Base Domain moegirl.icu
Scan Status Ok
Last Scan2025-10-31T12:23:45+00:00
Next Scan 2025-11-30T12:23:45+00:00

Last Scan

Scanned2025-10-31T12:23:45+00:00
URL https://moegirl.icu/robots.txt
Domain IPs 104.21.15.93, 172.67.205.172, 2606:4700:3031::6815:f5d, 2606:4700:3037::ac43:cdac
Response IP 172.67.205.172
Found Yes
Hash 04f2351a81ce276827fa11c188caa2ae70660818eea62569f832ecb26e9b4863
SimHash 691adb428f10

Groups

amazonbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

blexbot

Rule Path
Disallow /

*

Rule Path
Disallow /*%26mobileaction%3D
Disallow /*%26oldid%3D
Disallow /index.php?title=*&mobileaction=
Disallow /index.php?title=*&oldid=
Allow /index.php?title=*&variant=
Disallow /index.php
Disallow /Special%3A
Disallow /Special%3A
Back to top

Comments

  • Disable for redirect
  • Disallow: /zh/
  • Sitemap: https://moegirl.icu/sitemap.xml
Back to top