amourgirls.club
robots.txt

Robots Exclusion Standard data for amourgirls.club

Resource Scan

Scan Details

Site Domain amourgirls.club
Base Domain amourgirls.club
Scan Status Ok
Last Scan2024-05-28T14:02:46+00:00
Next Scan 2024-06-27T14:02:46+00:00

Last Scan

Scanned2024-05-28T14:02:46+00:00
URL https://amourgirls.club/robots.txt
Domain IPs 104.21.64.99, 172.67.181.20, 2606:4700:3032::6815:4063, 2606:4700:3033::ac43:b514
Response IP 172.67.181.20
Found Yes
Hash 033b22ecd71c133d89a2ca657a859c23e58dfa4472e0180d9994dc81943f1df1
SimHash 6009c98366b5

Groups

*

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

googlebot
googlebot-image
yandex
msnbot
msnbot-media
bingbot
yahoo-blogs
yahoo-mmcrawler

Rule Path
Disallow /image/

Other Records

Field Value
crawl-delay 600

Comments

  • robots.txt
  • disallow all
  • but allow only important bots
  • Directories