girlgames.ee
robots.txt

Robots Exclusion Standard data for girlgames.ee

Resource Scan

Scan Details

Site Domain girlgames.ee
Base Domain girlgames.ee
Scan Status Ok
Last Scan2024-11-11T22:40:54+00:00
Next Scan 2024-11-18T22:40:54+00:00

Last Scan

Scanned2024-11-11T22:40:54+00:00
URL https://girlgames.ee/robots.txt
Domain IPs 174.138.179.166
Response IP 174.138.179.166
Found Yes
Hash b42243941c3a252b2c9ee09876237c8268a0668760035583eb176d244d1dabbc
SimHash 492cc8c24493

Groups

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

baiduspider

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

yandex

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /readme.html
Disallow /refer/
Disallow /cgi-bin/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /comments/feed/
Disallow /wp-login.php
Disallow /?s=
Disallow /search/
Disallow /404/
Disallow /private/
Allow /wp-content/uploads/
Allow /wp-content/themes/*.css
Allow /wp-content/themes/*.js
Allow /wp-includes/*.js
Allow /wp-includes/*.css

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://girlgames.ee/sitemap.xml

Comments

  • Googlebot specific configuration
  • Bingbot specific configuration
  • Baidu specific configuration
  • Yandex specific configuration
  • General rules for all other bots
  • Block specific bad bots
  • Sitemap