Log In Sign Up

grkids.com
robots.txt

Robots Exclusion Standard data for grkids.com

Resource Scan

Scan Details

Site Domain grkids.com
Base Domain grkids.com
Scan Status Ok
Last Scan2024-10-01T07:45:46+00:00
Next Scan 2024-10-08T07:45:46+00:00

Last Scan

Scanned2024-10-01T07:45:46+00:00
URL https://grkids.com/robots.txt
Domain IPs 98.83.67.209
Response IP 98.83.67.209
Found Yes
Hash 2cba690a4717009d3785d5bdea122eb7c23376aafea2abc605ead217443a77c6
SimHash 491ecc50aaf9

Groups

*

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/

Other Records

Field Value
crawl-delay 600

bingbot

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /

googlebot

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /

googlebot-image

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /

facebookexternalhit

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /

meta-externalagent

Rule Path
Disallow /wp-login.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /

amazonbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

weborama-fetcher

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /
Back to top

Comments

  • Block all crawlers from accessing sensitive areas
  • Allow Bingbot to crawl the site but block restricted areas
  • Allow Googlebot to crawl the site but block restricted areas
  • Allow Googlebot-Image to crawl the site but block restricted areas
  • Allow Facebook crawlers to crawl the site but block restricted areas
  • Block Amazonbot entirely
  • Block Proximic
  • Block MJ12bot
  • Block ByteSpider
  • Block Weborama
  • Block CriteoBot
  • Block AhrefsBot
Back to top