Log In Sign Up

kepogaul.com
robots.txt

Robots Exclusion Standard data for kepogaul.com

Resource Scan

Scan Details

Site Domain kepogaul.com
Base Domain kepogaul.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-08T13:51:05+00:00
Next Scan 2024-11-07T13:51:05+00:00

Last Successful Scan

Scanned2024-09-09T09:49:43+00:00
URL https://kepogaul.com/robots.txt
Redirect https://www.kepogaul.com/robots.txt
Redirect Domain www.kepogaul.com
Redirect Base kepogaul.com
Domain IPs 104.21.31.42, 172.67.174.243, 2606:4700:3033::6815:1f2a, 2606:4700:3035::ac43:aef3
Redirect IPs 104.21.31.42, 172.67.174.243, 2606:4700:3033::6815:1f2a, 2606:4700:3035::ac43:aef3
Response IP 104.21.31.42
Found Yes
Hash 4dd86570bf22c4040e3ec4cc257921f800c3e178f92d4eb07c4c559627fccc42
SimHash 45345b95c7eb

Groups

googlebot
bingbot
baiduspider
yahoo! slurp
duckduckbot
yandexbot

Rule Path
Disallow */js/excluded
Disallow /test
Disallow /draft
Disallow /staging
Disallow /wp-admin/
Disallow /wp-login/
Disallow /xmlrpc.php
Disallow /cgi-bin/
Disallow *%26preview%3D
Disallow */comments
Disallow */feed
Disallow */search
Disallow */trackback
Disallow *?
Disallow *%26p%3D
Allow */amp$
Allow */?amp$

*

Rule Path
Disallow /
Back to top

Comments

  • Grouped User-agents
  • Exclude Non-Primary JS
  • Disallow /test /draft /staging
  • Disallow wp- & preview
  • Disallow feed, trackback, query string
  • Allow amp
  • To put as comment when Disallow everything
  • Disallow everyone else
Back to top