Log In Sign Up

mantalk.gr
robots.txt

Robots Exclusion Standard data for mantalk.gr

Resource Scan

Scan Details

Site Domain mantalk.gr
Base Domain mantalk.gr
Scan Status Ok
Last Scan2024-05-11T07:58:24+00:00
Next Scan 2024-05-18T07:58:24+00:00

Last Scan

Scanned2024-05-11T07:58:24+00:00
URL https://mantalk.gr/robots.txt
Domain IPs 104.21.43.44, 172.67.219.93, 2606:4700:3032::6815:2b2c, 2606:4700:3035::ac43:db5d
Response IP 104.21.43.44
Found Yes
Hash 90cd7291ac13b11df32f584f4d34d217ff39863264d3a2666a27d8e2c20da339
SimHash 092471436ff5

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /wp-includes/
Disallow /tag/
Disallow /category/
Disallow /author/
Disallow /cgi-bin/
Disallow /tmp/
Disallow /cache/
Disallow /backup/
Disallow *.pdf
Disallow *.docx
Disallow *.xlsx
Back to top

Comments

  • Disallow access to plugin directories
  • Disallow access to themes directories
  • Disallow access to core WordPress directories that are non-essential
  • SEO-related disallows
  • Additional folders
  • Disallow file types that are not usually necessary for crawlers
Back to top