Log In Sign Up

nietylkopasta.pl
robots.txt

Robots Exclusion Standard data for nietylkopasta.pl

Resource Scan

Scan Details

Site Domain nietylkopasta.pl
Base Domain nietylkopasta.pl
Scan Status Ok
Last Scan2024-07-04T18:34:15+00:00
Next Scan 2024-07-11T18:34:15+00:00

Last Scan

Scanned2024-07-04T18:34:15+00:00
URL https://nietylkopasta.pl/robots.txt
Domain IPs 195.201.140.93
Response IP 195.201.140.93
Found Yes
Hash aefc39c0e9d75bfcc01d194adcde062386db0ea09921ef32d57dcce753001b9c
SimHash d303592a2301

Groups

scrapy

Rule Path
Allow /
Back to top

Comments

  • User-agent: *
  • Disallow: /wp-admin/
  • Disallow: /wp-includes/
  • Disallow: /wp-trackback
  • Disallow: /wp-feed
  • Disallow: /wp-comments
  • Disallow: /wp-content/plugins
  • Disallow: /wp-content/themes
  • Disallow: /wp-login.php
  • Disallow: /wp-register.php
  • Disallow: /feed
  • Disallow: /trackback
  • Disallow: /cgi-bin
  • Disallow: /comments
  • Disallow: *?s=
Back to top