Log In Sign Up

agrofoto.by
robots.txt

Robots Exclusion Standard data for agrofoto.by

Resource Scan

Scan Details

Site Domain agrofoto.by
Base Domain agrofoto.by
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-12-14T18:10:59+00:00
Next Scan 2026-03-14T18:10:59+00:00

Last Successful Scan

Scanned2023-08-05T16:09:14+00:00
URL http://agrofoto.by/robots.txt
Redirect https://rs.agrofoto.pl/robots.txt
Redirect Domain rs.agrofoto.pl
Redirect Base agrofoto.pl
Domain IPs 136.243.115.98
Redirect IPs 136.243.115.98
Response IP 136.243.115.98
Found Yes
Hash 2ccec4f22a487e8242a49f4b5ab6c5698e747cbc9717a9b9c5283771813ee796
SimHash ca1a495f0bd0

Groups

*

Rule Path
Disallow /
Back to top

Comments

  • Sample robots.txt file - ensures that a Google Appliance can still access the spider page (if configured)
  • and assumes an installation in the site root. For sites in a subfolder you must move the robots.txt file
  • to the site root and alter the paths accordingly.
  • User-agent: *
  • Crawl-delay: 10
  • Disallow : /filestore
Back to top