toolman.gr
robots.txt

Robots Exclusion Standard data for toolman.gr

Resource Scan

Scan Details

Site Domain toolman.gr
Base Domain toolman.gr
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-23T23:38:04+00:00
Next Scan 2025-11-21T23:38:04+00:00

Last Successful Scan

Scanned2024-10-05T16:01:28+00:00
URL https://toolman.gr/robots.txt
Domain IPs 104.21.75.235, 172.67.183.160, 2606:4700:3031::6815:4beb, 2606:4700:3032::ac43:b7a0
Response IP 104.21.75.235
Found Yes
Hash fef93a251cc8e9b5d84ccedc7feae796b0a95169dec61c171c8f2056871daa60
SimHash f454da51e6a7

Groups

*

Rule Path
Disallow /*.CVS
Disallow /*.Zip$
Disallow /*.zip$
Disallow /*.Svn$
Disallow /*.svn$
Disallow /*.Idea$
Disallow /*.idea$
Disallow /*.Sql$
Disallow /*.sql$
Disallow /*.Tgz$
Disallow /*.tgz$

facebookcatalog/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

googlebot-image

Rule Path
Disallow

Comments

  • Crawlers Setup
  • CVS, SVN directory and dump files
  • Google Image Crawler Setup