git.exozy.me
robots.txt

Robots Exclusion Standard data for git.exozy.me

Resource Scan

Scan Details

Site Domain git.exozy.me
Base Domain exozy.me
Scan Status Ok
Last Scan2024-10-04T09:47:57+00:00
Next Scan 2024-10-05T09:47:57+00:00

Last Scan

Scanned2024-10-04T09:47:57+00:00
URL https://git.exozy.me/robots.txt
Domain IPs 2600:1700:5db5:4b30::48, 99.46.207.19
Response IP 99.46.207.19
Found Yes
Hash 398c94c582e374ae77d5b4e5cca9ed8ebd85a3ffc76748f07247fdd6cf15a1b1
SimHash c01a5c02c352

Groups

*

Rule Path
Disallow /*/*/archive/

Other Records

Field Value
crawl-delay 2

bytespider

Rule Path
Disallow /

Comments

  • https://forgejo.org/docs/next/admin/search-engines-indexation/#disallow-crawling-archives-to-save-disk-space