Log In Sign Up

metacpan.org
robots.txt

Robots Exclusion Standard data for metacpan.org

Resource Scan

Scan Details

Site Domain metacpan.org
Base Domain metacpan.org
Scan Status Ok
Last Scan2024-05-27T19:24:21+00:00
Next Scan 2024-06-26T19:24:21+00:00

Last Scan

Scanned2024-05-27T19:24:21+00:00
URL https://metacpan.org/robots.txt
Domain IPs 151.101.130.217, 151.101.194.217, 151.101.2.217, 151.101.66.217, 2a04:4e42:200::729, 2a04:4e42:400::729, 2a04:4e42:600::729, 2a04:4e42::729
Response IP 151.101.130.217
Found Yes
Hash a1bd8b58e8e868c73a260480eaef1f21876e059187dccb785583d87eab7a7894
SimHash a425d779cf65

Groups

*

Rule Path
Disallow /login/
Disallow */diff/
Disallow */source/
Disallow */source
Disallow */raw/
Disallow /account/
Disallow /search
Disallow /*?*size=*
Back to top

Other Records

Field Value
sitemap https://metacpan.org/sitemap-authors.xml.gz
sitemap https://metacpan.org/sitemap-releases.xml.gz
Back to top

Comments

  • http://www.robotstxt.org/wc/norobots.html
  • Do not allow changing the default per page as is not useful
Back to top