nlnet.nl
robots.txt

Robots Exclusion Standard data for nlnet.nl

Resource Scan

Scan Details

Site Domain nlnet.nl
Base Domain nlnet.nl
Scan Status Ok
Last Scan2024-10-01T11:00:27+00:00
Next Scan 2024-10-02T11:00:27+00:00

Last Scan

Scanned2024-10-01T11:00:27+00:00
URL https://nlnet.nl/robots.txt
Domain IPs 2a04:52c0:108:b1af::cafe, 5.255.103.93
Response IP 5.255.103.93
Found Yes
Hash 61b43cf3b5f372d8636650706cab7822e1ea51833bdb9d0d5fee66b6686fe2e5
SimHash a929705acad2

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /help*
Disallow /*?*

Other Records

Field Value
sitemap https://nlnet.nl/sitemap.xml

Comments

  • Disallow indexing of pages that take query parameters.