semgrep.dev
robots.txt

Robots Exclusion Standard data for semgrep.dev

Resource Scan

Scan Details

Site Domain semgrep.dev
Base Domain semgrep.dev
Scan Status Ok
Last Scan2025-08-18T23:07:58+00:00
Next Scan 2025-09-01T23:07:58+00:00

Last Scan

Scanned2025-08-18T23:07:58+00:00
URL https://semgrep.dev/robots.txt
Domain IPs 3.165.102.122, 3.165.102.128, 3.165.102.44, 3.165.102.9
Response IP 3.165.102.9
Found Yes
Hash 09538ecfa8904e880c3b2b210596e5118fe48705bd4206f221d3917a977d9c2f
SimHash 6f0c04504511

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
sitemap https://semgrep.dev/assets/sitemap
sitemap https://semgrep.dev/docs/sitemap.xml

Comments

  • https://www.robotstxt.org/robotstxt.html