Log In Sign Up

omny.fm
robots.txt

Robots Exclusion Standard data for omny.fm

Resource Scan

Scan Details

Site Domain omny.fm
Base Domain omny.fm
Scan Status Ok
Last Scan2024-11-10T04:17:47+00:00
Next Scan 2024-11-17T04:17:47+00:00

Last Scan

Scanned2024-11-10T04:17:47+00:00
URL https://omny.fm/robots.txt
Domain IPs 104.22.4.5, 104.22.5.5, 172.67.7.242, 2606:4700:10::6816:405, 2606:4700:10::6816:505, 2606:4700:10::ac43:7f2
Response IP 104.22.5.5
Found Yes
Hash 06dbfe00d1cad4607d0f82e9881133c2a2c9a01a8c6d35a0d6d8a2be98a0e79a
SimHash e846d2b1c874

Groups

*

Rule Path
Allow /

test certificate info

Rule Path
Disallow /
Back to top

Comments

  • Attempt to block noisy test crawler that accesses malformed URLs
  • See http://www.diaryofaninja.com/blog/2012/10/02/the-pitfalls-of-cut-and-paste-coding
Back to top