freshbets.net
robots.txt
Robots Exclusion Standard data for freshbets.net
Resource Scan
Scan Details
| Site Domain | freshbets.net |
| Base Domain | freshbets.net |
| Scan Status | Ok |
| Last Scan | 2026-03-22T02:20:01+00:00 |
| Next Scan | 2026-04-21T02:20:01+00:00 |
Last Scan
| Scanned | 2026-03-22T02:20:01+00:00 |
| URL | https://freshbets.net/robots.txt |
| Domain IPs | 104.21.44.175, 172.67.202.93, 2606:4700:3032::ac43:ca5d, 2606:4700:3034::6815:2caf |
| Response IP | 172.67.202.93 |
| Found | Yes |
| Hash | 8ca2e40fb93ae1613ef9041e82d0bbd3af6e88afbfa9fa19bffed2e449cbc4d2 |
| SimHash | 084c34f2cd03 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /*.js |
| Allow | /*.css |
| Allow | /*.jpg |
| Allow | /*.png |
| Allow | /*.svg |
| Allow | /*.gif |
| Allow | /*.jpeg |
| Allow | /*.webp |
| Allow | /build/frontend/*.js |
| Allow | /build/frontend/*.css |
| Allow | /build/frontend/*.jpg |
| Allow | /build/frontend/*.png |
| Allow | /build/frontend/*.svg |
| Allow | /build/frontend/*.gif |
| Allow | /build/frontend/*.jpeg |
| Allow | /build/frontend/*.webp |
| Disallow | /cdn-cgi/l/email-protection |
| Disallow | /*? |
| Disallow | /*%26 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://freshbets.net/sitemap.xml |