nilede.tech
robots.txt
Robots Exclusion Standard data for nilede.tech
Resource Scan
Scan Details
| Site Domain | nilede.tech |
| Base Domain | nilede.tech |
| Scan Status | Ok |
| Last Scan | 2025-09-21T14:45:48+00:00 |
| Next Scan | 2025-10-21T14:45:48+00:00 |
Last Scan
| Scanned | 2025-09-21T14:45:48+00:00 |
| URL | https://nilede.tech/robots.txt |
| Domain IPs | 104.21.13.55, 172.67.132.157, 2606:4700:3035::ac43:849d, 2606:4700:3037::6815:d37 |
| Response IP | 172.67.132.157 |
| Found | Yes |
| Hash | 45d491fe7efb2939072f27a7b9ac9a4af48b5da34732a084de83b80c07891693 |
| SimHash | 691d5461c5c9 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /work |
| Allow | /work/* |
| Allow | /contact |
| Allow | /privacy-policy |
| Allow | /terms-of-service |
| Allow | /sitemap.xml |
| Allow | /robots.txt |
| Allow | /favicon.ico |
| Allow | /*.xml |
| Allow | /*.webmanifest |
| Disallow | /api/ |
| Disallow | /_next/ |
| Disallow | /cdn-cgi/ |
| Disallow | /*.json |
| Disallow | /private/ |
| Disallow | /admin/ |
| Disallow | /dashboard/ |
| Disallow | /draft/ |
| Disallow | /preview/ |
| Disallow | /temp/ |
| Disallow | /*?* |
| Disallow | /404 |
| Disallow | /500 |
| Disallow | /error |
googlebot
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /work |
| Allow | /work/* |
| Allow | /contact |
| Allow | /privacy-policy |
| Allow | /terms-of-service |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
googlebot-image
| Rule | Path |
|---|---|
| Allow | /*.jpg |
| Allow | /*.jpeg |
| Allow | /*.gif |
| Allow | /*.png |
| Allow | /*.webp |
| Allow | /*.svg |
bingbot
duckduckbot
| Rule | Path |
|---|---|
| Allow | / |
| Allow | /work |
| Allow | /work/* |
| Allow | /contact |
| Allow | /privacy-policy |
| Allow | /terms-of-service |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 2 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://nilede.tech/sitemap.xml |
Warnings
- `host` is not a known field.