avenueng.com
robots.txt
Robots Exclusion Standard data for avenueng.com
Resource Scan
Scan Details
| Site Domain | avenueng.com |
| Base Domain | avenueng.com |
| Scan Status | Ok |
| Last Scan | 5/19/2025, 7:10:03 AM |
| Next Scan | 5/26/2025, 7:10:03 AM |
Last Scan
| Scanned | 5/19/2025, 7:10:03 AM |
| URL | https://avenueng.com/robots.txt |
| Domain IPs | 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001 |
| Response IP | 104.21.96.1 |
| Found | Yes |
| Hash | c6ce21637bc2918526068c7abba8872dde62612d9192b771ef550a86194908b6 |
| SimHash | 4b1058418583 |
Groups
baiduspider
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
yandexbot
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
applebot
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
duckduckbot
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
sogou web spider
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
facebookexternalhit
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
facebot
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
exabot
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
ia_archiver
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
archive.org_bot
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /wp-includes/ |
| Disallow | /wp-content/plugins/ |
| Disallow | /wp-content/themes/ |
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
*
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://avenueng.com/sitemap_index.xml |