bestattractions.org
robots.txt
Robots Exclusion Standard data for bestattractions.org
Resource Scan
Scan Details
| Site Domain | bestattractions.org |
| Base Domain | bestattractions.org |
| Scan Status | Ok |
| Last Scan | 2024-09-21T10:01:16+00:00 |
| Next Scan | 2024-09-28T10:01:16+00:00 |
Last Scan
| Scanned | 2024-09-21T10:01:16+00:00 |
| URL | https://bestattractions.org/robots.txt |
| Domain IPs | 104.21.20.164, 172.67.193.53, 2606:4700:3035::ac43:c135, 2606:4700:3037::6815:14a4 |
| Response IP | 104.21.20.164 |
| Found | Yes |
| Hash | 2aa435800c69dd38909915be26c2e3532d7ce79f6d9357fa759ba59fc68a39bb |
| SimHash | 21b021134f72 |
Groups
*
| Product | Comment |
|---|---|
| * | general rules for robots of all search engines |
| Rule | Path | Comment |
|---|---|---|
| Disallow | /cgi-bin | service folder for storing server scripts |
| Disallow | /? | all request parameters on the main |
| Disallow | /wp- | all files WP: /wp-json/, /wp-includes, /wp-content/plugins |
| Disallow | *?s= | site search |
| Disallow | *%26s%3D | site search |
| Disallow | /search/ | site search |
| Disallow | /author/ | author's archive |
| Disallow | /users/ | user archive |
| Disallow | */trackback | trackbacks, notifications in comments about a link to a web document |
| Disallow | */feed | all feeds |
| Disallow | */rss | rss feed |
| Disallow | */embed | all embeddings |
| Disallow | */wlwmanifest.xml | xml Windows Live Writer manifest file |
| Disallow | /xmlrpc.php | WordPress API file |
| Disallow | *utm*%3D | utm |
| Disallow | *openstat%3D | openstat tags |
| Disallow | /beardeddragon/ | - |
| Disallow | /ezais/ | - |
| Disallow | /porpoiseant/ | - |
| Disallow | /detroitchicago/ | - |
| Disallow | /edmontonalberta/ | - |
| Disallow | /ezoic/ | - |
| Disallow | /utilcave_com/ | - |
| Disallow | /tardisrocinante/ | - |
| Allow | */uploads | open the folder with files uploads |
| Allow | /*/*.js | open js files |
| Allow | /*/*.css | open css files |
| Allow | /wp-*.png | allow indexing images |
| Allow | /wp-*.jpg | allow indexing images |
| Allow | /wp-*.jpeg | allow indexing images |
| Allow | /wp-*.gif | allow indexing gifs |
| Allow | /wp-admin/admin-ajax.php | allow ajax |
Other Records
| Field | Value |
|---|---|
| sitemap | https://bestattractions.org/sitemap_index.xml |