are-mag.com
robots.txt
Robots Exclusion Standard data for are-mag.com
Resource Scan
Scan Details
| Site Domain | are-mag.com |
| Base Domain | are-mag.com |
| Scan Status | Ok |
| Last Scan | 2026-01-04T13:31:42+00:00 |
| Next Scan | 2026-01-11T13:31:42+00:00 |
Last Scan
| Scanned | 2026-01-04T13:31:42+00:00 |
| URL | https://are-mag.com/robots.txt |
| Domain IPs | 104.26.4.183, 104.26.5.183, 172.67.75.35, 2606:4700:20::681a:4b7, 2606:4700:20::681a:5b7, 2606:4700:20::ac43:4b23 |
| Response IP | 104.26.5.183 |
| Found | Yes |
| Hash | 880de13a2afd4cf991512059d92d9072b6357975ba8bd6ab4fb0785650270d21 |
| SimHash | 43a21ea6e113 |
Groups
oai-searchbot
| Rule | Path | Comment |
|---|---|---|
| Allow | / | - |
| Disallow | /wp-admin | - |
| Disallow | /wp-content/plugins/ | block access to plugins |
| Disallow | /wp-login.php | block access to management |
| Disallow | /feed | block feeds |
| Disallow | /search/ | block internal search results |
| Disallow | /?s= | block access to internal search result pages |
| Disallow | /?p= | block access to pages for which permalinks fails |
| Disallow | /%26p%3D | block access to pages for which permalinks fails |
| Disallow | /%26preview%3D | block preview |
| Disallow | /tag/ | block tags |
| Disallow | /author/ | blocking author pages |
chatgpt-user
| Rule | Path | Comment |
|---|---|---|
| Allow | / | - |
| Disallow | /wp-admin | - |
| Disallow | /wp-content/plugins/ | block access to plugins |
| Disallow | /wp-login.php | block access to management |
| Disallow | /feed | block feeds |
| Disallow | /search/ | block internal search results |
| Disallow | /?s= | block access to internal search result pages |
| Disallow | /?p= | block access to pages for which permalinks fails |
| Disallow | /%26p%3D | block access to pages for which permalinks fails |
| Disallow | /%26preview%3D | block preview |
| Disallow | /tag/ | block tags |
| Disallow | /author/ | blocking author pages |
gptbot
| Rule | Path | Comment |
|---|---|---|
| Allow | / | - |
| Disallow | /wp-admin | - |
| Disallow | /wp-content/plugins/ | block access to plugins |
| Disallow | /wp-login.php | block access to management |
| Disallow | /feed | block feeds |
| Disallow | /search/ | block internal search results |
| Disallow | /?s= | block access to internal search result pages |
| Disallow | /?p= | block access to pages for which permalinks fails |
| Disallow | /%26p%3D | block access to pages for which permalinks fails |
| Disallow | /%26preview%3D | block preview |
| Disallow | /tag/ | block tags |
| Disallow | /author/ | blocking author pages |
*
| Rule | Path | Comment |
|---|---|---|
| Allow | / | - |
| Allow | /wp-admin/admin-ajax.php | - |
| Disallow | /wp-admin | - |
| Disallow | /wp-content/plugins/ | block access to plugins |
| Disallow | /wp-login.php | block access to management |
| Disallow | /feed | block feeds |
| Disallow | /search/ | block internal search results |
| Disallow | /?s= | block access to internal search result pages |
| Disallow | /?p= | block access to pages for which permalinks fails |
| Disallow | /%26p%3D | block access to pages for which permalinks fails |
| Disallow | /%26preview%3D | block preview |
| Disallow | /tag/ | block tags |
| Disallow | /author/ | blocking author pages |