thejoai.com
robots.txt
Robots Exclusion Standard data for thejoai.com
Resource Scan
Scan Details
| Site Domain | thejoai.com |
| Base Domain | thejoai.com |
| Scan Status | Ok |
| Last Scan | 2025-10-02T10:13:36+00:00 |
| Next Scan | 2025-10-09T10:13:36+00:00 |
Last Scan
| Scanned | 2025-10-02T10:13:36+00:00 |
| URL | https://thejoai.com/robots.txt |
| Domain IPs | 104.18.2.139, 104.18.3.139, 2606:4700::6812:28b, 2606:4700::6812:38b |
| Response IP | 104.18.3.139 |
| Found | Yes |
| Hash | c689855a8528b34fd9fcf1c3b3515af11991869804d56552f93951f60c0e1794 |
| SimHash | 64005330a5b2 |
Groups
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | /search/ |
| Disallow | /cdn-cgi/ |
| Disallow | /internal/ |
| Disallow | /accounts/ |
| Disallow | /*?query= |
| Disallow | /*?page= |
| Disallow | /*?p= |
| Disallow | /*?*page= |
| Disallow | /*?*p= |
| Disallow | /*?*q= |
| Disallow | /*?*search= |
| Disallow | /*?*keyword= |
| Disallow | /*?*filter= |
| Disallow | /*?*category= |
Comments