thecybersecurityreview.com
robots.txt
Robots Exclusion Standard data for thecybersecurityreview.com
Resource Scan
Scan Details
| Site Domain | thecybersecurityreview.com |
| Base Domain | thecybersecurityreview.com |
| Scan Status | Ok |
| Last Scan | 2024-11-12T16:19:47+00:00 |
| Next Scan | 2024-11-19T16:19:47+00:00 |
Last Scan
| Scanned | 2024-11-12T16:19:47+00:00 |
| URL | https://thecybersecurityreview.com/robots.txt |
| Redirect | https://www.thecybersecurityreview.com/robots.txt |
| Redirect Domain | www.thecybersecurityreview.com |
| Redirect Base | thecybersecurityreview.com |
| Domain IPs | 44.233.221.199 |
| Redirect IPs | 44.233.221.199 |
| Response IP | 44.233.221.199 |
| Found | Yes |
| Hash | ec3d8fec5732577d3bd20d55b061f4c10d1d80da47fba90d8fc40c5d1b5e971d |
| SimHash | 7915dcd1d791 |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Disallow | / | Disallow all bots by default |
| Disallow | /scripts/ | - |
| Disallow | /*.php$ | - |
| Disallow | /zlib/ | - |
| Disallow | /lib/ | - |
| Disallow | /utils/ | - |
| Disallow | /magazine/ | - |
| Disallow | /admin/ | - |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.thecybersecurityreview.com/thecybersecurity_news_sitemap.xml |
| sitemap | https://www.thecybersecurityreview.com/thecybersecurityreviewnewnews_sitemap.xml |
| sitemap | https://www.thecybersecurityreview.com/vertical_sitemap.xml |
| sitemap | https://www.thecybersecurityreview.com/all_in_one_sitemap_main.xml |
| sitemap | https://www.thecybersecurityreview.com/contributors_sitemap.xml |
Comments