diversitynewsmagazine.org
robots.txt
Robots Exclusion Standard data for diversitynewsmagazine.org
Resource Scan
Scan Details
| Site Domain | diversitynewsmagazine.org |
| Base Domain | diversitynewsmagazine.org |
| Scan Status | Ok |
| Last Scan | 2024-11-14T19:50:13+00:00 |
| Next Scan | 2024-11-21T19:50:13+00:00 |
Last Scan
| Scanned | 2024-11-14T19:50:13+00:00 |
| URL | https://diversitynewsmagazine.org/robots.txt |
| Domain IPs | 104.21.74.109, 172.67.157.111, 2606:4700:3033::ac43:9d6f, 2606:4700:3034::6815:4a6d |
| Response IP | 172.67.157.111 |
| Found | Yes |
| Hash | ba01e13eb240ded359f11519f22c34055089c786afe08b5e0fe96495a13ebecb |
| SimHash | cb075a40aa67 |
Groups
*
| Rule | Path |
|---|---|
| Allow | /wp-admin/admin-ajax.php |
| Disallow | /wp-admin/ |
| Disallow | /wp-login.php |
| Disallow | /wp-includes/ |
| Disallow | /config/ |
| Disallow | /wp-json/ |
| Disallow | /temp/ |
| Disallow | /test-template/ |
| Disallow | /test/ |
| Disallow | /junk/ |
| Disallow | /useless_file.html |
| Disallow | /hide.me |
| Disallow | /wave.webaim.org/ |
user-agent: *
| Rule | Path |
|---|---|
| Allow | /sitecheckerbotcrawler/ |
| Disallow | /wp-login.php |
| Disallow | /wp-includes/ |
| Disallow | /wp-json/ |
| Disallow | /mediapartners-google/ |
| Disallow | /temp/ |
*
| Rule | Path |
|---|---|
| Disallow | /wp-content/uploads/wp-import-export-lite/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://diversitynewsmagazine.org/google-news-sitemap.xml |
| sitemap | https://diversitynewsmagazine.org/sitemap_index.xml |
| sitemap | https://diversitynewsmagazine.org/sitemap.xml |
| sitemap | https://diversitynewsmagazine.org/feed/ |
| sitemap | https://diversitynewsmagazine.org/news-sitemap.xml |
Comments