knowledgehut.com
robots.txt
Robots Exclusion Standard data for knowledgehut.com
Resource Scan
Scan Details
| Site Domain | knowledgehut.com |
| Base Domain | knowledgehut.com |
| Scan Status | Ok |
| Last Scan | 2024-06-16T22:43:05+00:00 |
| Next Scan | 2024-07-16T22:43:05+00:00 |
Last Scan
| Scanned | 2024-06-16T22:43:05+00:00 |
| URL | https://knowledgehut.com/robots.txt |
| Redirect | https://www.knowledgehut.com/robots.txt |
| Redirect Domain | www.knowledgehut.com |
| Redirect Base | knowledgehut.com |
| Domain IPs | 13.225.4.125, 13.225.4.4, 13.225.4.60, 13.225.4.62 |
| Redirect IPs | 13.225.4.125, 13.225.4.4, 13.225.4.60, 13.225.4.62, 2600:9000:21b4:1a00:14:1635:37c0:93a1, 2600:9000:21b4:3a00:14:1635:37c0:93a1, 2600:9000:21b4:4800:14:1635:37c0:93a1, 2600:9000:21b4:6600:14:1635:37c0:93a1, 2600:9000:21b4:7a00:14:1635:37c0:93a1, 2600:9000:21b4:7e00:14:1635:37c0:93a1, 2600:9000:21b4:8400:14:1635:37c0:93a1, 2600:9000:21b4:ac00:14:1635:37c0:93a1 |
| Response IP | 13.225.4.125 |
| Found | Yes |
| Hash | 9a98c6a4135190cf4dc2fec198c515cf96910d9d0592e23c51e46147cb987e80 |
| SimHash | 0c25cf43ebbe |
Groups
*
| Rule | Path | Comment |
|---|---|---|
| Allow | /* | allows to crawl all levels of url |
| Allow | /events/* | - |
| Allow | /blog/* | allows to crawl blog |
| Allow | /*.xml | - |
| Allow | /tutorials/* | - |
| Allow | /practice_tests/*/Questions/*%3D | - |
| Disallow | /auth/* | Restrcicts crawling Auth urls |
| Disallow | /*? | Restrcicts crawling of URL containg "?" |
| Disallow | /*%3D | Restrcicts crawling of URL containg "=" |
| Disallow | /*_ | Restrcicts crawling of URL containg "_" |
| Disallow | /*payment | Restrcicts crawling of URL containg payment term |
| Disallow | /*.php | - |
| Disallow | /*.index | - |
| Disallow | /home | - |
| Disallow | /blog/*? | blocks crawling of url containg "?" |
| Disallow | /wp-blog* | - |
| Disallow | /kh-prepo/* | Restrcicts crawling Kh-Prepo Urls |
| Disallow | /submit-review/* | Restrcicts crawling Submit Review URLs |
| Disallow | /pay/* | Restrcicts crawling pay URLs |
| Disallow | /lp/ | - |
| Disallow | /drip/ | - |
| Disallow | /blog/%5Bcategory%5D/%5Bslug%5D | - |
| Disallow | /blog/category/%5Bslug%5D | - |
| Disallow | /blog/author/%5Bslug%5D | - |
| Disallow | /blog/subscribe/%5B...index%5D | - |
| Disallow | /blog/tag/%5Bslug%5D | - |
| Disallow | /blog/type/%5Bslug%5D | - |
| Disallow | /online | - |
| Disallow | /online/* | - |
| Disallow | /metrics | - |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.knowledgehut.com/sitemap.xml |