netcomlearning.com
robots.txt
Robots Exclusion Standard data for netcomlearning.com
Resource Scan
Scan Details
| Site Domain | netcomlearning.com |
| Base Domain | netcomlearning.com |
| Scan Status | Failed |
| Failure Reason | Scan timed out. |
| Last Scan | 2024-05-04T04:49:15+00:00 |
| Next Scan | 2024-08-02T04:49:15+00:00 |
Last Successful Scan
| Scanned | 2023-06-17T04:47:05+00:00 |
| URL | https://netcomlearning.com/robots.txt |
| Redirect | https://www.netcomlearning.com/robots.txt |
| Redirect Domain | www.netcomlearning.com |
| Redirect Base | netcomlearning.com |
| Domain IPs | 20.98.235.75 |
| Redirect IPs | 20.98.235.75 |
| Response IP | 20.98.235.75 |
| Found | Yes |
| Hash | 509a87a38bad1bcf5bbdc7746aae99d0521ccc1682db4460cac51ac224f3727e |
| SimHash | 7a8ac3574d5b |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-content/ |
| Disallow | /wp-* |
| Disallow | /.well-known |
| Disallow | /aboutnetcom/employment/ |
| Disallow | /ajax/ |
| Disallow | /blog/ |
| Disallow | /blogs/*/*/$ |
| Disallow | /ca-en/ |
| Disallow | /cart/ |
| Disallow | /courses/details.phtml?sid= |
| Disallow | /education/contact-us.phtml |
| Disallow | /education/register.phtml |
| Disallow | /mta/ |
| Disallow | /portal/ |
| Disallow | /search/ |
| Disallow | /var/ |
| Disallow | /wbf/ |
| Disallow | /wobi/ |
| Disallow | /local/product/training/ |
| Disallow | /local/certification/training/ |
| Disallow | /training/*/*-*.html |
| Disallow | /training/categories/*/*-*.html |
| Disallow | /training/certifications/*/*-*.html |
| Disallow | /training/vendors/*/*-*.html |
semrushbot
semrushbot-sa
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
adbeat_bot
adsbot
ahrefsbot
ahrefscrawler
amazonbot
applebot
awariosmartbot
barkrowler
blexbot
baiduspider
buck
ccbot
censysinspect
cincraw
coccocbot
daum
dotbot
duckduckbot
duckduckgo-favicons-bot
ezinearticleslinkscanner
go-http-client
grapeshot
greenbrowser
kocmohabt
ltx71
mail.ru
masscan
mauibot
mediatoolkitbot
mojeekbot
monsidobot
mj12bot
netsystemsresearch
newspaper
nimbostratus-bot
obot
orbbot
pagething
petalbot
pinterestbot
proximic
qwantify
redditbot
rytebot
rsiteauditor
safednsbot
seekport crawler
seokicks
serpstatbot
seznambot
sogou web spider
startmebot
stormcrawler
tpradstxtcrawler
the knowledge ai
turnitinbot
webdatastats
wellknownbot
wikido
woorankreview
yandex
yeti
zoominfobot
zgrab
zoombot
| Rule | Path |
|---|---|
| Disallow | / |
Comments