netcomlearning.com
robots.txt

Robots Exclusion Standard data for netcomlearning.com

Resource Scan

Scan Details

Site Domain netcomlearning.com
Base Domain netcomlearning.com
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2024-10-31T04:49:22+00:00
Next Scan 2025-01-29T04:49:22+00:00

Last Successful Scan

Scanned2023-06-17T04:47:05+00:00
URL https://netcomlearning.com/robots.txt
Redirect https://www.netcomlearning.com/robots.txt
Redirect Domain www.netcomlearning.com
Redirect Base netcomlearning.com
Domain IPs 20.98.235.75
Redirect IPs 20.98.235.75
Response IP 20.98.235.75
Found Yes
Hash 509a87a38bad1bcf5bbdc7746aae99d0521ccc1682db4460cac51ac224f3727e
SimHash 7a8ac3574d5b

Groups

*

Rule Path
Disallow /wp-content/
Disallow /wp-*
Disallow /.well-known
Disallow /aboutnetcom/employment/
Disallow /ajax/
Disallow /blog/
Disallow /blogs/*/*/$
Disallow /ca-en/
Disallow /cart/
Disallow /courses/details.phtml?sid=
Disallow /education/contact-us.phtml
Disallow /education/register.phtml
Disallow /mta/
Disallow /portal/
Disallow /search/
Disallow /var/
Disallow /wbf/
Disallow /wobi/
Disallow /local/product/training/
Disallow /local/certification/training/
Disallow /training/*/*-*.html
Disallow /training/categories/*/*-*.html
Disallow /training/certifications/*/*-*.html
Disallow /training/vendors/*/*-*.html

semrushbot
semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

adbeat_bot
adsbot
ahrefsbot
ahrefscrawler
amazonbot
applebot
awariosmartbot
barkrowler
blexbot
baiduspider
buck
ccbot
censysinspect
cincraw
coccocbot
daum
dotbot
duckduckbot
duckduckgo-favicons-bot
ezinearticleslinkscanner
go-http-client
grapeshot
greenbrowser
kocmohabt
ltx71
mail.ru
masscan
mauibot
mediatoolkitbot
mojeekbot
monsidobot
mj12bot
netsystemsresearch
newspaper
nimbostratus-bot
obot
orbbot
pagething
petalbot
pinterestbot
proximic
qwantify
redditbot
rytebot
rsiteauditor
safednsbot
seekport crawler
seokicks
serpstatbot
seznambot
sogou web spider
startmebot
stormcrawler
tpradstxtcrawler
the knowledge ai
turnitinbot
webdatastats
wellknownbot
wikido
woorankreview
yandex
yeti
zoominfobot
zgrab
zoombot

Rule Path
Disallow /

Comments

  • robots.txt file for www.netcomlearning.com
  • if you are looking for these, go away or you will be banned
  • pages which should never, ever be indexed
  • old URL formats we no longer support
  • really old URL formats we no longer support
  • slow this one down
  • block by user-agent