nai.net
robots.txt

Robots Exclusion Standard data for nai.net

Resource Scan

Scan Details

Site Domain nai.net
Base Domain nai.net
Scan Status Ok
Last Scan2024-05-16T07:34:49+00:00
Next Scan 2024-05-23T07:34:49+00:00

Last Scan

Scanned2024-05-16T07:34:49+00:00
URL http://www.nai.net/robots.txt
Domain IPs 207.172.156.181, 207.172.156.182, 208.59.90.35, 2610:80:0:10a:214:4fff:feea:805c
Response IP 207.172.156.182
Found Yes
Hash b9037a6736affe4a55966ea0a64bc015fef76ff708a6670e76d376ee65c10708
SimHash e221d870e295

Groups

*

Rule Path
Disallow /manager/
Disallow /connectors/
Disallow /assets/components/
Disallow /config.core.php
Disallow /assets/contact/
Disallow /assets/bundles/
Disallow /images-OLD/
Disallow /config.core.php
Disallow /fwd.php
Disallow /pass.php
Disallow /business/manager/
Disallow /business/connectors/
Disallow /business/assets/components/
Disallow /business/config.core.php
Disallow /welcome/manager/
Disallow /welcome/connectors/
Disallow /welcome/assets/components/
Disallow /welcome/config.core.php

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap https://www.rcn.com/rcn-sitemap.xml
sitemap https://www.rcn.com/business/rcn-business-sitemap.xml
sitemap https://www.rcn.com/welcome/rcn-welcomeresi-sitemap.xml