nettivaraosa.com
robots.txt

Robots Exclusion Standard data for nettivaraosa.com

Resource Scan

Scan Details

Site Domain nettivaraosa.com
Base Domain nettivaraosa.com
Scan Status Ok
Last Scan2024-05-25T03:02:13+00:00
Next Scan 2024-06-01T03:02:13+00:00

Last Scan

Scanned2024-05-25T03:02:13+00:00
URL https://nettivaraosa.com/robots.txt
Redirect https://www.nettivaraosa.com/robots.txt
Redirect Domain www.nettivaraosa.com
Redirect Base nettivaraosa.com
Domain IPs 104.26.6.164, 104.26.7.164, 172.67.68.254, 2606:4700:20::681a:6a4, 2606:4700:20::681a:7a4, 2606:4700:20::ac43:44fe
Redirect IPs 104.26.6.164, 104.26.7.164, 172.67.68.254, 2606:4700:20::681a:6a4, 2606:4700:20::681a:7a4, 2606:4700:20::ac43:44fe
Response IP 104.26.6.164
Found Yes
Hash f5cc9b9c382e6596e9f233389d05a581e0dd394a4897b1962b71ec128c8e31a2
SimHash 2fa44064adb3

Groups

*

Rule Path
Disallow /pdfPif.php
Disallow /unSubScribe.php
Disallow /extra/admin/
Disallow /extra/fonts/
Disallow /extra/mail/
Disallow /extra/siteimage/
Disallow /extra/template/
Disallow /simple_dealer/template/
Disallow /sbb/
Disallow /ajaxChangeEntity.php
Disallow /en/ajaxChangeEntity.php
Disallow /en/updateCounter.php
Disallow /en/bannerCounter.php
Disallow /updateCounter.php
Disallow /bannerCounter.php

moget
ichiro

Rule Path
Disallow /

naverbot
yeti

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

baiduspider
baiduspider-video
baiduspider-image

Rule Path
Disallow /

yandexbot/3.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

http://yandex.com/bots

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 4

crystalsemanticsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.nettivaraosa.com/sitemap_index.xml