youth4work.com
robots.txt
Robots Exclusion Standard data for youth4work.com
Resource Scan
Scan Details
| Site Domain | youth4work.com |
| Base Domain | youth4work.com |
| Scan Status | Ok |
| Last Scan | 2024-10-03T08:01:57+00:00 |
| Next Scan | 2024-10-10T08:01:57+00:00 |
Last Scan
| Scanned | 2024-10-03T08:01:57+00:00 |
| URL | https://youth4work.com/robots.txt |
| Domain IPs | 20.192.170.11, 70.32.89.236 |
| Response IP | 20.192.170.11 |
| Found | Yes |
| Hash | 20c4b44a0a475eced243c45e51e35a92d93dfd17e902ae3526ddfc2b0c5ee23f |
| SimHash | 0c76db5263c1 |
Groups
googlebot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /bn/ |
| Disallow | /de/ |
| Disallow | /fr/ |
| Disallow | /gu/ |
| Disallow | /it/ |
| Disallow | /ja/ |
| Disallow | /kn/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /or/ |
| Disallow | /pa/ |
| Disallow | /ru/ |
| Disallow | /ta/ |
| Disallow | /te/ |
| Disallow | /tr/ |
| Disallow | /ur/ |
| Disallow | /zh/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /es/ |
| Disallow | /*?*returnurl=* |
| Disallow | /*?*refererurl=* |
applebot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
bingbot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
msnbot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
slurp
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
googlebot-image
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /bn/ |
| Disallow | /de/ |
| Disallow | /fr/ |
| Disallow | /gu/ |
| Disallow | /it/ |
| Disallow | /ja/ |
| Disallow | /kn/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /or/ |
| Disallow | /pa/ |
| Disallow | /ru/ |
| Disallow | /ta/ |
| Disallow | /te/ |
| Disallow | /tr/ |
| Disallow | /ur/ |
| Disallow | /zh/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /es/ |
googlebot-news
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /bn/ |
| Disallow | /de/ |
| Disallow | /fr/ |
| Disallow | /gu/ |
| Disallow | /it/ |
| Disallow | /ja/ |
| Disallow | /kn/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /or/ |
| Disallow | /pa/ |
| Disallow | /ru/ |
| Disallow | /ta/ |
| Disallow | /te/ |
| Disallow | /tr/ |
| Disallow | /ur/ |
| Disallow | /zh/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /es/ |
googlebot-video
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /bn/ |
| Disallow | /de/ |
| Disallow | /fr/ |
| Disallow | /gu/ |
| Disallow | /it/ |
| Disallow | /ja/ |
| Disallow | /kn/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /or/ |
| Disallow | /pa/ |
| Disallow | /ru/ |
| Disallow | /ta/ |
| Disallow | /te/ |
| Disallow | /tr/ |
| Disallow | /ur/ |
| Disallow | /zh/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /es/ |
googlebot-mobile
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /bn/ |
| Disallow | /de/ |
| Disallow | /fr/ |
| Disallow | /gu/ |
| Disallow | /it/ |
| Disallow | /ja/ |
| Disallow | /kn/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /or/ |
| Disallow | /pa/ |
| Disallow | /ru/ |
| Disallow | /ta/ |
| Disallow | /te/ |
| Disallow | /tr/ |
| Disallow | /ur/ |
| Disallow | /zh/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /es/ |
mediapartners-google
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
adsbot-google
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
adsbot-google-mobile
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
adsbot-google-mobile-apps
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
baiduspider
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /es/ |
| Disallow | /hi/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
seznambot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
teoma
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
yandex
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /hi/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
yeti
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
msnbot-media
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
duckduckbot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
daumoa
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
orangebot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
orangebot-collector
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
stackrambler
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
twitterbot/1.0
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
mail.ru_bot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /hi/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
naverbot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
vebidoobot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
mj12bot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
sogou
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
rogerbot
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
deepcrawl
| Rule | Path |
|---|---|
| Disallow | /Documents/New_Openings_Resume/ |
| Disallow | /Documents/Colleges/attache/ |
| Disallow | /Documents/PrepTest/StudyMaterial/ |
| Disallow | /tr/ |
| Disallow | /gu/ |
| Disallow | /fr/ |
| Disallow | /te/ |
| Disallow | /ta/ |
| Disallow | /ml/ |
| Disallow | /mr/ |
| Disallow | /ur/ |
| Disallow | /kn/ |
| Disallow | /ar/ |
| Disallow | /pt/ |
| Disallow | /ru/ |
| Disallow | /de/ |
| Disallow | /or/ |
| Disallow | /ja/ |
| Disallow | /pa/ |
| Disallow | /zh/ |
| Disallow | /bn/ |
| Disallow | /it/ |
| Disallow | /es/ |
*
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 24 invalid lines.