nidirect.gov.uk
robots.txt
Robots Exclusion Standard data for nidirect.gov.uk
Resource Scan
Scan Details
Site Domain | nidirect.gov.uk |
Base Domain | nidirect.gov.uk |
Scan Status | Ok |
Last Scan | 2024-09-18T08:31:38+00:00 |
Next Scan | 2024-10-02T08:31:38+00:00 |
Last Scan
Scanned | 2024-09-18T08:31:38+00:00 |
URL | https://nidirect.gov.uk/robots.txt |
Redirect | https://www.nidirect.gov.uk/robots.txt |
Redirect Domain | www.nidirect.gov.uk |
Redirect Base | nidirect.gov.uk |
Domain IPs | 151.101.194.216 |
Redirect IPs | 151.101.130.216, 151.101.194.216, 151.101.2.216, 151.101.66.216 |
Response IP | 151.101.130.216 |
Found | Yes |
Hash | c7301e8288bcc2aba828320975bb5933addf0dfff9c7411140445e06d2eef2a6 |
SimHash | 3894b509cf60 |
Groups
*
Rule | Path |
---|---|
Allow | /core/*.css$ |
Allow | /core/*.css? |
Allow | /core/*.js$ |
Allow | /core/*.js? |
Allow | /core/*.gif |
Allow | /core/*.jpg |
Allow | /core/*.jpeg |
Allow | /core/*.png |
Allow | /core/*.svg |
Allow | /profiles/*.css$ |
Allow | /profiles/*.css? |
Allow | /profiles/*.js$ |
Allow | /profiles/*.js? |
Allow | /profiles/*.gif |
Allow | /profiles/*.jpg |
Allow | /profiles/*.jpeg |
Allow | /profiles/*.png |
Allow | /profiles/*.svg |
Disallow | /core/ |
Disallow | /libraries/ |
Disallow | /modules/ |
Disallow | /profiles/ |
Disallow | /themes/ |
Disallow | /README.txt |
Disallow | /web.config |
Disallow | |
Disallow | /*.doc |
Disallow | /*.docx |
Disallow | /*.xls |
Disallow | /*.xlsx |
Disallow | /*.xlsm |
Disallow | /*.ppt |
Disallow | /*.pptx |
Disallow | /*.odt |
Disallow | /*.ods |
Disallow | /*.odp |
Disallow | /*.dot |
Disallow | /*.zip |
Disallow | /autoload.php |
Disallow | /robots_nidirect.txt |
Disallow | /update.php |
Disallow | /admin/ |
Disallow | /comment/reply/ |
Disallow | /filter/tips |
Disallow | /node/add/ |
Disallow | /search/ |
Disallow | /user/register/ |
Disallow | /user/password/ |
Disallow | /user/login/ |
Disallow | /user/logout/ |
Disallow | /index.php/admin/ |
Disallow | /index.php/comment/reply/ |
Disallow | /index.php/filter/tips |
Disallow | /index.php/node/add/ |
Disallow | /index.php/search/ |
Disallow | /index.php/user/password |
Disallow | /index.php/user/register |
Disallow | /index.php/user/login |
Disallow | /index.php/user/logout |
Disallow | /contacts/*letter |
Disallow | /publications/*date |
Disallow | /publications/*topic |
Disallow | /services/driving-instructors/*category |
Disallow | /services/health-conditions-a-z/*body-location |
Disallow | /services/health-conditions-a-z/*condition-type |
Disallow | /services/health-conditions-a-z/*letter |
Disallow | /search |
Other Records
Field | Value |
---|---|
crawl-delay | 10 |
bingbot
Rule | Path |
---|---|
Allow | /core/*.css$ |
Allow | /core/*.css? |
Allow | /core/*.js$ |
Allow | /core/*.js? |
Allow | /core/*.gif |
Allow | /core/*.jpg |
Allow | /core/*.jpeg |
Allow | /core/*.png |
Allow | /core/*.svg |
Allow | /libraries/*.css$ |
Allow | /libraries/*.css? |
Allow | /libraries/*.js$ |
Allow | /libraries/*.js? |
Allow | /libraries/*.gif |
Allow | /libraries/*.jpg |
Allow | /libraries/*.jpeg |
Allow | /libraries/*.png |
Allow | /modules/*.css$ |
Allow | /modules/*.css? |
Allow | /modules/*.js$ |
Allow | /modules/*.js? |
Allow | /modules/*.gif |
Allow | /modules/*.jpg |
Allow | /modules/*.jpeg |
Allow | /modules/*.png |
Allow | /profiles/*.css$ |
Allow | /profiles/*.css? |
Allow | /profiles/*.js$ |
Allow | /profiles/*.js? |
Allow | /profiles/*.gif |
Allow | /profiles/*.jpg |
Allow | /profiles/*.jpeg |
Allow | /profiles/*.png |
Allow | /themes/*.css$ |
Allow | /themes/*.css? |
Allow | /themes/*.js$ |
Allow | /themes/*.js? |
Allow | /themes/*.gif |
Allow | /themes/*.jpg |
Allow | /themes/*.jpeg |
Allow | /themes/*.png |
Disallow | /core/ |
Disallow | /libraries/ |
Disallow | /modules/ |
Disallow | /profiles/ |
Disallow | /themes/ |
Disallow | |
Disallow | /*.doc |
Disallow | /*.docx |
Disallow | /*.xls |
Disallow | /*.xlsx |
Disallow | /*.xlsm |
Disallow | /*.ppt |
Disallow | /*.pptx |
Disallow | /*.odt |
Disallow | /*.ods |
Disallow | /*.odp |
Disallow | /*.dot |
Disallow | /*.zip |
Disallow | /autoload.php |
Disallow | /robots_nidirect.txt |
Disallow | /update.php |
Disallow | /admin/ |
Disallow | /comment/reply/ |
Disallow | /filter/tips/ |
Disallow | /node/add/ |
Disallow | /search/ |
Disallow | /user/register |
Disallow | /user/password |
Disallow | /user/login |
Disallow | /user/logout |
Disallow | /?q=admin%2F |
Disallow | /?q=comment%2Freply%2F |
Disallow | /?q=filter%2Ftips%2F |
Disallow | /?q=node%2Fadd%2F |
Disallow | /?q=search%2F |
Disallow | /?q=user%2Fpassword%2F |
Disallow | /?q=user%2Fregister%2F |
Disallow | /?q=user%2Flogin%2F |
Disallow | /?q=user%2Flogout%2F |
Disallow | /contacts/*letter |
Disallow | /publications/*date |
Disallow | /publications/*topic |
Disallow | /services/driving-instructors/*category |
Disallow | /services/health-conditions-a-z/*body-location |
Disallow | /services/health-conditions-a-z/*condition-type |
Disallow | /services/health-conditions-a-z/*letter |
Disallow | /search |
Other Records
Field | Value |
---|---|
crawl-delay | 6 |
msnbot
Rule | Path |
---|---|
Allow | /core/*.css$ |
Allow | /core/*.css? |
Allow | /core/*.js$ |
Allow | /core/*.js? |
Allow | /core/*.gif |
Allow | /core/*.jpg |
Allow | /core/*.jpeg |
Allow | /core/*.png |
Allow | /core/*.svg |
Allow | /libraries/*.css$ |
Allow | /libraries/*.css? |
Allow | /libraries/*.js$ |
Allow | /libraries/*.js? |
Allow | /libraries/*.gif |
Allow | /libraries/*.jpg |
Allow | /libraries/*.jpeg |
Allow | /libraries/*.png |
Allow | /modules/*.css$ |
Allow | /modules/*.css? |
Allow | /modules/*.js$ |
Allow | /modules/*.js? |
Allow | /modules/*.gif |
Allow | /modules/*.jpg |
Allow | /modules/*.jpeg |
Allow | /modules/*.png |
Allow | /profiles/*.css$ |
Allow | /profiles/*.css? |
Allow | /profiles/*.js$ |
Allow | /profiles/*.js? |
Allow | /profiles/*.gif |
Allow | /profiles/*.jpg |
Allow | /profiles/*.jpeg |
Allow | /profiles/*.png |
Allow | /themes/*.css$ |
Allow | /themes/*.css? |
Allow | /themes/*.js$ |
Allow | /themes/*.js? |
Allow | /themes/*.gif |
Allow | /themes/*.jpg |
Allow | /themes/*.jpeg |
Allow | /themes/*.png |
Disallow | /core/ |
Disallow | /libraries/ |
Disallow | /modules/ |
Disallow | /profiles/ |
Disallow | /themes/ |
Disallow | |
Disallow | /*.doc |
Disallow | /*.docx |
Disallow | /*.xls |
Disallow | /*.xlsx |
Disallow | /*.xlsm |
Disallow | /*.ppt |
Disallow | /*.pptx |
Disallow | /*.odt |
Disallow | /*.ods |
Disallow | /*.odp |
Disallow | /*.dot |
Disallow | /*.zip |
Disallow | /autoload.php |
Disallow | /robots_nidirect.txt |
Disallow | /update.php |
Disallow | /admin/ |
Disallow | /comment/reply/ |
Disallow | /filter/tips/ |
Disallow | /node/add/ |
Disallow | /search/ |
Disallow | /user/register/ |
Disallow | /user/password/ |
Disallow | /user/login/ |
Disallow | /user/logout/ |
Disallow | /?q=admin%2F |
Disallow | /?q=comment%2Freply%2F |
Disallow | /?q=filter%2Ftips%2F |
Disallow | /?q=node%2Fadd%2F |
Disallow | /?q=search%2F |
Disallow | /?q=user%2Fpassword%2F |
Disallow | /?q=user%2Fregister%2F |
Disallow | /?q=user%2Flogin%2F |
Disallow | /?q=user%2Flogout%2F |
Disallow | /contacts/*letter |
Disallow | /publications/*date |
Disallow | /publications/*topic |
Disallow | /services/driving-instructors/*category |
Disallow | /services/health-conditions-a-z/*body-location |
Disallow | /services/health-conditions-a-z/*condition-type |
Disallow | /services/health-conditions-a-z/*letter |
Disallow | /search |
Other Records
Field | Value |
---|---|
crawl-delay | 6 |
Comments