gbr.databasesets.com
robots.txt
Robots Exclusion Standard data for gbr.databasesets.com
Resource Scan
Scan Details
| Site Domain | gbr.databasesets.com |
| Base Domain | databasesets.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-06-07T09:53:52+00:00 |
| Next Scan | 2025-08-06T09:53:52+00:00 |
Last Successful Scan
| Scanned | 2025-03-17T09:52:08+00:00 |
| URL | https://gbr.databasesets.com/robots.txt |
| Domain IPs | 104.26.10.79, 104.26.11.79, 172.67.75.116, 2606:4700:20::681a:a4f, 2606:4700:20::681a:b4f, 2606:4700:20::ac43:4b74 |
| Response IP | 172.67.75.116 |
| Found | Yes |
| Hash | d8affb96c6d71baba829616837917766570c06f514b69d4ecf90f11b7ae8d861 |
| SimHash | 2c06bf08c50d |
Groups
*
| Rule | Path |
|---|---|
| Allow | /sites/default/files/ |
| Disallow | /cdn-cgi/ |
| Disallow | /*comment/reply/* |
| Disallow | /*image_captcha* |
| Disallow | /node/26 |
| Disallow | /node/52 |
| Disallow | /node/51 |
| Disallow | /node/27 |
| Disallow | /node/32 |
| Disallow | /node/28 |
| Disallow | /node/34 |
| Disallow | /node/29 |
| Disallow | /node/30 |
| Allow | /*misc/*.css$ |
| Allow | /*misc/*.css? |
| Allow | /*misc/*.js$ |
| Allow | /*misc/*.js? |
| Allow | /*misc/*.gif |
| Allow | /*misc/*.jpg |
| Allow | /*misc/*.jpeg |
| Allow | /*misc/*.png |
| Allow | /*modules/*.css$ |
| Allow | /*modules/*.css? |
| Allow | /*modules/*.js$ |
| Allow | /*modules/*.js? |
| Allow | /*modules/*.gif |
| Allow | /*modules/*.jpg |
| Allow | /*modules/*.jpeg |
| Allow | /*modules/*.png |
| Allow | /*profiles/*.css$ |
| Allow | /*profiles/*.css? |
| Allow | /*profiles/*.css? |
| Allow | /*profiles/*.js$ |
| Allow | /*profiles/*.js? |
| Allow | /*profiles/*.gif |
| Allow | /*profiles/*.jpg |
| Allow | /*profiles/*.jpeg |
| Allow | /*profiles/*.png |
| Allow | /*themes/*.css$ |
| Allow | /*themes/*.css? |
| Allow | /*themes/*.js$ |
| Allow | /*themes/*.js? |
| Allow | /*themes/*.gif |
| Allow | /*themes/*.jpg |
| Allow | /*themes/*.jpeg |
| Allow | /*themes/*.png |
| Disallow | /*includes/ |
| Disallow | /*misc/ |
| Disallow | /*modules/ |
| Disallow | /*profiles/ |
| Disallow | /*scripts/ |
| Disallow | /*themes/ |
| Disallow | /*CHANGELOG.txt |
| Disallow | /*cron.php |
| Disallow | /*INSTALL.mysql.txt |
| Disallow | /*INSTALL.pgsql.txt |
| Disallow | /*INSTALL.sqlite.txt |
| Disallow | /*install.php |
| Disallow | /*INSTALL.txt |
| Disallow | /*LICENSE.txt |
| Disallow | /*MAINTAINERS.txt |
| Disallow | /*update.php |
| Disallow | /*UPGRADE.txt |
| Disallow | /*xmlrpc.php |
| Disallow | /*admin/ |
| Disallow | /*comment/reply/ |
| Disallow | /*filter/tips/ |
| Disallow | /*node/add/ |
| Disallow | /*search/ |
| Disallow | /*user/register/ |
| Disallow | /*user/password/ |
| Disallow | /*user/login/ |
| Disallow | /*user/logout/ |
| Disallow | /*?q=admin%2F |
| Disallow | /*?q=comment%2Freply%2F |
| Disallow | /*?q=filter%2Ftips%2F |
| Disallow | /*?q=node%2Fadd%2F |
| Disallow | /*?q=search%2F |
| Disallow | /*?q=user%2Fpassword%2F |
| Disallow | /*?q=user%2Fregister%2F |
| Disallow | /*?q=user%2Flogin%2F |
| Disallow | /*?q=user%2Flogout%2F |
| Disallow | /*/user/login/ |
Other Records
| Field | Value |
|---|---|
| sitemap | https://gbr.databasesets.com/sitemap.xml |
| sitemap | https://gbr.databasesets.com/rss.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-england_urn.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-england_local-authority-name.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-england_phase-of-education.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-northern-ireland_reference.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-northern-ireland_town-name.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-northern-ireland_county-name.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-northern-ireland_institution-type.xml |
| sitemap | https://gbr.databasesets.com/sitemap_schools-in-northern-ireland_management-type.xml |
| sitemap | https://gbr.databasesets.com/sitemap_datasets-all_dataset-name.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_company-number.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_accounts-account-category.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_company-category.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_country-of-origin.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_reg-address-country.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_reg-address-county.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_reg-address-post-town.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_reg-address-post-code.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_sic-code-sic-text-1.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_sic-code-sic-text-2.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_sic-code-sic-text-3.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_sic-code-sic-text-4.xml |
| sitemap | https://gbr.databasesets.com/sitemap_basic-company-data_incorporation-date.xml |
| sitemap | https://gbr.databasesets.com/sitemap_mot-vehicle-testing-stations_number.xml |
| sitemap | https://gbr.databasesets.com/sitemap_mot-vehicle-testing-stations_city.xml |
| sitemap | https://gbr.databasesets.com/sitemap_mot-vehicle-testing-stations_post-code.xml |
| sitemap | https://gbr.databasesets.com/sitemap_list-of-industry-sponsors-in-the-uk_industry-sector.xml |
| sitemap | https://gbr.databasesets.com/sitemap_list-of-industry-sponsors-in-the-uk_organisation-name.xml |
Comments