group-ib.com
robots.txt
Robots Exclusion Standard data for group-ib.com
Resource Scan
Scan Details
| Site Domain | group-ib.com |
| Base Domain | group-ib.com |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2024-08-26T06:55:54+00:00 |
| Next Scan | 2024-09-25T06:55:54+00:00 |
Last Successful Scan
| Scanned | 2024-07-21T06:52:11+00:00 |
| URL | https://group-ib.com/robots.txt |
| Domain IPs | 3.72.181.255 |
| Response IP | 3.72.181.255 |
| Found | Yes |
| Hash | 3045ff653d471bb4fbc10c4112a635a6e7995ec98058b6d5660d3aa2a33aebf4 |
| SimHash | 6750b7726e18 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin |
| Disallow | /cgi-bin |
| Disallow | /tags |
| Disallow | /brochures |
| Disallow | /certificates |
| Disallow | /speakers |
| Disallow | /category |
| Disallow | /blog/tags |
| Allow | /*.js |
| Allow | /wp-admin/admin-ajax.php |
| Allow | /*.css |
| Disallow | /remote-login.php |
| Disallow | /? |
| Disallow | *?s= |
| Disallow | *%26s%3D |
| Disallow | /search |
| Disallow | /author/ |
| Disallow | */embed$ |
| Disallow | */xmlrpc.php |
| Disallow | *openstat%3D |
| Disallow | /media/wp-admin/*.js |
| Disallow | /media/wp-admin/*.css |
| Allow | /media/wp-admin/*.jpg |
| Disallow | /media/wp-admin/*ajax |
| Disallow | *?year= |
| Disallow | *?view=article&id= |
| Disallow | /?view=article&id= |
| Disallow | /index.php |
| Disallow | /?id= |
| Disallow | *?url= |
| Disallow | *?p= |
| Disallow | /Ochee7 |
| Disallow | /Ih1pei |
| Disallow | /confirmation.html |
| Disallow | /resources/podcasts/sandbox.html |
| Disallow | /poll/questionnaire.html |
| Disallow | /partners-survey-1.html |
| Disallow | /resources/webinars/test.html |
| Disallow | /webinar-nrg.html |
| Disallow | /webinar-telecom.html |
| Disallow | /media/wp-admin |
| Disallow | /media/wp-includes |
| Disallow | /media/wp-comments |
| Disallow | /media/wp-content/plugins |
| Disallow | /media/wp-content/themes |
| Disallow | /media/wp-login.php |
| Disallow | /media/wp-register.php |
| Disallow | */trackback |
| Disallow | */feed |
| Disallow | /cgi-bin |
| Disallow | *?s= |
| Disallow | /api/ |
| Disallow | /media-center/press-releases/tags |
| Disallow | /media-center/news/* |
| Disallow | /partners/* |
| Disallow | /careers/vacancies/tags/ |
| Disallow | /careers/vacancies/tags/* |
| Disallow | /careers/vacancies/location/ |
| Disallow | /careers/vacancies/location/* |
| Disallow | /careers/vacancies/department/ |
| Disallow | /careers/vacancies/department/* |
| Disallow | /careers/tags/ |
| Disallow | /careers/tags/* |
| Disallow | /careers/location/ |
| Disallow | /careers/location/* |
| Disallow | /careers/department/ |
| Disallow | /careers/department/* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.group-ib.com/sitemap.xml |
Warnings
- `host` is not a known field.