sem1-brandtools.srver.xyz
robots.txt
Robots Exclusion Standard data for sem1-brandtools.srver.xyz
Resource Scan
Scan Details
| Site Domain | sem1-brandtools.srver.xyz |
| Base Domain | srver.xyz |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-12-01T11:48:17+00:00 |
| Next Scan | 2025-12-31T11:48:17+00:00 |
Last Successful Scan
| Scanned | 2025-10-08T10:47:32+00:00 |
| URL | https://sem1-brandtools.srver.xyz/robots.txt |
| Domain IPs | 2a02:4780:16:6197:bfa5:741:85c7:c8d3, 2a02:4780:38:2f85:b138:aba3:ec0c:def6, 84.32.84.112, 84.32.84.71 |
| Response IP | 179.61.189.231 |
| Found | Yes |
| Hash | f73039bed75d14484a1b3abad9a69224fbd7a95bbfde9ee2c670aecfc4b7e990 |
| SimHash | 7cb01612c4ad |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /blog/*?img=* |
| Disallow | /archive/graphs.php |
| Disallow | /forrester-marketing-silos/* |
| Disallow | /partner/ |
| Disallow | /blog/search/* |
| Disallow | /analytics/seomagic/ |
| Disallow | /my_reports/reports/* |
| Disallow | /archive/ |
| Disallow | /admin/ |
| Disallow | /users/ |
| Disallow | /clients/ |
| Disallow | /custom_report/ |
| Disallow | /limit_hits/ |
| Disallow | /payment/ |
| Disallow | /no_cookies/ |
| Disallow | /double_activation/ |
| Disallow | /projects/?* |
| Disallow | /?*%2F |
| Disallow | /activation_successful* |
| Disallow | /api.html?* |
| Disallow | /partners/ |
| Disallow | /partner/ |
| Disallow | */dvd-mode/ |
| Disallow | /unsubscribe/ |
| Disallow | /unsubscribe/* |
| Disallow | /demonstration/ |
| Disallow | /demonstration/* |
| Allow | /projects/ |
| Disallow | */custom-report/ |
| Disallow | */request-demo/ |
| Allow | */info/kdt/ |
| Disallow | */pleasereg/ |
| Disallow | */custom-report/ |
| Disallow | */custom-report |
| Allow | */sensor/* |
| Disallow | /swa/api/* |
| Disallow | /redirect?url=* |
| Disallow | /sso/ |
| Disallow | /_/report_blank/ |
| Disallow | /_/toolkit_blank/ |
| Disallow | /_/site_info_blank/ |
| Disallow | /seo-old/* |
| Disallow | /projects2/* |
| Disallow | /free-tools/plagiarism-checker/?q= |
| Allow | /news/*?page=* |
| Allow | /blog/* |
| Disallow | /blog/search/* |
| Disallow | /blog/*utm_source%3D* |
| Disallow | /blog-manager/* |
| Allow | /user/* |
| Disallow | /features/index2/ |
| Disallow | /features/for-* |
| Allow | /login/$ |
| Allow | /signup/$ |
| Disallow | /login/* |
| Disallow | /signup/* |
| Disallow | /solutions/$ |
| Disallow | /enterprise/$ |
| Allow | /enterprise/seo/ |
| Disallow | /landing/ |
| Disallow | /lp/ |
| Allow | /lp/affiliate-program/* |
| Allow | /lp/for-agencies/en/ |
| Allow | /lp/enterprise-aio/* |
| Allow | /lp/enterprise-crawler/* |
| Allow | /lp/enterprise-partner-program/* |
| Allow | /lp/seasonal-ai/en/ |
| Allow | /lp/black-friday/* |
| Disallow | /lp/black-friday-aff/* |
| Allow | /lp/cyber-week/* |
| Allow | /lp/insights/* |
| Allow | /lp/growth-quadrant/ |
| Allow | /lp/mfour/* |
| Allow | /lp/semrush-circle/* |
| Allow | /seo/ |
| Allow | /lp/seo-consultation/* |
| Allow | /content-marketing/ |
| Allow | /ppc/ |
| Allow | /competitive-research/ |
| Allow | /analytics/traffic/overview/ |
| Disallow | /listings-management/superpower-landing2/ |
| Allow | /academy/* |
| Disallow | /academy/*utm_source%3D* |
| Disallow | /academy/*/results |
| Disallow | /academy/certificates/* |
| Allow | /website/* |
| Disallow | /website/search/* |
| Disallow | /*/?* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://sem1-brandtools.srver.xyz/sitemap.xml |
| sitemap | https://pt.semrush.com/blog/sitemap/ |
| sitemap | https://es.semrush.com/blog/sitemap/ |
| sitemap | https://it.semrush.com/blog/sitemap/ |
| sitemap | https://fr.semrush.com/blog/sitemap/ |
| sitemap | https://de.semrush.com/blog/sitemap/ |
| sitemap | https://ja.semrush.com/blog/sitemap/ |
| sitemap | https://sem1-brandtools.srver.xyz/academy/sitemap.xml |
| sitemap | https://es.semrush.com/academy/sitemap.xml |
| sitemap | https://pt.semrush.com/academy/sitemap.xml |
| sitemap | https://it.semrush.com/academy/sitemap.xml |
| sitemap | https://fr.semrush.com/academy/sitemap.xml |
| sitemap | https://de.semrush.com/academy/sitemap.xml |
| sitemap | https://sem1-brandtools.srver.xyz/free-tools/sitemap.xml |
Warnings
- 2 invalid lines.
Comments