shopmania.ro
robots.txt
Robots Exclusion Standard data for shopmania.ro
Resource Scan
Scan Details
| Site Domain | shopmania.ro |
| Base Domain | shopmania.ro |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a server error. |
| Last Scan | 2024-09-19T21:01:04+00:00 |
| Next Scan | 2024-12-18T21:01:04+00:00 |
Last Successful Scan
| Scanned | 2024-05-23T17:21:31+00:00 |
| URL | https://shopmania.ro/robots.txt |
| Domain IPs | 5.9.133.17 |
| Response IP | 5.9.133.17 |
| Found | Yes |
| Hash | 946698a7fb61d8a622fbcf18f1f64fed5aa66485203c099113b8fbc3f2cf93ec |
| SimHash | 2b1ed534e94a |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api/ |
| Disallow | /html/ |
| Disallow | /scripts/ |
| Disallow | /connect |
| Disallow | /forgot |
| Disallow | /login |
| Disallow | /profile_loader |
| Disallow | /redirect |
| Disallow | /remote_file |
| Disallow | /review |
| Disallow | /screenshot |
| Disallow | /signup |
| Disallow | /site_review |
| Disallow | /*?action=* |
| Disallow | /*?*&action=* |
| Disallow | /*?request=* |
| Disallow | /*?*&request=* |
| Disallow | /*?sort=* |
| Disallow | /*?*&sort=* |
| Disallow | /*?sorting=* |
| Disallow | /*?*&sorting=* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 5 |
ahrefsbot
amazonbot
archive.org_bot
baiduspider
barkrowler
blexbot
becomebot
ccbot
dotbot
etaospider
exabot
mj12bot
npbot
omniexplorer_bot
rogerbot
searchmetricsbot
seekportbot
semrushbot
semrushbot-ba
semrushbot-bm
semrushbot-ct
semrushbot-sa
semrushbot-seoab
semrushbot-si
semrushbot-swa
sentibot
seokicks
seokicks-robot
seznambot
shopwiki
sistrix
sistrix crawler
sistrix
smtbot
sogouspider
sogou web spider
stanford
stanford comp sci
synthesio
unisterbot
yandexbot
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 1 invalid line.
Comments