arw.ro
robots.txt
Robots Exclusion Standard data for arw.ro
Resource Scan
Scan Details
| Site Domain | arw.ro |
| Base Domain | arw.ro |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-10-15T01:38:40+00:00 |
| Next Scan | 2025-11-14T01:38:40+00:00 |
Last Successful Scan
| Scanned | 2025-09-08T22:43:37+00:00 |
| URL | https://arw.ro/robots.txt |
| Redirect | https://www.arw.ro/robots.txt |
| Redirect Domain | www.arw.ro |
| Redirect Base | arw.ro |
| Domain IPs | 104.21.57.57, 172.67.159.145, 2606:4700:3032::ac43:9f91, 2606:4700:3037::6815:3939 |
| Redirect IPs | 104.21.57.57, 172.67.159.145, 2606:4700:3032::ac43:9f91, 2606:4700:3037::6815:3939 |
| Response IP | 104.21.57.57 |
| Found | Yes |
| Hash | 29253f9af630a755a7057a6c6ec3c9c9dc510f8194960b118447eec9ccab80d0 |
| SimHash | 7b87ec60cdd1 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /images/b$ |
| Disallow | /images/b/* |
| Disallow | /*?print_version* |
| Disallow | /*/pagina*/filtre/* |
| Disallow | /wishlist$ |
| Disallow | /wishlist/* |
| Disallow | /ajax$ |
| Disallow | /ajax/* |
| Disallow | /anunta-un-prieten$ |
| Disallow | /anunta-un-prieten/* |
| Disallow | /asistenta-clienti$ |
| Disallow | /asistenta-clienti/* |
| Disallow | /rating$ |
| Disallow | /rating/* |
| Disallow | /click$ |
| Disallow | /click/* |
| Disallow | /galerie-imagini$ |
| Disallow | /galerie-imagini/* |
| Disallow | /incarca-foto$ |
| Disallow | /incarca-foto/* |
| Disallow | /comentarii$ |
| Disallow | /comentarii/* |
| Disallow | /data/feeds* |
| Disallow | /compara$ |
| Disallow | /compara/* |
| Disallow | /shopping$ |
| Disallow | /shopping/* |
| Disallow | /payment$ |
| Disallow | /payment/* |
| Disallow | /search$ |
| Disallow | /search/* |
| Disallow | /logout$ |
| Disallow | /logout/* |
| Disallow | /opinie-produs$ |
| Disallow | /opinie-produs/* |
| Disallow | /package-tracking$ |
| Disallow | /package-tracking/* |
| Disallow | /view-messages$ |
| Disallow | /view-messages/* |
| Disallow | /fotografii-clienti$ |
| Disallow | /fotografii-clienti/* |
| Disallow | /notifications$ |
| Disallow | /notifications/* |
| Disallow | /*htaccess* |
| Disallow | /admin$ |
| Disallow | /admin/* |
| Disallow | /*/remote$ |
| Disallow | /*/remote/* |
| Disallow | /runtime$ |
| Disallow | /runtime/* |
| Disallow | /*/include$ |
| Disallow | /*/include/* |
| Disallow | /*/utils$ |
| Disallow | /*/utils/* |
| Disallow | /monitor$ |
| Disallow | /monitor/* |
| Disallow | /*/parsers$ |
| Disallow | /*/parsers/* |
| Disallow | /*/spiders$ |
| Disallow | /*/spiders/* |
| Disallow | /*.php |
| Disallow | /*.tpl |
| Disallow | /*.csv |
| Disallow | /*.parser |
| Disallow | /*.svn* |
| Disallow | /*.svn* |
| Disallow | /*?hide_sensitive |
| Disallow | /*?show_sensitive |
| Disallow | /*2pau |
| Disallow | /*2ptt |
| Disallow | /*2ptu |
| Disallow | /*2prp |
| Disallow | /*2pdlst |
| Disallow | /*%2C* |
| Disallow | /*/filtre/*/*/* |
| Disallow | /*/filtre/pret-%d-pana-la-%d/* |
| Disallow | /*/*%2C*/* |
| Disallow | /*tag-*/tag-* |
| Disallow | /produse-in-stoc/*/ |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.arw.ro/sitemap.xml |
Warnings
- 1 invalid line.