algeciras.artelista.com
robots.txt
Robots Exclusion Standard data for algeciras.artelista.com
Resource Scan
Scan Details
| Site Domain | algeciras.artelista.com |
| Base Domain | artelista.com |
| Scan Status | Ok |
| Last Scan | 2025-05-20T09:08:26+00:00 |
| Next Scan | 2025-06-19T09:08:26+00:00 |
Last Scan
| Scanned | 2025-05-20T09:08:26+00:00 |
| URL | https://algeciras.artelista.com/robots.txt |
| Domain IPs | 2a05:d018:769:8300:6df7:6b8e:6ae1:e640, 2a05:d018:769:8303:4c63:f0d3:fdb4:2418, 34.248.167.53, 54.155.216.67 |
| Response IP | 34.248.167.53 |
| Found | Yes |
| Hash | 7e7b0b7d3918cf00e00d0c082f07f8bbea1bdef9b32dbc04c34ba370dfd317c0 |
| SimHash | 484cc8d227bb |
Groups
shopwiki
accelobot
sapphirewebcrawler
dotbot
charlotte
kalooga
httrack
linguee
ltx71
megaindex
blexbot
sogou spider
proximic
domaincrawler
mj12bot
| Rule | Path |
|---|---|
| Disallow | / |
bingbot
yandex
duckduckbot
claudebot
ahrefsbot
semrushbot
petalbot
criteobot
grapeshot
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 1 |
*
| Rule | Path |
|---|---|
| Allow | / |
| Disallow | */menu/* |
| Disallow | */tags* |
| Disallow | */search_results.php* |
| Disallow | */autor-seguido-por/* |
| Disallow | */favorite/* |
| Disallow | */autor-seguidores/* |
| Disallow | */followers/* |
| Disallow | */obra-favorita/* |
| Disallow | */artwork-followers/* |
| Disallow | */coleccion/ |
| Disallow | */colecciones/ |
| Disallow | */landings/ |
| Disallow | /dynamic/* |
Comments