athens.indymedia.org
robots.txt
Robots Exclusion Standard data for athens.indymedia.org
Resource Scan
Scan Details
| Site Domain | athens.indymedia.org |
| Base Domain | indymedia.org |
| Scan Status | Ok |
| Last Scan | 2024-11-11T07:57:46+00:00 |
| Next Scan | 2024-11-25T07:57:46+00:00 |
Last Scan
| Scanned | 2024-11-11T07:57:46+00:00 |
| URL | https://athens.indymedia.org/robots.txt |
| Domain IPs | 142.132.196.168, 172.104.232.45, 51.77.117.40 |
| Response IP | 51.77.117.40 |
| Found | Yes |
| Hash | 08fee19d5a5189b369b416483243bc63af9a4eb24b586ed5f037859d55147201 |
| SimHash | f35cd2a0ccf8 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /hidden |
| Disallow | /hidden/ |
| Disallow | /calendar_beta |
| Disallow | /calendar_beta/ |
| Disallow | */*up |
| Disallow | /a |
| Disallow | /search |
semrushbot
sogou
sogou spider
sogou web spider
mauibot
exabot
yandex*
blexbot
trendkite-akashic-crawler
ahrefsbot
magpie-crawler
adsbot/3.1
megaindex
velenpublicwebcrawler
seekport crawler
trendictionbot
buck
buck/2.2
petalbot
dotbot
barkrowler
bdcbot
dataforseobot
twingly recon
amazonbot
seokicks
bingbot
adidxbot
msnbot
msnbot-*
bingpreview
imagesiftbot
bytespider
semrushbot
omgili
gptbot
claudebot
seolizer
awariobot
mj12bot
| Rule | Path |
|---|---|
| Disallow | / |
Warnings
- 1 invalid line.