firmitas.org
robots.txt
Robots Exclusion Standard data for firmitas.org
Resource Scan
Scan Details
| Site Domain | firmitas.org |
| Base Domain | firmitas.org |
| Scan Status | Ok |
| Last Scan | 2024-10-02T21:19:06+00:00 |
| Next Scan | 2024-10-09T21:19:06+00:00 |
Last Scan
| Scanned | 2024-10-02T21:19:06+00:00 |
| URL | https://firmitas.org/robots.txt |
| Domain IPs | 155.133.138.8, 2001:4b98:dc5:253::8 |
| Response IP | 155.133.138.8 |
| Found | Yes |
| Hash | 107c7171460da8ed3ae3e1c5cf01879efb0c64cd54a200f8c91df3a96eb932fb |
| SimHash | 99315531c6e0 |
Groups
test-bot
adsbot
alphaseobot
anitya
applebot
applebot
linguee
charityengine
aspiegel
baidu
blexbot
bubing
barkrowler
elisabot
facebookexternalhit
feedfetcher-google
garlikcrawler
grapeshot
iabot
ioncrawl
wikimedia
jooblebot
krowler
linespider
mj12bot
mail.ru_bot
nextgensearchbot
owler
pandalytics
petalbot
qwantify
smtbot
scrapy
semrushbot
seznam
seznambot
surdotlybot
turnitinbot
tweetmemebot
vegi bot
webster pro
worldbrewbot
wotbox
yeti
admantx
ia_archiver
ltx71
maxpoint.crawler
panscient.com
spbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | ios.php |
| Disallow | /imgs/ |
| Disallow | /cgi-bin/ |
| Disallow | /members/ |
*
No rules defined. All paths allowed.
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Warnings
- 1 invalid line.