simpef-nazionale.it
robots.txt
Robots Exclusion Standard data for simpef-nazionale.it
Resource Scan
Scan Details
| Site Domain | simpef-nazionale.it |
| Base Domain | simpef-nazionale.it |
| Scan Status | Ok |
| Last Scan | 2024-09-15T09:08:42+00:00 |
| Next Scan | 2024-10-15T09:08:42+00:00 |
Last Scan
| Scanned | 2024-09-15T09:08:42+00:00 |
| URL | https://simpef-nazionale.it/robots.txt |
| Domain IPs | 195.35.24.150 |
| Response IP | 195.35.24.150 |
| Found | Yes |
| Hash | d0137296d2d2c8c05c211ff9b6d60f242cd1a36c7e76e698901bb1232dc83d5a |
| SimHash | b61e5a7284b3 |
Groups
baiduspider
yandex
uptimebot
dataprovider.com
mj12bot
ahrefsbot
ccbot
petalbot
wellknownbot
blexbot
seznambot
sogou spider
seokicks-robot
seokicks
discobot
blekkobot
blexbot
sistrix crawler
ezooms robot
netestate ne crawler
wiseguys robot
turnitin robot
babya discoverer
exabot
zealbot
sitesnagger
webstripper
webcopier
fetch
offline explorer
teleport
teleportpro
acunetix
webzip
linko
httrack
larbin
libwww
zyborg
download ninja
k2spider
webreaper
woorank
checkmarknetwork/1.0 (+http://www.checkmarknetwork.com/spider.html)
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /*login |
| Disallow | /*/*login |
| Disallow | /*/accesso_negato |
| Disallow | /*/image_captcha |
| Disallow | /*.py |
| Disallow | /*.ini |
| Disallow | /sito.cfg |
| Disallow | /cgi-bin/* |
| Disallow | /old_contatti |
| Disallow | /asset* |
| Disallow | /blog/* |
| Disallow | /content* |
| Disallow | /event* |
| Disallow | /file* |
| Disallow | /glossary* |
| Disallow | /link* |
| Disallow | /forum* |
| Disallow | /misc* |
| Disallow | /old* |
| Disallow | /sites* |
| Disallow | /system* |
| Disallow | /taxonomy* |
| Disallow | /tmp/* |
| Disallow | /user/* |
| Disallow | /wp-* |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 10 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://simpef-nazionale.it/sitemap_index.xml |
Warnings
- `host` is not a known field.
- `request-rate` is not a known field.
Comments