topnegozi.it
robots.txt
Robots Exclusion Standard data for topnegozi.it
Resource Scan
Scan Details
| Site Domain | topnegozi.it |
| Base Domain | topnegozi.it |
| Scan Status | Ok |
| Last Scan | 2024-11-02T10:51:02+00:00 |
| Next Scan | 2024-11-09T10:51:02+00:00 |
Last Scan
| Scanned | 2024-11-02T10:51:02+00:00 |
| URL | https://topnegozi.it/robots.txt |
| Redirect | https://www.topnegozi.it/robots.txt |
| Redirect Domain | www.topnegozi.it |
| Redirect Base | topnegozi.it |
| Domain IPs | 172.66.40.118, 172.66.43.138, 2606:4700:3108::ac42:2876, 2606:4700:3108::ac42:2b8a |
| Redirect IPs | 172.66.40.118, 172.66.43.138, 2606:4700:3108::ac42:2876, 2606:4700:3108::ac42:2b8a |
| Response IP | 172.66.40.118 |
| Found | Yes |
| Hash | be60d5da873ce2e84b7e1cc703650a8d402e7224dd63fb37d84777444d7be771 |
| SimHash | c3da5811cff0 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /redirect.php |
| Disallow | /redirect_offers_new.php |
| Disallow | /offer-out |
| Disallow | /blog/wp-admin/* |
| Disallow | /blog/wp-login.php |
| Disallow | /ajax/ |
| Disallow | /get-cdata* |
magpie-crawler
brandwatch
yandexbot
trendictionbot
sogou
sogou spider
seznambot
yahoo! slurp
slurp
coccocbot-web
coccocbot-image
hubspot
blexbot
netestate
netestate ne crawler
seokicks
seokicks-robot
ccbot
megaindex.ru/2.0
megaindex.ru
megaindex.ru
youdaobot
mj12bot
mj12bot/v1.4.3
uptimerobot/2.0
uptimerobot
ezooms robot
ezooms
wiseguys robot
turnitin robot
turnitinbot
turnitin bot
turnitinbot/3.0
baiduspider
baiduspider-video
baiduspider-image
baiduspider/2.0
baiduspider/3.0
baiduspider/4.0
baiduspider/5.0
doc
zao
twiceler
zealbot
msiecrawler
sitesnagger
webstripper
webcopier
httrack
fetch
offline explorer
teleport
teleportpro
webzip
linko
httrack
xenu
microsoft.url.control
microsoft.url
larbin
libwww
zyborg
download ninja
nutch
spock
omniexplorer_bot
becomebot
geniebot
mlbot
linguee bot
aihitbot
exabot
sbider/nutch
jyxobot
magent
speedy spider
shopwiki
huasai
datacha0s
atomic_email_hunter
mp3bot
betabot
core-project
panscient.com
libwww-perl
java
garlikcrawler/1.2
garlikcrawler
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.topnegozi.it/sitemap_index.xml |
Warnings
- 2 invalid lines.
Comments