Log In Sign Up

archivium.biz
robots.txt

Robots Exclusion Standard data for archivium.biz

Resource Scan

Scan Details

Site Domain archivium.biz
Base Domain archivium.biz
Scan Status Ok
Last Scan2024-09-30T16:38:40+00:00
Next Scan 2024-10-07T16:38:40+00:00

Last Scan

Scanned2024-09-30T16:38:40+00:00
URL https://archivium.biz/robots.txt
Domain IPs 77.81.225.78
Response IP 77.81.225.78
Found Yes
Hash e07c3aa2055b231ca89a0716baa67e9b859286e31e75e15757c4624b0adc14de
SimHash 62907a59b7dc

Groups

*

Rule Path
Disallow /_ads/*
Disallow /_comuni/*
Disallow /_favicons/*
Disallow /_font/*
Disallow /_functions/*
Disallow /_google/*
Disallow /_grafica/*
Disallow /_immagini/*
Disallow /_javas/*
Disallow /_jsonld/*
Disallow /_mailer/*
Disallow /_pagina/*
Disallow /_utilita/*
Disallow /test/*
Disallow /*.php
Disallow /*.sql
Disallow /*.csv
Disallow /Z_Test.php
Disallow php_test.php
Disallow php_info.php

dataforseo-bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zeabot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

httrack

Rule Path
Disallow /

xenu

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

kspider

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

nimblecrawler

Rule Path
Disallow /

usyd-nlp-spider

Rule Path
Disallow /

shim-crawler

Rule Path
Disallow /

myengines-bot

Rule Path
Disallow /

kfsw-bot

Rule Path
Disallow /

sbider

Rule Path
Disallow /

localcombot

Rule Path
Disallow /

iccrawler

Rule Path
Disallow /

knowitall

Rule Path
Disallow /

dcbspider

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

cfetch

Rule Path
Disallow /

findlinks

Rule Path
Disallow /

gonzo
gonzo
gonzop
gonzop

Rule Path
Disallow /

moni

Rule Path
Disallow /

georgios

Rule Path
Disallow /

cydralspider

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

hoowwwer

Rule Path
Disallow /

jemmathetourist

Rule Path
Disallow /

btbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

irlbot

Rule Path
Disallow /

becomebot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

amfibibot

Rule Path
Disallow /

gridbot

Rule Path
Disallow /

sna

Rule Path
Disallow /

tamu_cs_irl_crawler

Rule Path
Disallow /

npt

Rule Path
Disallow /

bruinbot

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

molbsy

Rule Path
Disallow /

phpdig

Rule Path
Disallow /

goforit.com

Rule Path
Disallow /

goforit

Rule Path
Disallow /

larbin

Rule Path
Disallow /

appie

Rule Path
Disallow /

libwww

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

sohu-search

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

webzip

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

linko

Rule Path
Disallow /

rpt-httpclient

Rule Path
Disallow /

dumbot

Rule Path
Disallow /

cowbot

Rule Path
Disallow /

superget

Rule Path
Disallow /

psbot

Rule Path
Disallow /

szukacz

Rule Path
Disallow /

antibot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

speedy

Rule Path
Disallow /

npbot

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

searchspider

Rule Path
Disallow /

lachesis

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

haste

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nutch

Rule Path
Disallow /

nutchorg

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

steeler

Rule Path
Disallow /

ultraseek

Rule Path
Disallow /

spinne

Rule Path
Disallow /

spider_monkey

Rule Path
Disallow /

ixe crawler

Rule Path
Disallow /

coolbot

Rule Path
Disallow /

vse/.

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://archivium.biz/_sitemaps/sitemap_archivium.php
Back to top

Comments

  • MARKETING AND REALTIME BIDDING BOTS
  • DataForSeoBot/1.0; +https://dataforseo.com/dataforseo-bot
  • ORACLE https://www.oracle.com/corporate/acquisitions/grapeshot/crawler.html
  • https://ahrefs.com/robot
  • FA RICHIESTE A /calendar/view.php?view=month&time=1338501600&lang=it CHE NON CAPISCO
  • http://www.semrush.com/bot.html
  • https://www.trendiction.com/en/publisher/bot
  • http://www.proximic.com/info/spider.php
  • Some bots are known to be trouble, particularly those designed to copy
  • entire sites. Please obey robots.txt.
Back to top

Warnings

  • 2 invalid lines.
Back to top