dataevent.com
robots.txt

Robots Exclusion Standard data for dataevent.com

Resource Scan

Scan Details

Site Domain dataevent.com
Base Domain dataevent.com
Scan Status Ok
Last Scan2024-10-05T08:05:12+00:00
Next Scan 2024-10-12T08:05:12+00:00

Last Scan

Scanned2024-10-05T08:05:12+00:00
URL https://dataevent.com/robots.txt
Domain IPs 2001:41d0:1:1b00:213:186:33:2, 46.105.204.2
Response IP 46.105.204.2
Found Yes
Hash 206669ea76e15b154954cc11be07ef5f6ca2e31c8ed77d8f8a70672eb38619fd
SimHash 2b1e76715c2b

Groups

*

Rule Path
Disallow /cgi-bin/
Disallow /tmp/
Disallow /junk/

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

seokicks-robot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

dataforseo_bot

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

baeaglebot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

careerbot

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

coccoc

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

jetbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

facebot

Rule Path
Disallow /

*

Rule Path
Disallow /publi*/inscription.php?ids=*
Disallow /publi*/inscription.php
Disallow /publi*/inscription.php-*
Disallow /publi*/password.php?ids=*
Disallow /publi*/password.php
Disallow /publi*/password.php-*
Disallow /index.html

Comments

  • Bots spécifiques à bloquer
  • Bots de scraping génériques
  • Autres bots couramment bloqués
  • Bloquer les bots Facebook (si nécessaire)
  • Restrictions spécifiques