Log In Sign Up

digisoup.net
robots.txt

Robots Exclusion Standard data for digisoup.net

Resource Scan

Scan Details

Site Domain digisoup.net
Base Domain digisoup.net
Scan Status Ok
Last Scan2025-08-03T01:50:21+00:00
Next Scan 2025-09-02T01:50:21+00:00

Last Scan

Scanned2025-08-03T01:50:21+00:00
URL https://digisoup.net/robots.txt
Domain IPs 38.207.3.32
Response IP 38.207.3.32
Found Yes
Hash 1aeae96e368545a300633129b7ecca049d6d8caaa39883e72b64677c924bed6e
SimHash 5132d5c8f4b2

Groups

baiduspider

Rule Path
Allow /

sogou spider

Rule Path
Allow /

sosospider

Rule Path
Allow /

googlebot

Rule Path
Allow /

mediapartners-google

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

etaospider

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

zmeu phpmyadmin

Rule Path
Disallow /

winhttp

Rule Path
Disallow /

httpclient

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

yyspider

Rule Path
Disallow /

jaunty

Rule Path
Disallow /

obot

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

indy library

Rule Path
Disallow /

flightdeckreports bot

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

swiftbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

youdaobot

Rule Path
Disallow /

yodaobot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ezooms.bot

Rule Path
Disallow /

speedy spider

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

crawldaddy

Rule Path
Disallow /

asktbfxtv

Rule Path
Disallow /

alexa toolbar

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

coolpadwebkit

Rule Path
Disallow /

java

Rule Path
Disallow /

feedly

Rule Path
Disallow /

universalfeedparser

Rule Path
Disallow /

apachebench

Rule Path
Disallow /

microsoft url control

Rule Path
Disallow /

zmeu

Rule Path
Disallow /

lightdeckreports bot

Rule Path
Disallow /

digext

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

headlesschrome

Rule Path
Disallow /

feeddemon

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

opensiteexplorer

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

hubspot

Rule Path
Disallow /

opensiteexplorer

Rule Path
Disallow /

python

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /
Back to top

Warnings

  • 2 invalid lines.
Back to top