Log In Sign Up

tw.shushenwu.cc
robots.txt

Robots Exclusion Standard data for tw.shushenwu.cc

Resource Scan

Scan Details

Site Domain tw.shushenwu.cc
Base Domain shushenwu.cc
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-10T22:09:57+00:00
Next Scan 2026-01-08T22:09:57+00:00

Last Successful Scan

Scanned2025-05-21T05:07:38+00:00
URL https://tw.shushenwu.cc/robots.txt
Domain IPs 104.21.37.46, 172.67.203.243, 2606:4700:3036::ac43:cbf3, 2606:4700:3037::6815:252e
Response IP 104.21.37.46
Found Yes
Hash b64d030dfb06739494637b42cd23abdfca32d01b63f17c780c3cd6f8744786be
SimHash c54b7963ce1d

Groups

*

Rule Path
Allow /
Disallow /*?*
Disallow /com/
Disallow /api/
Disallow /error_page/
Disallow /*.php$

perplexityai

Rule Path
Disallow /

adverification

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

dripfeedbot

Rule Path
Disallow /

adscanner

Rule Path
Disallow /

webwhacker

Rule Path
Disallow /

adsbot-google-mobile-apps

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

webzip

Rule Path
Disallow /

contentcrawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

cognitiveseo

Rule Path
Disallow /

deepcrawl

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

redditbot

Rule Path
Disallow /

adbeat

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

discordbot

Rule Path
Disallow /

webauto

Rule Path
Disallow /

bardbot

Rule Path
Disallow /

researchscan

Rule Path
Disallow /

deepmindbot

Rule Path
Disallow /

yahoo! slurp

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

websauger

Rule Path
Disallow /

siteexplorer

Rule Path
Disallow /

alexa crawler

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

webimagecollector

Rule Path
Disallow /

similarweb/1.0

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

adidxbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

rankflexbot

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

jaspercrawler

Rule Path
Disallow /

rankactivebot

Rule Path
Disallow /

admediabot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

webmeup

Rule Path
Disallow /

screamingfrog

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

webdataextractor

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

copyleaksbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

tumblr bot

Rule Path
Disallow /

spyfubot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

grokonbot

Rule Path
Disallow /

similarweb/2.0

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

contentking

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

netpeakchecker

Rule Path
Disallow /

larbin

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

mediapartners-google

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

coherecrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

similarwebbot

Rule Path
Disallow /

anthropicbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

adspider

Rule Path
Disallow /

octopus

Rule Path
Disallow /

originalityai

Rule Path
Disallow /

lexibot

Rule Path
Disallow /

stabilityaibot

Rule Path
Disallow /

writerlycrawler

Rule Path
Disallow /

webvac

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

webminer

Rule Path
Disallow /

dnyzbot

Rule Path
Disallow /

huggingfacebot

Rule Path
Disallow /

websucker

Rule Path
Disallow /

mozbot

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

webfetch

Rule Path
Disallow /

telegrambot

Rule Path
Disallow /

adixml

Rule Path
Disallow /

sitebulb

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

alexa toolbar

Rule Path
Disallow /

teleport

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

competebot

Rule Path
Disallow /

prowebwalker

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

slackbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

facebot

Rule Path
Disallow /

siteanalyzerbot

Rule Path
Disallow /

scaleai-crawler

Rule Path
Disallow /

weibobot

Rule Path
Disallow /

websphinx

Rule Path
Disallow /

alexa site audit

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

alexa web information service

Rule Path
Disallow /

exabot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://tw.shushenwu.cc/sitemap.xml
Back to top