Log In Sign Up

marysminiatures.net
robots.txt

Robots Exclusion Standard data for marysminiatures.net

Resource Scan

Scan Details

Site Domain marysminiatures.net
Base Domain marysminiatures.net
Scan Status Ok
Last Scan2025-04-08T01:59:14+00:00
Next Scan 2025-04-22T01:59:14+00:00

Last Scan

Scanned2025-04-08T01:59:14+00:00
URL https://marysminiatures.net/robots.txt
Redirect https://www.marysminiatures.net/robots.txt
Redirect Domain www.marysminiatures.net
Redirect Base marysminiatures.net
Domain IPs 67.43.13.211
Redirect IPs 67.43.13.211
Response IP 67.43.13.211
Found Yes
Hash d83d95be603913ca4d2b2437c3a94adf94e2e701bbae4958b4e19d3717da19f2
SimHash 3d5051c6e840

Groups

googlebot

Rule Path
Disallow /cgi_bin/
Disallow *back_in_stock

slurp

Rule Path
Disallow /cgi-bin/

sogou web spider/4.0

Rule Path
Disallow /

romanceonly.com

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

ltx71

Rule Path
Disallow /
Disallow /

wp_is_mobile

Rule Path
Disallow /

frost.torexit

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

daum

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

semrushbot-coub

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexdirect

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

xaldon_webspider

Rule Path
Disallow /

naver

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

photon/1.0

Rule Path
Disallow /

photon

Rule Path
Disallow /

dispatch/0.11.3

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

re-re studio

Rule Path
Disallow /

masscan/1.3

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Back to top

Other Records

Field Value
sitemap https://www.marysminiatures.net/sitemap.xml
Back to top

Comments

  • Robots.txt file for https://www.marysminiatures.net/
  • Updated 6/21/2021 PRO-Webs
Back to top

Warnings

  • 2 invalid lines.
  • `user agent` is not a known field.
Back to top