Log In Sign Up

belocalnaughty.com
robots.txt

Robots Exclusion Standard data for belocalnaughty.com

Resource Scan

Scan Details

Site Domain belocalnaughty.com
Base Domain belocalnaughty.com
Scan Status Ok
Last Scan2025-04-06T02:21:22+00:00
Next Scan 2025-05-06T02:21:22+00:00

Last Scan

Scanned2025-04-06T02:21:22+00:00
URL https://belocalnaughty.com/robots.txt
Domain IPs 104.21.42.228, 172.67.167.169, 2606:4700:3031::6815:2ae4, 2606:4700:3035::ac43:a7a9
Response IP 104.21.42.228
Found Yes
Hash e95aae5da686c5caabddc0823af922663caddea7d87bf458e741150003bdcdae
SimHash c13c8862c2f3

Groups

*

Rule Path
Disallow /*utm_

mj12bot

Rule Path
Disallow /

texttrust crawler

Rule Path
Disallow /

black hole

Rule Path
Disallow /

blowfish/1.0

Rule Path
Disallow /

cherrypicker

Rule Path
Disallow /

erocrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

lnspiderguy

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

mister pix

Rule Path
Disallow /

moget

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5
Back to top

Comments

  • block bots
Back to top

Warnings

  • `host` is not a known field.
Back to top