Log In Sign Up

leromlaw.com
robots.txt

Robots Exclusion Standard data for leromlaw.com

Resource Scan

Scan Details

Site Domain leromlaw.com
Base Domain leromlaw.com
Scan Status Ok
Last Scan2025-03-23T09:50:53+00:00
Next Scan 2025-04-22T09:50:53+00:00

Last Scan

Scanned2025-03-23T09:50:53+00:00
URL https://leromlaw.com/robots.txt
Redirect https://www.leromlaw.com/robots.txt
Redirect Domain www.leromlaw.com
Redirect Base leromlaw.com
Domain IPs 75.2.70.75, 99.83.190.102
Redirect IPs 52.68.134.190, 54.238.67.66, 57.180.160.60
Response IP 57.180.160.60
Found Yes
Hash b4661d45dd8bbd18359e30f08f6520b9c797cc7ff044461cd95cd644fb90bc6c
SimHash 5c191940a3f2

Groups

*

Rule Path
Disallow
Disallow /cgi-bin/

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

image2dataset

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

dalvik/2.1.0

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

googleother

Rule Path
Disallow /

ioncrawl

Rule Path
Disallow /

isscyberriskcrawler

Rule Path
Disallow /

java

Rule Path
Disallow /

mozlila

Rule Path
Disallow /

orbbot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

bw/1.1

Rule Path
Disallow /

wp_is_mobile

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

censysinspect

Rule Path
Disallow /

expanse

Rule Path
Disallow /

internet-measurement

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

baidu

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

seznam

Rule Path
Disallow /

sogou

Rule Path
Disallow /

cốc cốc

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

medialogia bot

Rule Path
Disallow /

honeybadgerbot

Rule Path
Disallow /

prerender

Rule Path
Disallow /

marketgoo

Rule Path
Disallow /

botify

Rule Path
Disallow /

siteimprove crawl

Rule Path
Disallow /

addthis

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 6.0; windows nt 5.1)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 7.0; windows nt 6.0)

Rule Path
Disallow /

mozilla/4.0 (compatible; msie 8.0; windows nt 6.0; trident/4.0)

Rule Path
Disallow /

mozilla/5.0 (windows nt 5.1; rv:2.0) gecko/20100101 firefox/4.0

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.leromlaw.com/sitemap.xml
Back to top

Comments

  • Blocking specific bad bots
  • Blocking SEO Crawlers
  • Blocking Security Scanners
  • Blocking Search Engine Crawlers
  • Blocking Other Bots and Scrapers
  • Blocking Old Browser Versions
Back to top