lawnewz.com
robots.txt

Robots Exclusion Standard data for lawnewz.com

Resource Scan

Scan Details

Site Domain lawnewz.com
Base Domain lawnewz.com
Scan Status Ok
Last Scan2024-05-10T22:01:54+00:00
Next Scan 2024-05-17T22:01:54+00:00

Last Scan

Scanned2024-05-10T22:01:54+00:00
URL http://lawnewz.com/robots.txt
Redirect https://lawandcrime.com/robots.txt
Redirect Domain lawandcrime.com
Redirect Base lawandcrime.com
Domain IPs 198.20.95.19
Redirect IPs 104.22.20.95, 104.22.21.95, 172.67.10.184, 2606:4700:10::6816:145f, 2606:4700:10::6816:155f, 2606:4700:10::ac43:ab8
Response IP 104.22.20.95
Found Yes
Hash 90b46e9988f17a8941646d12462e8c3cbe230a89a4b892b5669bf3b739e6611a
SimHash 4805406281b2

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

claritybot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://lawandcrime.com/sitemap.xml
sitemap https://lawandcrime.com/sitemap.rss