adamkatzsinding.com
robots.txt

Robots Exclusion Standard data for adamkatzsinding.com

Resource Scan

Scan Details

Site Domain adamkatzsinding.com
Base Domain adamkatzsinding.com
Scan Status Ok
Last Scan2025-12-07T19:34:08+00:00
Next Scan 2026-01-06T19:34:08+00:00

Last Scan

Scanned2025-12-07T19:34:08+00:00
URL https://adamkatzsinding.com/robots.txt
Domain IPs 104.21.70.3, 172.67.217.41, 2606:4700:3031::6815:4603, 2606:4700:3037::ac43:d929
Response IP 172.67.217.41
Found Yes
Hash f88b5a8d689fca80e17da7323a778d2072016fc769231b6ecaf8c963c72e05b3
SimHash 623d50c1722c

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/

Other Records

Field Value
crawl-delay 120

yandexbot

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 1200

yandeximages

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

slurp

Rule Path
Disallow /

qwantify

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 60000

msnbot-media

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 60000

bingbot

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 60000

pinterest

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 3600

pinterestbot

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 3600

pinterestbot/1.0

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 3600

googlebot

Rule Path
Disallow /wp-admin/

Other Records

Field Value
crawl-delay 60

googlebot-image

Rule Path
Disallow /

googleimageproxy

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /