familywithkids.com
robots.txt

Robots Exclusion Standard data for familywithkids.com

Resource Scan

Scan Details

Site Domain familywithkids.com
Base Domain familywithkids.com
Scan Status Ok
Last Scan2024-06-17T15:34:47+00:00
Next Scan 2024-07-17T15:34:47+00:00

Last Scan

Scanned2024-06-17T15:34:47+00:00
URL https://familywithkids.com/robots.txt
Redirect https://www.familywithkids.com/robots.txt
Redirect Domain www.familywithkids.com
Redirect Base familywithkids.com
Domain IPs 104.26.4.40, 104.26.5.40, 172.67.72.134, 2606:4700:20::681a:428, 2606:4700:20::681a:528, 2606:4700:20::ac43:4886
Redirect IPs 104.26.4.40, 104.26.5.40, 172.67.72.134, 2606:4700:20::681a:428, 2606:4700:20::681a:528, 2606:4700:20::ac43:4886
Response IP 172.67.72.134
Found Yes
Hash e93f95eb6eaf9855888b36112e50371aafb5a427513d5e341417ec64f7bc84af
SimHash 412cdf601ed1

Groups

blexbot
scrapy
megaindex.ru
dotbot
bubing
mauibot
semrushbot
semrushbot-sa

Rule Path
Disallow /

*

Rule Path
Disallow */admin/
Disallow */catalog/filials
Disallow */catalog/districts/
Disallow */comment/reply/
Disallow */dev/
Disallow */events?date=*
Disallow */events/?date=*
Disallow */forms/
Disallow */image_captcha
Disallow /includes/
Disallow /js/ac/
Disallow */logout/
Disallow */mails/
Disallow */messages/
Disallow /misc/
Allow /modules/*.js
Allow /modules/*.css
Disallow */modules/
Disallow */node/
Allow */catalog/*?*page=*
Allow */camps/*?*page=*
Disallow *?*page=*
Disallow *?*filter=*
Disallow */person/login
Disallow */person/password
Disallow */person/register
Allow */profile/person/question/
Allow */profile/ownership/
Disallow */profile/
Disallow */profiles/
Disallow /search/
Disallow /search-results
Disallow /sendinfo
Disallow */tags/
Disallow */user/

Warnings

  • `host` is not a known field.