sengerson.com
robots.txt

Robots Exclusion Standard data for sengerson.com

Resource Scan

Scan Details

Site Domain sengerson.com
Base Domain sengerson.com
Scan Status Ok
Last Scan2025-11-06T05:57:51+00:00
Next Scan 2025-11-13T05:57:51+00:00

Last Scan

Scanned2025-11-06T05:57:51+00:00
URL https://sengerson.com/robots.txt
Redirect https://www.sengerson.com/robots.txt
Redirect Domain www.sengerson.com
Redirect Base sengerson.com
Domain IPs 104.21.14.20, 172.67.202.52, 2606:4700:3033::6815:e14, 2606:4700:3036::ac43:ca34
Redirect IPs 104.18.37.69, 172.64.150.187, 2606:4700:4408::ac40:96bb, 2a06:98c1:3100::6812:2545
Response IP 172.64.150.187
Found Yes
Hash 9e16a0b757b6f49481355e54c506d3e7642563b13d23e5a9522613f8d9ed6c4d
SimHash a46d79a0e455

Groups

*

Rule Path
Disallow /cdn-cgi/
Disallow /*add-to-cart%3D*

*

Rule Path
Disallow */tag/*
Disallow */?p=*
Disallow */?s=*
Disallow */?m=*
Disallow *?cat*
Disallow */?tag=*
Disallow */page/*
Disallow */feed/*
Disallow *?paged=*
Disallow *?page_id=*
Disallow *?replytocom*
Disallow *preview%3Dtrue*
Disallow *?attachment_id=*
Disallow *1pixelout_player.swf*
Disallow *wp-login.php?*
Disallow /author/
Disallow /wp-admin/
Disallow /our-home/wp-admin/
Disallow /our-family/wp-admin/
Disallow /motherhood/wp-admin/
Disallow /consulting/wp-admin/
Disallow /photography/wp-admin/
Disallow /entertaining/wp-admin/
Disallow /baby-love-shop/wp-admin/
Disallow /the-photographer/wp-admin/

Other Records

Field Value
sitemap https://www.sengerson.com/sitemap_index.xml

Comments

  • Prevent Crawling Unnecessary Endpoints - Dynamically added by BigScoots