cslewis.com
robots.txt

Robots Exclusion Standard data for cslewis.com

Resource Scan

Scan Details

Site Domain cslewis.com
Base Domain cslewis.com
Scan Status Ok
Last Scan2025-10-23T19:06:54+00:00
Next Scan 2025-11-22T19:06:54+00:00

Last Scan

Scanned2025-10-23T19:06:54+00:00
URL https://cslewis.com/robots.txt
Domain IPs 104.21.47.63, 172.67.145.60, 2606:4700:3034::ac43:913c, 2606:4700:3037::6815:2f3f
Response IP 104.21.47.63
Found Yes
Hash 59c5f110337b2b08f81afa832b8568c0a9fbba721bd31edfaefd69a0a594295e
SimHash 6805d001c1b7

Groups

*

Rule Path
Disallow /

applebot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

bingbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

discordbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

duckduckbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

facebookexternalhit

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

googlebot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

googlebot-image

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

ia_archiver

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

linkedinbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

msnbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

pinterestbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

slurp

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

teoma

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

twitterbot

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

yandex

Rule Path
Disallow /wp-admin/
Disallow /search*
Allow /
Allow /wp-admin/admin-ajax.php

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.cslewis.com/sitemap_index.xml