hometalk.com
robots.txt

Robots Exclusion Standard data for hometalk.com

Resource Scan

Scan Details

Site Domain hometalk.com
Base Domain hometalk.com
Scan Status Ok
Last Scan2024-11-02T09:06:27+00:00
Next Scan 2024-11-09T09:06:27+00:00

Last Scan

Scanned2024-11-02T09:06:27+00:00
URL https://hometalk.com/robots.txt
Redirect https://www.hometalk.com/robots.txt
Redirect Domain www.hometalk.com
Redirect Base hometalk.com
Domain IPs 108.61.152.35, 45.77.153.86
Redirect IPs 151.101.1.91, 151.101.129.91, 151.101.193.91, 151.101.65.91, 2a04:4e42:200::347, 2a04:4e42:400::347, 2a04:4e42:600::347, 2a04:4e42::347
Response IP 199.232.45.91
Found Yes
Hash bdf0e00ef53e782d673b535a27e903721d0c70b48efa82ef9fceae865c28d4f2
SimHash 010e8b50d4b3

Groups

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

mediapartners-google

Rule Path
Allow /

opebot-v (https://www.1plusx.com)

Rule Path
Allow /

*

Rule Path
Disallow /?*activity_id=*
Disallow /ajax/
Disallow /tour/
Disallow /browser
Disallow /delete
Disallow /sharebox
Disallow /account
Disallow /flag
Disallow /logging/
Disallow /track_ht/
Disallow /js/errors
Disallow /homemade
Disallow /comment
Disallow /hashtags
Disallow /outdated
Disallow /board
Disallow /notifications
Disallow /paid-acquisition
Disallow /rapid
Disallow /react
Disallow *size%3D
Disallow *sort%3D
Disallow *se%3D
Disallow *%3B
Disallow *force_refresh%3D
Disallow *split_test%3D
Disallow /l/
Disallow /shop/
Disallow */buy.html*
Disallow /*?*min=*
Disallow /*?*max=*
Disallow /search/
Disallow /api/slots/
Disallow /api/cmshub/
Disallow /api/posts/*/popular_posts
Allow /member/255437/
Allow /member/112206/
Allow /member/25692837/
Allow /member/181132/
Allow /member/3317959/
Disallow /member/
Disallow /posts/

Other Records

Field Value
sitemap https://www.hometalk.com/rfs/sitemaps/hometalk.sitemootx.xml