huggies.com.vn
robots.txt

Robots Exclusion Standard data for huggies.com.vn

Resource Scan

Scan Details

Site Domain huggies.com.vn
Base Domain huggies.com.vn
Scan Status Ok
Last Scan2025-12-20T05:42:20+00:00
Next Scan 2026-01-19T05:42:20+00:00

Last Scan

Scanned2025-12-20T05:42:20+00:00
URL https://huggies.com.vn/robots.txt
Redirect https://www.huggies.com.vn/robots.txt
Redirect Domain www.huggies.com.vn
Redirect Base huggies.com.vn
Domain IPs 104.18.36.86, 172.64.151.170, 2606:4700:4404::6812:2456, 2a06:98c1:3109::ac40:97aa
Redirect IPs 104.18.36.86, 172.64.151.170, 2606:4700:4404::6812:2456, 2a06:98c1:3109::ac40:97aa
Response IP 104.18.36.86
Found Yes
Hash df97b9ca27865c4645d7820b4fdf1a23ff5f92d6678150020bfc29471d320dc6
SimHash 603a3057cdfd

Groups

oai-searchbot
gptbot
chatgpt-user
perplexitybot
firecrawlagent
andibot
exabot
phindbot
youbot
rag bots
inference bots
google-extended
chatgpt-user
qai-searchbot

Rule Path
Allow /

claudebot
ccbot
bytespider
chatgpt

Rule Path
Disallow /

googlebot
bingbot

Rule Path
Allow /

*

Rule Path
Disallow /Config/
Disallow /bin/
Disallow /config/
Disallow /data/
Disallow /install/
Disallow /masterpages/
Disallow /python/
Disallow /scripts/
Disallow /umbraco/
Disallow *user
Disallow /xslt/
Disallow *%26
Disallow /account
Disallow *clid
Disallow *openstat
Disallow *utm
Disallow /tags
Disallow *.swf
Disallow /categories
Disallow /Error
Disallow /Home
Disallow /recovery
Disallow /subscriptions
Disallow *.asp
Disallow *search
Disallow *?version
Disallow *external-test
Disallow *mindbox-click-id
Disallow /search
Disallow /articlesearch
Disallow *result
Disallow *email-protection
Disallow */old
Disallow *?url
Disallow *?type
Disallow /cdn-cgi/
Allow /-/*.svg
Allow /scripts/*.js
Allow /js/*.js
Allow *.css
Allow *.js
Allow *.png
Allow *.pdf
Allow *jpg

Other Records

Field Value
sitemap https://www.huggies.com.vn/sitemap.xml

Comments

  • Allow AI search and agent use
  • Disallow AI training data collection
  • Allow traditional search indexing
  • Disallow access to admin areas for all bots