webz.io
robots.txt

Robots Exclusion Standard data for webz.io

Resource Scan

Scan Details

Site Domain webz.io
Base Domain webz.io
Scan Status Ok
Last Scan2025-08-27T07:27:05+00:00
Next Scan 2025-09-26T07:27:05+00:00

Last Scan

Scanned2025-08-27T07:27:05+00:00
URL https://webz.io/robots.txt
Domain IPs 104.26.0.146, 104.26.1.146, 172.67.70.63, 2606:4700:20::681a:192, 2606:4700:20::681a:92, 2606:4700:20::ac43:463f
Response IP 104.26.0.146
Found Yes
Hash 4b685cbae1025720d9b701708d685c290afe2d7cea39bfc9ebdd2c8362034ea1
SimHash 4914bb108e13

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /thank-you/resource
Disallow /news-api/*
Disallow /cnt/

chatgpt-user
oai-searchbot
gptbot

Rule Path
Allow /

perplexitybot
perplexity-user

Rule Path
Allow /

claudebot

Rule Path
Allow /

Other Records

Field Value
sitemap https://webz.io/sitemap_index.xml