ketab.land
robots.txt
Robots Exclusion Standard data for ketab.land
Resource Scan
Scan Details
| Site Domain | ketab.land |
| Base Domain | ketab.land |
| Scan Status | Ok |
| Last Scan | 2025-11-01T20:15:01+00:00 |
| Next Scan | 2025-12-01T20:15:01+00:00 |
Last Scan
| Scanned | 2025-11-01T20:15:01+00:00 |
| URL | https://ketab.land/robots.txt |
| Domain IPs | 88.135.68.9 |
| Response IP | 88.135.68.9 |
| Found | Yes |
| Hash | abe42b29fd718a9f135cdfad6b15b1813d872b7636ca70efa6c34dc1180fa150 |
| SimHash | c80c98d30f3b |
Groups
*
openai
gptbot
anthropic
claudebot
google-deepmind
gemini
perplexity
mistral
cohere
ai21
youbot
| Rule | Path |
|---|---|
| Allow | /wp-admin/admin-ajax.php |
| Allow | /*.js$ |
| Allow | /*.css$ |
| Disallow | *?* |
| Disallow | */?* |
| Disallow | *%D8%9F* |
| Disallow | /?v* |
| Disallow | *%20* |
| Disallow | *%2B* |
| Disallow | *%28* |
| Disallow | *%29* |
| Disallow | /?cms_block* |
| Disallow | /?mweb |
| Disallow | /?mweb=* |
| Disallow | /woodmart_layout* |
| Disallow | /e-floating* |
| Disallow | /wp-login/ |
| Disallow | /wp-admin/ |
| Disallow | /cart/ |
| Disallow | /cgi-bin |
| Disallow | /readme.html |
| Disallow | /*rss* |
| Disallow | /*feed* |
| Disallow | /rss/ |
| Disallow | /feed/ |
| Disallow | /checkout/ |
| Disallow | *my-account* |
| Disallow | /wishlist/ |
| Disallow | /wp-includes |
| Disallow | /login/ |
| Disallow | /test/ |
| Disallow | /signin/ |
| Disallow | /author/ |
| Disallow | /uncategorized/ |
| Disallow | /archive/ |
| Disallow | */?orderby* |
| Disallow | */comment-page* |
| Disallow | */woodmart_layout/* |
Other Records
| Field | Value |
|---|---|
| sitemap | https://ketab.land/custom-html-sitemap.html |
| sitemap | https://ketab.land/sitemap-index.xml |
| sitemap | https://ketab.land/sitemap-index.xml |
Comments