smartours.com
robots.txt
Robots Exclusion Standard data for smartours.com
Resource Scan
Scan Details
| Site Domain | smartours.com |
| Base Domain | smartours.com |
| Scan Status | Ok |
| Last Scan | 2025-10-18T13:37:13+00:00 |
| Next Scan | 2025-11-17T13:37:13+00:00 |
Last Scan
| Scanned | 2025-10-18T13:37:13+00:00 |
| URL | https://smartours.com/robots.txt |
| Domain IPs | 13.35.202.129, 13.35.202.42, 13.35.202.61, 13.35.202.71 |
| Response IP | 13.35.202.42 |
| Found | Yes |
| Hash | 7050213145cbb727b7b62e896ab2597299b7b030567dba879d5e04fcac3228e9 |
| SimHash | 771a0953c082 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /api |
| Disallow | /static/js |
| Disallow | /redirect-partner |
| Disallow | /partner-reservation |
| Disallow | /refine |
| Disallow | /bookmark |
google-extended
gptbot
oai-searchbot
chatgpt-user
chatgpt-user/2.0
anthropic-ai
claudebot
claude-user
claudebot-searchbot
claude-web
cohere-ai
applebot
applebot-extended
amazonbot
perplexitybot
perplexity-user
facebookbot
meta-externalagent
linkedinbot
bytespider
duckassistbot
ai2bot
ccbot
diffbot
omgili
timpibot
youbot
mistralai-user
| Rule | Path |
|---|---|
| Disallow | /api |
| Disallow | /redirect-partner |
| Disallow | /partner-reservation |
| Disallow | /refine |
| Disallow | /bookmark |
Other Records
| Field | Value |
|---|---|
| crawl-delay | 20 |
Other Records
| Field | Value |
|---|---|
| sitemap | https://smartours.com/sitemap.xml |
| sitemap | https://smartours.com/gz/site-map-all.xml |