Log In Sign Up

feeltravelling.com
robots.txt

Robots Exclusion Standard data for feeltravelling.com

Resource Scan

Scan Details

Site Domain feeltravelling.com
Base Domain feeltravelling.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-09-12T07:27:59+00:00
Next Scan 2025-11-11T07:27:59+00:00

Last Successful Scan

Scanned2025-06-21T22:08:11+00:00
URL https://feeltravelling.com/robots.txt
Domain IPs 104.21.60.158, 172.67.198.4, 2606:4700:3032::6815:3c9e, 2606:4700:3035::ac43:c604
Response IP 104.21.60.158
Found Yes
Hash e2d3c1513308f79de00947b556638418037adb09d9f5b7671aac5e6e7e60d5a8
SimHash 24781b59e4c3

Groups

*

Rule Path
Disallow
Disallow /cdn-cgi/
Allow /css/
Allow /js/
Allow /images/
Allow /static/
Allow /content/
Disallow /wp-admin/
Disallow /private/
Disallow /temp/
Disallow /cgi-bin/
Disallow /search
Back to top

Other Records

Field Value
sitemap https://www.example.com/sitemap.xml
Back to top

Comments

  • Allow all user agents to access everything
  • Disallow crawling of specific Cloudflare challenge pages (if used)
  • Allow access to the main assets and pages
  • Block sensitive directories or files that shouldn't be indexed
  • Block access to search results pages (to avoid duplicate content)
  • Sitemap location
Back to top