smartupworld.com
robots.txt

Robots Exclusion Standard data for smartupworld.com

Resource Scan

Scan Details

Site Domain smartupworld.com
Base Domain smartupworld.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-22T03:00:10+00:00
Next Scan 2026-02-20T03:00:10+00:00

Last Successful Scan

Scanned2025-07-26T02:24:21+00:00
URL https://smartupworld.com/robots.txt
Domain IPs 2a02:4780:16:aca1:edc7:f404:1659:9cc8, 2a02:4780:38:d8a0:83ee:62b5:6a1:c617, 84.32.84.252, 84.32.84.60
Response IP 93.127.201.240
Found Yes
Hash e489be8ab9e558f204a877b8be0757a38b7630bb8ba6685b32b95bb537a85402
SimHash 49e2df556b0b

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /cgi-bin/
Disallow /xmlrpc.php
Disallow /wp-login.php
Disallow /wp-register.php
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://smartupworld.com/sitemap_index.xml

Comments

  • Allow all user agents full access except disallowed paths
  • Allow admin-ajax for proper AJAX functionality
  • Sitemap location
  • Optional: crawl-delay (in seconds) - usually unnecessary unless you have server overload issues
  • Crawl-delay: 10