temuapp.dev
robots.txt

Robots Exclusion Standard data for temuapp.dev

Resource Scan

Scan Details

Site Domain temuapp.dev
Base Domain temuapp.dev
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2025-09-16T11:28:46+00:00
Next Scan 2025-12-15T11:28:46+00:00

Last Successful Scan

Scanned2024-05-02T08:53:10+00:00
URL https://temuapp.dev/robots.txt
Redirect https://shopperhand.com/robots.txt
Redirect Domain shopperhand.com
Redirect Base shopperhand.com
Domain IPs 104.21.39.155, 172.67.146.111, 2606:4700:3030::ac43:926f, 2606:4700:3034::6815:279b
Redirect IPs 104.21.17.74, 172.67.223.60, 2606:4700:3030::ac43:df3c, 2606:4700:3036::6815:114a
Response IP 104.21.17.74
Found Yes
Hash db5fada3f27e9cab78f8c2cc4660bf34a8641e8492c415b72b3f071747efe9ce
SimHash 345459e10512

Groups

googlebot

Rule Path
Allow *.js
Allow *.css
Allow /wp-admin/admin-ajax.php
Disallow /?s=
Disallow /search/
Disallow /wp-admin
Disallow /*/feed/
Disallow /wp-login.php
Disallow /wp-register.php
Disallow /trackback/

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin
Disallow /wp-login.php
Disallow /trackback/
Disallow /wp-register.php

Other Records

Field Value
sitemap https://shopperhand.com/sitemap_index.xml

Comments

  • Allow files critical for rendering
  • Allow AJAX - Do Not Remove
  • Prevent crawl-budget waste on search pages
  • Prevent private admin areas from being crawled
  • Prevent duplicate /feed/ pages from being crawled
  • Prevent login page crawls etc
  • Prevent register page crawls etc
  • Prevent Trackback Neg SEO
  • Allow AJAX - Do Not Remove
  • Add all sitemaps