Log In Sign Up

shoptagr.com
robots.txt

Robots Exclusion Standard data for shoptagr.com

Resource Scan

Scan Details

Site Domain shoptagr.com
Base Domain shoptagr.com
Scan Status Ok
Last Scan2025-05-29T22:02:15+00:00
Next Scan 2025-06-05T22:02:15+00:00

Last Scan

Scanned2025-05-29T22:02:15+00:00
URL https://shoptagr.com/robots.txt
Redirect https://www.karmanow.com:443/robots.txt
Redirect Domain www.karmanow.com
Redirect Base karmanow.com
Domain IPs 3.131.150.69
Redirect IPs 13.35.238.110, 13.35.238.121, 13.35.238.35, 13.35.238.41, 2600:9000:2085:5a00:a:c74f:2dc0:93a1, 2600:9000:2085:6800:a:c74f:2dc0:93a1, 2600:9000:2085:7600:a:c74f:2dc0:93a1, 2600:9000:2085:9200:a:c74f:2dc0:93a1, 2600:9000:2085:a00:a:c74f:2dc0:93a1, 2600:9000:2085:a200:a:c74f:2dc0:93a1, 2600:9000:2085:d400:a:c74f:2dc0:93a1, 2600:9000:2085:e00:a:c74f:2dc0:93a1
Response IP 13.35.238.110
Found Yes
Hash 43f01ebeb7b1d477dd8c1ba181f77d4f30e0d6809df43148df1acf5f857f530f
SimHash 820509856074

Groups

axios/0.19.2

Rule Path
Disallow /

axios

Rule Path
Disallow /

apptus_tce

Rule Path
Disallow /

apptus_tce/1.2

Rule Path
Disallow /

msrbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

vsyncrawler/1.0

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /ref
Disallow /ref/
Disallow /ref/*
Disallow /redirect*
Disallow /v3
Disallow /v3/
Disallow /v3/*
Disallow /demo/signup
Disallow /demo/signup/

*

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.karmanow.com/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top

Warnings

  • 2 invalid lines.
Back to top