Log In Sign Up

giverrang.com
robots.txt

Robots Exclusion Standard data for giverrang.com

Resource Scan

Scan Details

Site Domain giverrang.com
Base Domain giverrang.com
Scan Status Ok
Last Scan2025-09-01T08:39:11+00:00
Next Scan 2025-10-01T08:39:11+00:00

Last Scan

Scanned2025-09-01T08:39:11+00:00
URL https://giverrang.com/robots.txt
Domain IPs 108.138.7.111, 108.138.7.21, 108.138.7.26, 108.138.7.41, 2600:9000:2175:1600:18:216f:4340:93a1, 2600:9000:2175:7e00:18:216f:4340:93a1, 2600:9000:2175:8600:18:216f:4340:93a1, 2600:9000:2175:9400:18:216f:4340:93a1, 2600:9000:2175:a00:18:216f:4340:93a1, 2600:9000:2175:d200:18:216f:4340:93a1, 2600:9000:2175:d400:18:216f:4340:93a1, 2600:9000:2175:ee00:18:216f:4340:93a1
Response IP 18.165.72.32
Found Yes
Hash a976ccbd13296444a37ce54ad5f060a92291b96f1204388dbe64382d2742d084
SimHash 6008ca44d3da

Groups

ahrefsbot
googlebot-image
bytespider
imagesiftbot

Rule Path
Disallow /gift-card/
Disallow /indie-gift-cards/
Disallow /static/images/
Disallow /merchant-assets/
Disallow /programs/
Disallow /*-gift-cards/
Disallow /*-gift-cards-*/

claudebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1
Back to top

Other Records

Field Value
sitemap https://giverrang.com/sitemap-index.xml
Back to top

Comments

  • Block image bots from using up our Cloudinary quota
  • Limit ClaudeBot to one request per second
Back to top