Log In Sign Up

32auctions.com
robots.txt

Robots Exclusion Standard data for 32auctions.com

Resource Scan

Scan Details

Site Domain 32auctions.com
Base Domain 32auctions.com
Scan Status Ok
Last Scan2026-01-18T00:04:33+00:00
Next Scan 2026-01-25T00:04:33+00:00

Last Scan

Scanned2026-01-18T00:04:33+00:00
URL https://32auctions.com/robots.txt
Redirect https://www.32auctions.com/robots.txt
Redirect Domain www.32auctions.com
Redirect Base 32auctions.com
Domain IPs 104.20.27.60, 172.66.172.42, 2606:4700:10::6814:1b3c, 2606:4700:10::ac42:ac2a
Redirect IPs 104.20.27.60, 172.66.172.42, 2606:4700:10::6814:1b3c, 2606:4700:10::ac42:ac2a
Response IP 104.20.27.60
Found Yes
Hash c39d0b50e289f461018a8c9fb786b1080c3ea8613dabf9c8290ba4fcb0f64a72
SimHash da014d95c671

Groups

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Disallow
Allow /organizations/

mediapartners-google*

Rule Path
Allow /

*

Rule Path
Disallow /admin
Disallow /my_auctions
Disallow /my_invitations
Disallow /organizations/
Disallow /system/
Disallow /users/
Disallow /help
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-Agent: *
  • Disallow: /
  • Always allow the facebook crawler
  • Only allow Twitter in a specific directory
  • Allow adsense anywhere
  • All other bots are blocked from specific pages
Back to top