bids.auctionbymayo.com
robots.txt

Robots Exclusion Standard data for bids.auctionbymayo.com

Resource Scan

Scan Details

Site Domain bids.auctionbymayo.com
Base Domain auctionbymayo.com
Scan Status Ok
Last Scan2026-02-09T03:02:24+00:00
Next Scan 2026-03-11T03:02:24+00:00

Last Scan

Scanned2026-02-09T03:02:24+00:00
URL https://bids.auctionbymayo.com/robots.txt
Domain IPs 172.66.40.182, 172.66.43.74, 2606:4700:3108::ac42:28b6, 2606:4700:3108::ac42:2b4a
Response IP 172.66.40.182
Found Yes
Hash 1aec7ada04545d52bf6aef1f19410ae13d583996fbcd52bf15c52c99f7deb569
SimHash 290e4c02ac80

Groups

*

Rule Path
Disallow /admin/
Disallow /api/
Disallow /asset/

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot-sa

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

petalbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://bids.auctionbymayo.com/sitemap.xml
sitemap https://bids.auctionbymayo.com/sitemap/auctions
sitemap https://bids.auctionbymayo.com/sitemap/past-auctions

Comments

  • Reduce Semrush crawl interval