Process walnuts in a food processor until finely ground; transfer to a large bowl. Process mushrooms in food processor until finely chopped. Add butter, processing until just combined. Transfer mushroom-butter mixture, brown rice, oats, beet, soy sauce, salt, pepper, garlic, and 2 tablespoons of the mayonnaise to bowl with ground walnuts; stir well to combine. Chill 1 hour.
Shape mixture into 4 (1/2-inch thick) patties. Heat a grill pan over high. Coat with cooking spray. Cook patties until heated through and lightly charred, about 3 minutes per side. Spread ketchup, mustard, and remaining 1 tablespoon mayonnaise evenly on buns. Serve beet burgers on buns topped with lettuce, tomato, and onion.