Bees harvest plants for their pollen, not their perfumes. Your family didn't use the place all year so the bees may well have begun their occupation when your family wasn't there. They don't abandon hives very readily when they're well-established unless they get totally smoked-out with something a lot more noxious than a little human-generated methane. ;)

We had bumblebees who lived under our compost pile happily for years. My dad used to knock on the corner post every day when he took a load of veggie peelings or grass clippings down so that they would have some warning, and they would fly out and buzz a little, but he never got stung. They got a warm place to live, and actually aerated the compost for us a bit and killed pests, plus they did a lot of pollination. It was a much safer situation than yours, though.