HA! so I'm not the only one who's bike has decided to do that. When mine did it, it ripped 1/2 the spokes out of the wheel, but didn't really damage the tensioner or bike frame too much. When I got the wheel back together, it tried to do it again. This time I noticed it wouldn't sit tight no matter what I did so I drilled a hole thru both halves of the chain tensioner bracket and the chain-stay on the frame and used a piece of the broken spoke as a pin to pin it all together. Its really quite solid, about as solid as I think its going to ever be without welding.
(Ignore the bits of inner tube in the bracket, they were their to help keep it still while I drilled, never bothered to take them out)