If you think about it
By the 2nd or 3rd season, Raylon Given had single-handedly destroyed so many intricate criminal endeavors and cost so many lives that, realistically, the criminal element would have just had him killed. Sure killing Givens would be easier said than done, but blowing up his PoS hotel room, bombing his car, or shooting him with a sniper rifle would have happened at some point. He basically hid behind his badge and personal fighting skills for the entire show. In short, if killing him up close and personal is too difficult, and time and again that was shown to be the case, then do it from afar and accept the risk of collateral damage.
And let's face it, as a deputy US Marshal, he was dirty as hell. Of course, I tuck through the show from start to finish so I found it very entertaining...