They are weak due to being born and raised in very low-g, so how are they ever any match for Earthers when it comes to hand to hand fighting or acceleration? Mars at least has their military personnel train in 1g as stated.
In the very first episode they made a point of how skinny and weak Belters are, but it seemed like it was very random depending-on-what-the-plot-demands after that.
I guess the idea is that they are reasonably well able to tolerate relatively short-term high G, while strapped into acceleration couches. It makes some sense, I suppose. In such a case, weaker muscles and bones wouldn't be such a handicap, though I wonder in reality how much the weaker muscles around the Belter's chest would cope with the sense of crushing weight in high G, and allow them to breathe.
As far as fighting, they are shown to be weaker at hand to hand. There's a scene after Miller -- who is shown to be a pretty competent fighter in other scenes -- comes about the Rocinante and he punches Amos. Amos barely even flinches, and then proceeds to almost literally wipe the floor with him. And there is no sense at all the Amos is even remotely challenged. Amos is strong, even by Earther standards, and Miller the Belter just doesn't stand a chance, despite having possibly equal martial arts skill.