Why leave Tsurumaru behind?

For me, it was one of the most heartbreaking parts of the film. His sister, who is implied to at least visit him, takes him away - which he is obviously very nervous about. And rather than staying with him, and keeping him safe and happy, she leaves him behind for absolutely no reason. Sure, she was worried about the old woman, but it was her brother, her blind brother who was scared and defenseless.

For me, it poses one of the few issues I have with the film. Namely that it's so unrelentingly grim that I was just left cold at the end. I so wanted Tsurumaru to have a happy ending, after everything he'd suffered, but he was left without a family, without any companionship.
At that point, it just felt like needless spite, like kicking a one-legged puppy.