Certain things off-putting about the father (spoilers)
.... we realise the father was bricked into the attic and could move around the dark walls .. considering he was terribly malnourished, wouldn't expect him to be strong enough to be a credible threat after months (maybe years?, I'm forgetting) of being locked up... esp when he attacked Allie (even if we consider she was in shock)...
... have to assume that Jack was keeping him alive in some way by feeding him from time to time...