Read the book
Probably so few people watch this film that my remarks will ring silently, but here goes. I'm a big John O'Hara fan. His short stories about small Pennsylvania cities and their residents are fabulous studies of psychological complexity. Butterfield 8 and Appointment in Samarra are must-read novels.
Paul Newman and Joanne Woodward and Myrna Loy are just fine in their assigned roles. The problem is that Hollywood butchered this story. The questions that many of the IMDB commenters have would be resolved if the producers had stuck to the story. Hollywood was unable because the story is huge. It doesn't begin when Alfred comes home from World War 2. It begins at his birth around 1900 and follows his development, in detail, through his early years, the death of his brother, his estrangement from his old man, his mother's affair and alcoholism, through his boarding school years (where he meets Lex and his family) and into his enlistment in the navy in World War One!!
The story covers the odd combination of hot and cold in his relationship with his wife, the slow development of his relationship with Natalie. Heterosexuality, homosexuality, parental stuff, loyalty, betrayals. It's a great read. This movie is a insult to a deep and intense plot.
Ohara had demons and his literary reputation took some heat, but his brain was a fertile playing field for some tough concepts. His characters are flawed, their decisions ugly at times, just like many of us. Pick up one of his short story collections. Great stuff. Thanks for listening to my rant.