Olivia's' abortion
Does Fitz understand her choice because he believes it was Jakes child?
When Olivia left the White House after being outed ( season 5 ep "Yes") she met up with Jake, Fitz saw this video of them on the news, even though Olivia and Jake didn't have sex Fitz might assume that they did and rationalize the abortion as not his child but Jakes.
We're all on our way out, act accordingly.