Negan's conversation with Maggie, mentioning Hershel (her father)

I don't know if I was hearing it wrong or what, but it sounded like during the conversation between Negan and Maggie in Rick's last episode, that Negan seemed to be taking credit for what happened to her father. Maybe I'm remembering wrong, but I think he said what he did to her father and not to "Hershel" (which could've referred to Maggie and Glenn's son). The Governor was the one who killed Hershel, not Negan. Did I mishear that? Was he referring to Hershel, the baby, and him taking a father away from his son or did they mistakenly mix-up Negan and the Governor? I'd recheck myself, but I deleted the episode from my DVR already.