I assumed it was a pact between the mother and brother. It looks like some folks on the board believe it was simply "a pact of silence;" that the mother knew her brother was a monster and protected him. I believe it went beyond that, though, and that she actively fed him victims.
I also believe that the bad memories the girls had of the closet were from being abused by the uncle (likely in the dark, so they didn't know who was doing what), but there was an agreement that he wouldn't kill the girls.
I took it to mean the bond between the sisters. When she is asked by her cousin why she came back it seems like an important reply "Because we are sisters" and sisters have an unwritten pact. She wasn't going to leave until she found out what happened to her either. Part of the sister pact. If it was about the mother and uncle (which seems was definitely a pact) we have to have watched the whole movie before we get it which would just making explaining the premise of the film to friends difficult. "So what's the pact?" "OK I'll tell you, but... spoiler alert" Of course it could mean both.