SPOILERS: Lee's sister and the first house
At the beginning of the film,Em is talking to Kevin on the phone and the connection dies. She says "I lost you",and I believe that 'her' Kevin is now gone. The house we first see has only Mike, Lee and Beth at first, so they all belong to that house. The other five guests then arrive. We know Laurie is a visitor, because she does not recognize Mike as being in the show 'Roswell', and does not teach yoga. Since Amir arrived with Laurie, he must be a visitor also. About nine minutes into the film, Lee tells the story of how her sister got to be Halley's comet in a school play when they were kids. These are people who have known each other for years, yet we hear Amir say "I've never met her sister- have you guys met her sister?"
And Em and Kevin look at her with quizzical expressions. I think Em and Kevin and Amir came from a reality where Lee had no sister. Since Hugh arrived with Kevin, he must be a visitor also, and I have to go look at the film again to see if there are any more clues to indicate he was already in the wrong house when he came in.
So only Mike, Lee and Beth belong to the first house, the rest are already visitors. And when they all go outside, even though they did not go into the 'extra dark area', they return to a different house, one with a broken glass. So from this point on they are all visitors.
Btw, I judge a film on how much it makes me think, and Coherence certainly does that.