Roger Moore's age really wasn't a problem at face value?
The real problem is that they wrote his James Bond as if he was ostensibly, still a younger man, like he was when he did Live & Let Die and The Man With the Golden Gun or even when Moore was younger than that, when he was on The Saint.
So of course, Bond in A View to a Kill is performing stunts as if it's no big deal and we're supposed to accept or believe that it's clearly not Roger Moore. And he's so lecherous, that he has little qualms with hooking up and bed hopping with multiple women (many of whom, such as Tanya Roberts, literally look like they could be his daughters) in a single film.
Had they written Moore's Bond like he was in For Your Eyes Only, then don't you think that the public would've been more accepting of the fact that a 50+ year old man was playing 007? In other words, that movie seemed to allude much further that we were watching a more mature, wiser Bond.