Why do you think the promise to get together in their 40s worked for Chandler and Monica, but not for Ted and Robin? Why does one feel right while the other feels forced?
Here's the thing: it DIDN'T work for Chandler and Monica. That's not why they got married. They were each other's FIRST choice, not a back-up plan, and the writers of Friends were careful to show that - while HIMYM completely fucked itself over by insisting that it's romantic to settle for someone you've repeatedly made clear you'd never, ever see as your dream spouse.
While those moments of Chandler offering to be Monica's back up husband were cute and do say something about how comfortable he is with her that he, a notoriously anxious and even commitment-phobic man, could imagine being, at the very least, okay with spending his life with her, they were still JOKES, and the punchline was always "this is never happening and Chandler regrets offering."
The first time the backfires because Monica asks why exactly she WOULDN'T be married at 40 and thus NEED a back-up plan, to which Chandler panics at both her overreaction and his own inability to handle it, aka proof that this back-up is a very bad idea.
The second time it's when he's helping her after a jelly-fish sting and she says "You'll always be the guy that peed on me" to which Chandler laughs and drops the topic. It's the show reinforcing the idea that, while they have some chemistry (pretty much the entire cast does, hence every ship being teased at some point) and trust and like each other, they still primarely see each other as friends. JUST friends. They enjoy hanging out and getting up to stupid shenanigans together, but they both want more than just that from a relationship and are not yet in a place where they can offer it to each other.
THE exception is that one episode in which we're seeing how everyone almost hooked up with everyone before, and Chandler and Monica are hugging while she's wearing just a towel, and talking about how it feels great and right, and it seems like they're talking about feeling a certain spark with each other - only for it to be revealed they're just big fans of a high quality towel. This one is notable because while it has a joke, it did not outright shut the door on this ship being a possibility in the future, as the scene still ends in a tender moment.
And then the trip to London happens. It starts off in the same typical way Mondler was teased as a ship before - Monica feels bad about herself, Chandler comforts her in a way that implies he might be into her and they'll be a thing... only this time the show goes all out. There's no moment in which anyone says the wrong thing and ruins the mood, or in which both characters back off or turn out to be talking about something else, nope.
Chandler is just completely sincere in saying Monica is the most beautiful woman in most rooms and that he can't understand how someone could NOT want her and thus she takes the chance to jump his bones to see if he means it. And their night together is so great that they want to do it again because once was not enough. Then it turns out twice is not enough either. Nor three, nor four, nor five, nor six times.
And would you look at that, not only is there great sexual chemistry, it is SO great that they don't want to sleep with anybody else. And Chandler got better at handling the fact that Monica is high maintanence (to the point that he even grows to like it) and Monica is understanding when he messes up to. And now their stupid shenanigans are no longer all they have to offer to each other, and end up becoming a bonus, hence Monica highlighting their friendship when saying her vows.
At no point was the "back-up husband" suggestion brought up. They even have the episode that explains Monica was originally looking for Joey when she first hooked up with Chandler (as a nod to the show's original idea for them to be each other's endgame) spell out for us that, no, the plot would not have gone to same way if Chandler had been replaced with someone else, because no other guy would have brought that specific kind of chemistry, history, joy and peacefulness that only Chandler had managed to bring to her life.
And when her ex wants her back right as Chandler had the terrible idea of pretending he didn't want to get married to surprise her with a proposal? Monica still says that none of it is fair, and that Richard should have either made that decision BEFORE or that Chandler should be the one proposing NOW - because he's the one she's in love with, not her ex. Him and Richard are not interchangeable, and the happy memories of the past wouldn't be enough to give her the future she wanted because it'd be with the wrong guy.
Even in the goddamn AU episode in which Monica is dating Doctor Boring, it still ends with her losing her virginity to Chandler as "practice", and it being so great that he wants to be her actual boyfriend and as soon as he says that Monica leaps to his arms and ditches the other guy, whom she didn't even seem that into and canonically found frustrating and was all but said to only be with because she thought no one else would want her - aka what settling looks like.
The show makes it clear over and over again that these two's relationship is NOT a safety net in case they can't find someone else. The back-up husband thing was a hint of a spark that could, potentially, turn into something more, something real, not a promise of a mere consolation prize.
And then we have Ted and Robin. Two people that already dated and, more importantly, already broke up due to not wanting the same things for their future. Robin wants a life full of adventure and going from place to place for her job, doesn't care much if she'll be married or not, and has been very clear about not wanting to be a mom. Meanwhile Ted wants a wife, kids and a quiet life in which he grows old in a neat little house with a white picket fence in the suburbs.
Right away this promise to be each other's back up is a disaster due to being the characters trying to go back to a relationship they both admited was doomed to fail - the show even compared them to Ted's DIVORCED parents - instead of a form of hope for the future, even if not an ideal scenario.
It gets worse when we remember that, before their break up, Ted and Robin already tried living together as a couple and failed, and that when they first became roomates they struggled so much to not get on each other's nerves that they HAD to have sex just so they wouldn't kill each other (though at least they eventually managed to work through it).
And then the final nail on the coffin happened. Ted himself confronted Robin about this promise, saying that, as long as it existed, it would not allow him to move on with his life, and that he needed to know ONE simple thing: Did Robin actually love him?
Was the reason they couldn't be together at the moment some timing issue, her having feelings for more than one person, their incompatible plans for the future, or any other kind of obstacle that could maybe someday be dealt? Or was it a simple case of "You would NEVER be my first choice. Maybe not even my second, third, or fourth"?
Robin told him, to his face, that no, she didn't love him. And then she went on to marry Barney, while Ted met the real love of his life - someone he didn't say "I love you" to right after giving a whole speech about how he was depressed because he didn't love anyone, because yes, THAT'S the reason why he tried pursuing Robin again. Even he eventually recognized that in the last season, saying he was not the same man he was when they first met, and that they had not been in love with each other for YEARS at that point, and he had simply been holding onto a past love that no longer made any sense for his life.
And then the show has them get back together anyway and the writers SWEAR that this is a romantic, super happy ending because these two were always "meant to be" and are totally not settling - even though they're only pursuing each other again because Robin got divorced and Ted's wife DIED.
Chandler offering to marry Monica if she didn't find anyone before turning 40 is a detail that is funny in the moment and cute in hindsight due to the context of "Little do they know that they're gonna get married WAY before that because they'll genuinely fall head over heels in love with each other and be one of the most beloved couples in television history."
Ted and Robin saying that, if everything in their life went to shit, they'd settle for each other, and then actually settling after losing their soulmates is such an obviously terrible decision for a story that is supposed to be about true love that it actually killed the show and the reputation of the writers.
