I could go on for hours about how important it is to me that Killian calls her Swan, mostly the meaning behind it and their relationship.
Mainly, that's the name Emma got to choose for herself. It represents the first 28 years of her life where all she remembers is being alone and abandoned. She'd been in foster care, known what it was like to be in a loving family, then sent back. She found a love (it pains me to refer to Neal as that) that abandoned her and left her in jail, so then she had to have Henry at 18 completely alone and give him up because she thought she wasn't good for him.
For the next ten years she made an armour for herself, she found a job and was making a life for herself. Was she lonely until Henry appeared? Evidently, but she'd gotten through so much.
When her and Hook meet it's an instantly "armour on" situation for her because he's flirting, trying to get to know her, and that always meant that they'd end up leaving. So imagine her surprise when there's this amazingly patient, understanding (and obviously handsome) pirate at her beck and call. She pushes him away and all he does is comprehend why Emma puts those walls up, and let her drop them at her own pace.
When they get together it's this amazingly romantic acts of service type of love and she's so surprised that he gave up his most precious possession for her because she's usually the one that gets dropped.
You can see the shock on her face and slowly the understanding that he truly cares for her.
I kind of went off the rails but... Through all this he's always known her as Swan, even though everyone called her Emma. He respected the person she was and didn't see it as a facade, but more so, a tough part of her keeping the softness safe.
All throughout s4 you can see how she starts to be softer with him, which to many felt like "ruining her character because she got with a man", when in reality he was the one person she didn't feel the need to be a saviour around.
Emma was the name her parents gave her, and while she loves them very much, that mostly represented her as the saviour from the first instant.
It was hardly ever used to refer to her as an actual human in a way, not a sort of weapon.
Later on (I feel like in s5 especially) you can see him understanding how she feels the pressure of being the saviour constantly. Though he's always seen every part of her, when she turns into The Dark One he's the one that grounds her and reminds her of who she is, and that she's not only the saviour.
"I know The Dark Ones inmortal, love, but Emma isn't. Bring her home to me." While in this part he doesn't call her Swan, it represents so well how he cared for her. (I quoted that from memory so the wording might be wrong but it gets the point across)