Honestly, despite my ongoing beef with the episode 5.16 dark side of the moon, the reason why it is so contentious for me is because (in my opinion) it is actually a great episode that offers so much insight to the characters, their relationships and their psychology.
It's no surprise that this sort of supernatural intervention trope in having characters look into each other's memories is used to delve into psychoanalytical readings, often representing characters seeing each other in different perspectives and developing greater understanding between them. Having the winchesters be plunged into this oddly nihalistic interpretation of heaven where they get to see not only their top memories on remix but the few moments of their lives without the other is a very interesting (and also blatent) representation of this.
From sam's side, he sees dean's memories without him, or rather the one memory which is a simple breakfast with his mum as a toddler (as he never sees dean's memory of the fourth of july fireworks, which is sad). Seeing this, sam sees a new side to his family that he's never been privy to - a life before monster hunting (Mary's death acting as a catalyst for this change). Here he sees the roots of dean and john's militiristic relationship on display, sympathising that he didn't realise how long dean had been cleaning up their dad's messes. Although he is vaguely pushed out of experiencing the memory by dean, who remarks that it's "not your memory, sam" when he attempts to get the attention of their dead mum, sam's only take away from this memory is a new insight to dean. Seeing this helps him understand dean's complicated relationship with their father, which was a relationship that sam himself never really understood (having a completely different relationship with their dad) and overall just greater understanding in why dean is the way he is.
On the complete opposite end of the spectrum, dean, passing through the three memories he sees in sam's heaven, sees only another reason to lose faith in his brother's innocence and cast him out. Sam's grand top three reliveable memories being some else's thanksgiving dinner, flagstaff and leaving for standford are three times he abandoned his family - which is the most unforgiveable thing a person can do in dean's perspective. Each example is seen as a betrayal, from running away for two weeks to simply leaving for university or having thankgiving dinner with another family. Although sam points out that he enjoyed this dinner because he was 11 and the winchester alternative was "a bucket of extra-crispy and dad passed out on the couch", this only further aggravates dean as, according to him, any suggestion of critique on their childhood is "bailing on your family" and, more specifically, a complete rejection of him as sam's family.
Thus, instead of giving any thought to what these memories suggest about sam or giving him any leeway at all, driven primiarily by their time restraint and his abandonment issues, dean immediately leaps to the implication that sam is fundamentally disloyal and doesn't care for dean the same way dean cares about him and is thus doomed to betray him to the side of darkness (read: dean's biggest fear). Which is a little bit ironic, considering if sam interpreted dean's memory in the same way, it would be that dean was happiest before he was born and thus the implication is that sam has ruined not only dean's life but has torn apart his entire family.
I don't blame dean for being hurt here or thinking this way, like I said, I think it offers interesting insight to his character, but as a viewer it is definitely frustrating to see dean's steadfast refusal to try and see things from sam's perspective - like sam says "I never got the crusts cut off my PB & J. I just don’t look at family the way you do."
How i saw it throughout the series is that dean's view of family is that he is only worth anything in the context of serving and protecting his family, whereas to sam, family is a prison that, although suffocates him, he still deeply craves the affection and stability of. This dichotomy in viewpoints and dean's stubborn failure to understand sam's perspective is cause for a lot of both sam and his own misery throughout the show as i saw it, and it's especially prominant in this episode. And it's why it's so frustrating for me because although I love the utilisation of genre to provide psychological insight into the characters and the ultimately messy and realistic way they react to it, I just feel like the pay off is never really reached.
Sam's departure to stanford has long been a source of friction in the winchester family dynamic since episode 1, and with the climatic circumstances of s5 forcing their trust in one another to be the only thing they can rely on (to trust that the other won't give in to fate and say yes to their respective archangles), I just feel like it would be a golden oppotunity in concluding this arc to resolve this issue. Having dean explicitly voice his understanding to sam about his decision to leave to stanford would not only represent a significant character development for dean, but a healed wound to their relationship. And also a perfect way to bookend this chapter of spn in my opinion.
Their relationship did take that much needed shift at the end of s5 where dean was forced to put his faith in sam's inherent goodness and fighting spirit to do the impossible and overthrow lucifer's control from inside his mind, just as his complete faith and devotion to his little brother is shown by his willingness to stick by him even whilst lucifer, controlling sam's body, ruthlessly beats him in the conclusion of swan song. But to me, although it demonstrated dean's committment even when sam has fallen to the hands fo evil, it still didn't quite reach the conclusion that dean would accept sam leaving the world of hunting all together. It was like dean could accept sam failing in his attempt to make the ultimate sacrifice to save the world (forfitting his mind and body to endless torment), but not sam... going to university like he always wanted. In all honesty, this step was never reached in their dynamic, metaphorically represented by the fact that dean never did get the amulet back in s5 or beyond, even just to have the viewer see him rest it on his night stand or something. Maybe this was because if dean ever accepted this, sam would be free to leave the monster hunting world which would be unsustainable for the show's set up, or maybe they just wanted this constant conflict. Or it was just forgotten about. But in the many seasons of spn I watched after s5, for all its many, many faults, this was the one piece that I thought was really missing.