The reason villainous characters often get redemption arcs is that sooner or later, to justfy them continuing to be present in the story, rather than killed, driven off or just, leaving (or thrown in prison), they need to stop being evil. Eventually, the bad guy has to get beaten in some form, or else the show gets insanely repetitive, and the continued failure of the good guys to win undermines the show. (Except in formulaic shows that always end with 'I'll get you next time <hero name> or something like that, which is mostly relegated to comedy these days)
Some shows can get away with having a character of evil morals hanging with the main characters all the time without becoming good - depends on the show, the nature of the characters, the specific moral dimensions of the show (what is/are the central moral axes and questions the show asks and presents, et cetera) but many of them... they just can't. The evil person has gotta be less evil, at least in practice, if they're gonna stick around.
There's a lot of reasons redemption arcs happen - from the power fantasy of being able to get evil to come to your side, from showcasing how anyone can come back from evil if they want it, if they show remorse, if they try to make up for their sins, etc.
But there's also just the sheer show logic that if you want to keep that popular villainous character on the show as a regular or near-regular, he/she/they/it has to be redeemed at some point, or the character will get stale, boring and turn off much of the GA.
