Hear me out on this. Assuming the whole episode happened in Sherlock’s mind palace (which would make tons of sense with all the repeated lines from earlier episodes, the pool flashes, the weird tone) then everything we’re seeing is how Sherlock sees it, not necessarily how it really is. We know that Sherlock gets things wrong sometimes. Like in episode one, when he thought “Harry” was John’s brother, not John’s sister. He puts the clues together to reach logical assumptions, but isn’t always the best with emotions. So what if Sherlock caught a glimpse of John’s phone, and the texts to E? What if he made up everything about John having an affair to explain it? And everything about the affair being way out of character for John, because John loves Mary so much, Sherlock could’ve ignored that. Maybe he just didn’t understand how much John loved Mary. Maybe he wanted John to love Mary less so that maybe Sherlock himself would still have a chance. Those were probably subconscious, but they still could’ve influenced his deductions. Logically, if it was any man but John Watson, the texts to “E” could definitely have been an affair. Were probably an affair.