so i spent far too long on this.
for those unaware, the spaghetti wall of letters and numbers is a base64-encoded JPEG image (and not a URL as some guessed). in certain cases when you tried to insert/paste an image into whatâs ostensibly a text-only box, this could happen.
the thing thatâs bugging me however is that thereâs image data there. we have fairly a clear (albeit with JPEG artifacts) screenshot of text that, thanks to how Windows ClearType renders text, each character is identical to each other, that is to say, an uppercase Q will always look more or less pixel-perfect each time, meaning we donât have to guess what a Q looks like, we simply have to pixel-accurate match it.
as an aside, this is why regular OCR struggles so much with this kind of data retrieval, such as code even when itâs clearer than a physical paper scan. ordinarily, OCR will try to best-guess every single letter because it expects each letter to be slightly different from each other (as would be the unpredictable nature in a scanned document), and on top of that most OCR today will try to autocorrect because it expects the scanned text to contain words in some human written language.
so, all we have to do is make a program to recognize each character and piece back together the whole base64 string, right? wellâŚ
first i stitched all 7 images back into a single block of text, observing the consistency of the line spacing. some of the screenshots have little bits of the previous one sticking out of it, which helps with alignment and to make sure theyâre in the right order.
after that i had to sample every single letter off this file. this means going around the file and finding one example of each different character weâre trying to identify, saving it as its own separate file so that the program can load them as references to compare against in the full image. for base64, the alphabet consists of a-z, A-Z, 0-9, +, / and =. once i had the initial code in placeâŚ
âŚclose! but oh so far. if any one single character in a base64 string is wrong or missing, the resulting decode will be wrong. the issues i was having were mostly with the lowercase r and j because of how the kerning affected the pixels around those letters. i was also getting false matches for r where there should be an m. what followed was grueling hours of tweaking the matching code and my known font set to better fit the original image and get as close as possible to a 100% match. here is the resulting code, maybe itâll be useful for someone and this wonât have been a complete waste of time.
once i was confident through the verification image that i had all characters recognized, i put it through a base64 to JPEG decoder. i actually did this several times as i improved the recognition and what follows is the best result that came out of it yet. i suspect some of the data might be missing (perhaps a line or block of text got lost in between screenshots), or i have a wrong character somewhere resulting in a wrong value. this is the image extracted from OPâs base64 string:
we can finally know what they meant when they said âme in a relationshipâ and i can finally go the fuck to sleep.
