RC11

Here I extracted all the window information from images of Cologne Cathedral and used it to build a “structural library” of architectural details.

This pipeline can also be applied to video, detecting and annotating each frame. For now I only trained the “window” class, so the video demo is just window detection as a functionality test.

I trained a custom YOLO model on about 50 manually annotated images, focusing specifically on architectural details. Then I ran the model on the entire image set, but only kept a few key classes, such as window, arch, and spire. This allows complex urban or architectural scenes to be reduced into a clear “structural map” of essential elements, which is useful for further analysis and visualization.

I built a small website that uses color to analyze the emotion of an image. You can upload any picture, and the system will extract the main colors and infer emotions from their warmth, brightness, and saturation. In the end, the analysis is turned into a simple 3D emotional form, so the mood of the image can be “seen” as colored shapes in space.

By mapping motion to temperature in this way, the color field becomes much more intuitive: even without thinking about RGB values, you can immediately sense where the image is “warm” or “cold,” and watch the overall color atmosphere of Cologne Cathedral come alive in motion.

Finally, I extracted all the pixel colors from the photos and fed them into an SOM model, letting it “line up” the colors by itself in color space. After training, I got a color map / palette generated by the SOM. It condenses all the color information from the photos and turns the Cologne Cathedral in people’s camera lenses into a unique color fingerprint.

I then created two visual charts to illustrate how emotional scores are distributed and how they relate to architectural components. The first scatter plot shows that most comments fall between 7–10, indicating strong positive public reactions (higher score = more positive emotion). The second plot maps emotion scores to architectural elements, revealing that facades, light rays, and vault ceilings tend to trigger higher emotional responses, while war ruins and broken walls are associated with lower emotional value

I collected user comments about Cologne Cathedral from Instagram and generated an emotion_score for each comment (higher values indicate more positive emotion). Expressions such as “Just stunning!”, frequently reach 8–10, showing intense positive reactions. Based on sentiment grouping, I created an “Architectural Components × Emotion” matrix: for example, the cathedral façade strongly correlates with peace and sacred, while war ruins and broken walls more often evoke sadness. These findings indicate that emotional perception is anchored to specific architectural details rather than the overall form, providing a quantifiable foundation for understanding emotion-driven heritage experience.

I processed these images using SAM to segment architectural details, and YOLO to identify frequently appearing components—such as stained glass, spires, arches, and interior columns. By transforming emotional perception into a computable visual structure, I am building a “Emotion × Architectural Components” matrix, hoping to explore how emotions shape heritage experience in contemporary cities.