Junqiu - Momo

01 People's action

I used YOLO pose-detection to extract a large number of people fragments from street view videos. Each fragment retains metadata such as street_prediction_id, war_prediction_id, tile location, confidence level, and original frame information.

02 Match War map

skeleton--pose vector--match

For each street-view skeleton, the script compares its pose vector with the prepared War pose vectors and finds the nearest match based on pose distance.

pose vector = normalized keypoints + visibility/confidence + movement + pose category + behaviour category + bbox scale

Web: Prepared War SOM Matching

03 Segment

04 3D and combination

Car Parking Site

Base Modules

Som Reflection

Floating Space

17_base_pavilion_real_som_overlay_filtered.html

Art

Urban_Image_Spatial_Database

1. Urban video input layer: Urban Image Source

2. YOLO detection layer: Object Detection

05_label_translation_dictionary.csv

04_design_trigger_database.csv

3. Spatial Grid Layer: 20 × 12 Video-Frame Grid

Each cell represents a spatial location within the video frame.

When YOLO detects an object, the code calculates the centre point of the bounding box:

cx = (x1 + x2) / 2

cy = (y1 + y2) / 2

It then determines which grid cell this centre point falls within.

4. Grid Cell Data Aggregation Layer: Spatial Evidence Aggregation

12_interactive_cell_spatial_database_METAPHOR_EN.html

Number of objects: 389

Number of frames: 318

Dominant category: Buildings

Across all video frames, there are 389 objects detected by YOLO within this cell; these detections are distributed across 318 frames; the most frequent and consistent category is ‘Buildings’.

5. Database Semantic Scoring Layer: Database Scoring

Correlates the YOLO results for each cell with the database.

Art / Fluidity

Theology / Threshold

Language / Metaphor

Law / Order Reference

War / Social Gathering

Calculating the score

database_score = object_density × class_to_theme_weight × semantic_similarity × spatial_context_weight

Art / Fluidity: 0.0186

Theology / Threshold: 0.4856

Language / Metaphor: 0.0673

Law / Order Reference: 0.1394

Social / Gathering: 0.0024

6. Spatial Module Translation Layer: Spatial Module Translation

Converts each cell into a spatial module based on its dominant category and dominant database.

7. Blender Parameter Generation Layer: Geometry Parameters

Rows / Columns→ X / Y positions

Module Type→ Geometric Type

Number of Objects→ Scale / Dimensions

Database Strength→ Height / Thickness / Elevation

Dominant Force→ Morphological Behaviour

Adjacent Cells→ Rotation / Continuity / Direction

Art, Language, Myth

three_modal_geometric_art_som_compact_interactive.html

Language

We choose this site because it sits on the edge of Rotterdam city centre, between housing, water, roads and public green space. It is not a fully defined park, but an open transitional urban space. This condition allows the project to test different forms of movement, occupation, viewing and interaction. The site therefore becomes a suitable field for recombining our order, conflict and language-based modules into a three-dimensional spatial system.

The square tile was divided through a fixed geometric rule, using mainly 45-degree and 135-degree cuts, together with some vertical and horizontal divisions.

1 Vertical arrangement

2 Free arrangement

Choose 12 boxes

Segment

The cutting rule was based on diagonal divisions, vertical divisions, and a proportional structure. These fragments could then be rotated, mirrored, and recombined while still keeping a consistent angular order.

1

The first recomposition test used a random growth logic. Each fragment was placed one by one, starting from a seed piece. The fragments were allowed to rotate by 0, 90, 180, or 270 degrees, and they could also be mirrored, but they were not allowed to overlap.

2

The fragments are encouraged to align with the hidden vertical structure, but they can still extend beyond it. The left image shows the final composition, while the right image shows the hidden alignment logic. The blue vertical lines are not part of the final drawing. They are only used as an invisible framework to guide the placement of fragments.

As the video moves through time, the corresponding state is activated on the drawing.

Choose

The selected tiles are a procedural base, from which routes, boundaries, waiting areas, and activity nodes can be extracted.

Meaning of layers

Combination： Law plan

High or low indicates the priority of this layer in spatial translation.

Site: Het Park

1. Rotterdam’s largest national green heritage site (21 hectares), designed in 1852 by Zocher & Sons in the English landscape style, featuring undulating terrain and a varied natural environment.

2. The 1960 Floriade World Horticultural Exposition was held here – a historical precedent for temporary pavilions, which informs the narrative of constructing a ‘play pavilion’ for each of the subsequent 12 themes.

3. Situated adjacent to the River Maas, the dense urban fabric of the Scheepvaartkwartier, and the Euromast landmark.

SOM of Art and Myth

From each of the 12 clusters, a representative God was selected. For each cluster, a theme was derived from the keywords and the deity’s narrative text.

For example, the keywords for cluster_04 are Fertility, abundance, sacrifice, negotiation and love, and the corresponding representative deities are Nemesis, Cupid, Eris and Metaneira.

Nemesis — Nemesis / Goddess of Retribution

Cupid — Cupid / God of Love

Eris — Eris / Goddess of Discord

Metaneira — Metaneira (a devotee of Demeter in Greek mythology)

Art fragments and metaphors were then matched to these themes. The corresponding lines of poetry containing metaphors are: ‘Cooling us with Shafts of Granite is like cooling thoughts.’ | ‘For his judgment, lo, a pudden!’ |'The thrush on the oaktop in the lane sang his last song, or last but one; and as he ended another had just begun. They knew no more than I the day was done.'

Definition of a metaphor (language → fusion operator for each step):

When a path spans from symbol A to B, locate the metaphor in the (literal → metaphor) pairings extracted from 1,000 poems that most closely match (A, B), and use its type selection operator: strong equivalence ‘A is B’ → intersect, attribute transfer → Union, weak association → Subtract.

2. Processing the theological database to convert the ‘Divine Character Cards CSV’ into structured data suitable for plotting narrative arcs.

Each deity → their multiple narrative actions → each action corresponds to a theme, intensity and text vector.

Read out each deity from the theological CSV, extract their narrative actions, break the actions down into individual ‘narrative beats’, and then assign the following to each beat:

theme category

intensity narrative tension intensity

text original action text

vec action text vector

three_modal_art_language_theology_matching.csv

01 Data + Colour Logic

02 Pose to Geometry Mapping

03 Pose + Spatial Organization

04 Relational / Graphic Language

05 Map

01 War Pose Dataset

I first sampled the war videos every 2seconds, transforming continuous footage into a sequence of analyzable action samples. Then human pose detection was applied to each sampled frame to extract body key points and skeleton-based pose data. Key points were converted into pose features, including movement, arm movement, visibility, body scale, and pose direction. Each action sample became a pose feature vector / pose embedding.

02 Som

These pose embeddings were used to train a SOM, so similar bodily actions were organized into nearby positions. Each sample then received a som_x / som_y coordinate.

03 Data to Geometry Mapping

Each pose sample is placed on the SOM map according to som_x / som_y. Similar bodily actions are organized into nearby areas, so the overall layout comes from the data structure of the SOM.

04 Graphic Language

The body is no longer drawn as a complete skeleton. Instead, it is translated into black fragments.

05 Map