The Crack Theorist

Before diving in, a bit about myself and this project to preface: I’m a data scientist with three years of industry experience and a big fan of Digimon Survive since its release, in which I’ve played both the JP and EN versions (you probably also know me as Shuuji’s passionate defense attorney lol). Emphasis on industry–my academic background is actually in a non-IT engineering field, and I have limited experience in data science applications in a formal academic setting. This means I tend to focus more on practical implications than on academic rigor (you can probably see what I mean by this as you read on). That said, this project is a labor of love aimed at understanding and celebrating a game that’s very close to my heart, so I did the best that I could with the data I have.

1. Data Collection and Preprocessing

Seeing the wealth of quality reviews available on Steam, I decided to use it as my data source for this project. I used a modified version of the Steam Review Scraper to gather the review data. Out of the 3,019 reviews available on the site (as of today), I scraped 1,533 reviews–all of which are in English–which accounts for 50.8% of the total. I focus on English reviews for the time being because it’s easier to clean and analyze English reviews with the NLP libraries currently available on Python (for those who aren’t familiar with it, Python is a programming language popular among data scientists, and it’s what I used for this project). Here’s a snapshot of the raw data to give you an idea of what it’s like:

This dataset isn’t quite ready for analysis though–we need to preprocess and clean it first to ensure more effective and accurate analysis. Here’s an example of what the data looks like after going through the general data cleaning (it’ll need more data cleaning later, but for now this should suffice as a first step):

Something to note here is that I keep the “helpfulness” dimension (column) as a way to weight the reviews (i.e., reviews that more people find helpful hold more weight than those that aren’t). I think this is a fair thing to do since I view the “helpfulness” score as people agreeing with the opinion voiced in the review.

2. Exploratory Data Analysis (EDA)

2.1. Sentiment Breakdown

What is the general sentiment surrounding the game? To understand that, I simply mapped the reviews based on whether or not the reviewers recommended the game (i.e., “Recommended” = positive, “Not Recommended” = negative), because I think it’s a pretty apt thing to do rather than running a whole new sentiment analysis. And here’s what we got:

Positive reviews: 1,257 (82.0%)

Negative reviews: 276 (18.0%)

Based on the data, we can see that the majority of people have a positive sentiment about the game, with only a relatively small minority viewing it unfavorably.

2.2. Word Counts

Now, let’s look at the frequency of words in both positive and negative reviews.

Interestingly, the top words based solely on their frequency are similar between the two categories. While it provides a preliminary picture, it’s hard to tell what truly makes the game good for the people who like it and bad for the people who don’t, so we need more nuanced analyses for better understanding.

3. Latent Dirichlet Allocation (LDA)

Next, I employed Latent Dirichlet Allocation (LDA), which is a popular technique used for topic modelling that can discover hidden themes in texts and summarize them into topics. In the case of this project, using LDA can help us find out what are some of the most frequent talking points that players discuss in their reviews. For those curious of how it works, this diagram I snatched from this article explains how LDA works in a simple way:

Because this post was initially done in the form of a Twitter thread, I decided to go with 4 n-components (number of topics) to be identified each in positive and negative review topics (*do note that this might not be the best number to choose for the n-components). Also, the results are going to be presented in the form of word clouds.

3.1. Positive Review Topics

Topic 1 seems to focus mostly on game mechanics like “battle” and “choice”. Despite the generally favorable reviews of Survive, even the people who like it still acknowledge the gameplay as an issue for the game. There's a recurring comparison to another Digimon game, Cyber Sleuth, where Survive's mechanics are probably seen as a notable point of comparison.

Topic 2 is centered around how the game design ties to the narrative design. Specifically, players seem to love the different endings and evolutions, where the branching storylines and multiple outcomes keep players engaged and invested in their choices.

Topic 3 is the most frequently mentioned among all the topics and it centers around the characters and story. The characters and storyline seem to resonate deeply with fans, presumably especially true for long-time fans of the anime series. This topic shows that the emotional connection to the story and its characters is a significant factor in the positive reception of the game.

Topic 4, much like Topic 1, is largely about the gameplay, specifically about the combat. It seems that opinions are divided–despite the criticisms surrounding the actual (SRPG) gameplay part of Survive, there are some who do enjoy it for what it is. This makes the gameplay a polarizing aspect to Survive that’s both a positive experience and an area needing improvement.

3.2. Negative Review Topics

Topic 1 of the negative reviews–in contrast to the first topic uncovered from the positive reviews–seem to center around character enjoyment instead. Despite gameplay issues (mainly the combat system) hindering the overall enjoyment of the game, even the critics of the game still appreciate its characters and acknowledge it as one of the game’s strongest suits. However, translation problems in the game are also noted, which affect the immersion and enjoyment of the game.

Topic 2 seems to mostly come from people who are disappointed about Survive being largely a visual novel. Some players don’t like the heavy focus on story and dialogues, wishing for more battle elements. This expectation mismatch leads to dissatisfaction.

Echoing the first topic, Topic 3 is also about gameplay critique. There seems to be significant frustration over the game’s mechanics. Terms like “anime” and “cutscene” likely refer to the complaints about the game freezing after the animated cutscene that plays at the beginning of the game, which has now actually been fixed. There are also mentions of “price”, probably from people who think that the price point of the game isn’t worth it considering the lack of proper gameplay and the aforementioned bug. 

Similar to the positive reviews, comparison to Cyber Sleuth is also a recurring theme in the negative reviews, as shown in Topic 4 (which is also the most frequent topic in the negative reviews). It seems that some recommend Cyber Sleuth over Survive for a better gameplay experience when it comes to Digimon games.

3.3. Key Takeaway from LDA Results

We found out that most people agree that Survive’s strength lies on its story and characters. Meanwhile, its gameplay mechanics receive mixed reviews with most finding it not executed as well as its narrative design.

4. Text Network Analysis (TNA)

Ever thought about how words in reviews connect with each other? Text Network Analysis (TNA) can help us analyze and visualize these connections. Simply put, TNA views text as a network of interconnected words, revealing how these words are related based on their co-occurrence in the reviews. Aside from that, we can also measure stuff like centrality and detect communities. What are these, anyway? To avoid confusion, let me briefly explain these terms first before proceeding into the analysis:

Betweenness centrality measures how often a node (a word) appears on the shortest paths between other nodes (i.e. other words). In the context of TNA, it indicates the influence of a word in connecting different parts of the network–a high betweenness centrality means that the word is crucial in linking various concepts within the reviews.

Eigenvector centrality measures the influence of a node (a word) based on the centrality of its neighbors, i.e., a word is considered important if it is connected to other important words. High eigenvector centrality means the word is connected to many other influential words, so words with high eigenvector centrality are central to the review network and play a significant role in shaping the overall sentiment and themes.

Closeness centrality measures how close a node is to all other nodes in the network, which is calculated as the reciprocal of the average shortest path length to all other nodes. High closeness centrality means the word is well-positioned to quickly interact with many other words. The implication that comes with this is that words with high closeness centrality are pivotal in spreading information throughout the network, meaning they are central to the overall discussion.

Before I explain what the Louvain community is, let me explain first about the concept of community and community detection. In the context of TNA, communities are groups of words that are more closely related to each other that we can think of as a group/cluster (i.e., community). Community detection is the process of identifying these groups within a network of words. One of the techniques we can use is Louvain community detection. Using this helps us understand different themes in the reviews.

Now with that out of the way, let’s move onto the actual analysis. We’ll start first with the words co-occurrence where we take the top 100 pairs of words each from positive and negative reviews, and then move onto the centrality and community analysis. As a note, for the words co-occurrence graphs, the size of the nodes indicates how often the respective word is mentioned in the reviews, while the thickness of the edges represents how strongly connected a certain pair of words is.

4.1. Positive Reviews

4.1.1. Words Co-occurrence in Positive Reviews

In the positive reviews, the terms “story”, “visual”, “novel”, and “character” are among the most frequent words. They also make up prominent pairs like “visual-novel” and combinations involving “story” with positive adjectives (e.g., “like”, “good”, “great”, “love”, “enjoy”). This shows that the story is a central aspect of enjoyment for players. “Story” also connects strongly with “character”, indicating how the characters are a pivotal part of what makes the story good.

4.1.2. Centrality and Community Detection in Positive Reviews

Centrality Measures: We see that the word “story” holds the highest values across betweenness, eigenvector, and closeness centrality, which makes it the central theme in the positive reception of Survive. Other high centrality words like "visual-novel" and "character" also support this finding. The key takeaway from this is that the narrative elements are what people like the most from the game, aligning with what we previously saw from the LDA analysis.

Louvain Communities:

Community 0 (“visual”, “novel”, “combat”, “great”, “gameplay”, “far”, “love”, “recommend”, “rpg”, “enjoy”, and “tactical”) seems to revolve around story and gameplay integration, recommending Survive to fans of visual novels and tactical RPGs.

Community 1 (“like”, “good”, “play”, “character”, “time”, “fan”, “battle”, and “first”) likely represents the opinions of first-time players who appreciate good characters (as fans of the Digimon anime, perhaps?)

Community 2 (“story”, “take”, “fun”, “give”, “look”, “choice”, “different”, “dark”, “experience”, and “world”) highlights the dark story and world, also how the different choices can lead to varied experiences. This suggests that the players who like it welcome the dark twist in the story, and enjoy how their choices can shape their experience in the game.

4.2. Negative Reviews

4.2.1. Words Co-occurrence in Negative Reviews

As we can see, the frequencies of words are more balanced in the negative reviews than in the positive reviews, although terms like “story”, “visual”, and “novel” still pop up more than others. For the popular pairs, “visual”-“novel” tops, followed by “like”-“story”, “like”-“play”, and “like”-“novel”. We can infer from this finding that most negative reviews seem to talk about how this game is only recommended for those who enjoy visual novels and story-heavy games, and not for people who actually look for an actual, proper gameplay.

4.2.2. Centrality and Community Detection in Positive Reviews

Centrality Measures: The words with the highest values across different centrality measures are “like”, “story”, “visual”, “play”, and “novel”. We can quickly see how this most likely means that the game’s critics acknowledge that this game is only meant for people who like visual novels and story-heavy games, aligning with our previous findings.

Louvain Communities:

Community 0 (“cyber” and “sleuth”) very likely comes from people who enjoy Cyber Sleuth a lot more and keep making comparisons between the two games.

Community 1 (“like”, “actually”, “recommend”, “still”, “keep”, “dialogue”, “look”, “take”, “free”, “try”, “read”, “boring”, and “first”) represents the general opinion that Survive is for people who like story-heavy games, as it has a lot of dialogues and requires you to read a lot, otherwise you’ll find it boring.

Community 2 (“story”, “price”, “start”, “choice”, “give”, “end”, and “love”) is similar to Community 1 in that it also mainly talks about how Survive is a game for people who love story-heavy games. It also perhaps talks about how the price isn’t justifiable considering how a large part of it only revolves around reading the story with minimal proper gameplay.

Community 3 (“visual”, “play”, “novel”, “character”, “time”, “battle”, “good”, “hour”, “bad”, “gameplay”, “combat”, “point”, and “rpg”), again, essentially repeats the points of Communities 1 and 2–it’s a game for visual novel enjoyers and not for those looking for an engaging RPG. It probably highlights how the visual novel portion is the good part of the game, while the RPG portion is–simply put–bad.

4.3. Key Takeaway from TNA Results

Both positive and negative reviews frequently bring up the game’s narrative elements, which shows that this is a polarizing aspect of Survive. Positive reviews praise the game for its rich, story-heavy content. On the other hand, the negative reviews criticize the heavy focus on narrative, perceiving it as boring. It shows that Survive is mostly meant for people who appreciate a game more for its story than gameplay. This finding aligns with our previous finding in LDA.

5. Conclusion and Final Thoughts

So, what's the final takeaway? Overall, Digimon Survive is loved for its compelling story and characters, perhaps particularly by fans of the Digimon anime. However, even most positive reviews of the game acknowledge that the gameplay could use some areas of improvement. I personally agree with this general sentiment, even as someone who likes Survive for what it is–improved combat elements could’ve provided a more balanced experience that could elevate Survive to be an even better game.

Comparisons with Cyber Sleuth pop up a lot in both positive and negative reviews alike, though I observe they recur more often in negative ones. Cyber Sleuth is most likely brought up as an alternative for people who don’t like Survive–if you don’t enjoy visual novels, then you’re better off just skip Survive and try Cyber Sleuth instead.