An interview with Team E3 at KRAFTON SP2 Batch 3
Deep learning is garnering attention as a core aspect of artificial intelligence (AI) technology. Here are developers who are now seeking to use deep learning technology to create fun for new games. KRAFTON is currently operating an incubation program called Special Project 2 (SP2). From Batch 1 in 2020 to Batch 3 today, a number of teams at KRAFTON studied deep learning and combined them with new game ideas to prove the core fun in a process that continues to date. In order to continue this process of taking on new challenges, KRAFTON is looking for new teams for Batch 4. We interviewed three members of Batch 3 Team E3 — Hyeonil Kwak, Seonggeun Kim, and Junyeop Lee — and asked them why they joined SP2 and what kind of experiences they’ve had.
Hello, nice to meet you! Would you introduce yourselves briefly for the readers?
Hyeonil Gwak (Gwak): Hello, I’m Hyeonil Gwak, the leader of SP2 Team E3.
Seonggeun Kim (Kim): I’m Seonggeun Kim, a member of Team E3. My main role is developing various mechanisms in the client using deep learning.
Junyeop Lee (Lee): I’m Junyeop Lee of Team E3. We’re all involved in the development process in our team. We tend to work in a parallel system where we set a specific direction for the team, then each develop a prototype based on our own ideas. In that regard, you could say that we each take on all kinds of tasks from game design to client development.
What kind of development experience have you had before joining SP2?
Kim: I majored in computer science at university. I spent about a year at my first job developing Java-based applications, then I entered into game development as a client programmer.
Lee: I also studied something programming-related in college, then I was hired for my first job. In fact, Kim, Kwak and I all joined the same company at the same time. While I was working as a client programmer, I saw and answered an in-house recruitment ad for a game AI-related group, where I studied game design for puzzle games applying reinforcement learning.
Kwak: My story is a little complicated. I first encountered deep learning in a research project while majoring in computer science at university. Then, later on, I felt that I wasn’t really suited to the major, so I began to look into writing instead. I was writing novels at the time. Eventually, I entered the same company as these two around the same period. I was hired as a game designer. I worked in game design and data analysis for a while, then I wanted to do something different, so I left the company after a year. This time, I worked in media art. More specifically, media art utilizing deep learning. I did that for about a year again, then I realized I liked video games more. That’s when I got into indie game development with Lee.
Kwak, you had so many different experiences in a diverse range of fields, but the keywords AI and deep learning seem to have come up a few times along the way. Am I right in thinking that your belief in deep learning technology and its potential led you to return to the field with SP2?
Kwak: Actually, when I first learned about deep learning as a student, I had very little faith in it. At the time, only about three out of ten professors could even tell me what deep learning is. There just wasn’t much information out there. However, the field grew exponentially within the past three to four years. It’s become mainstream enough to be used by art students like those I worked with when I was in media art, and utilizes a more diverse range of elements now. So, I began to gradually realize its potential and imagine that it could be well-suited for application in video games, too.
Lee and Kim, did you have similar experiences in deep learning and AI prior to joining SP2?
Lee: I studied computer vision while completing my undergraduate graduation project. It was to allow AI to recognize objects. Later, when I began to work in the field, I had the opportunity to experience reinforcement learning in a game AI-related department. I created AI for baduk, and automated puzzle game testing processes. That’s when I found myself gradually drawn to AI, which naturally led me to SP2, where I could work on both video games and AI.
Kim: I’m the polar opposite to Lee and Kwak, in that I had absolutely zero interest in deep learning before. I didn’t have any experience in the field, aside from a brief taste of it while I was studying machine learning as an undergraduate. But the important thing was that I like to test myself in new fields. I saw the SP2 recruitment notice and felt that it sounded interesting.
Kwak and Lee, you were working for a large company when you left and started your own indie game development company. That can’t have been easy. What kind of games did you make and what kind of challenges did you face?
Kwak: At the time, we were making a casual battle royale game, where little toy dolls ran around on top of huge objects. I’ve always told people that I want to make quirky, small-scale indie games. So, rather than thinking of it as launching a startup, I thought of it as the process of making the games that I want to make. I didn’t feel that it was tough to leave the company I was working for.
Lee: I had fun, too. When we were working for the same company, we even worked in the same department, where Kwak worked as a game designer and I worked as a client programmer. When we left the company and got into indie game development, I didn’t know anything about game design, but I started to analyze other games and imagine how we should make our own game, which was a more fun, self-directed way to do it. On the other hand, I did feel a little anxious because I wanted to see the outcome of our work as soon as possible.
How did Kim come to join Team E3? Compared to Kwak and Lee, Kim seems to have had a different perception of deep learning to begin with. I’d like to know how you brough him into the team.
Kwak: I kept half-jokingly trying to talk him into quitting his job and joining our team for about three years, but he never listened. He kept giving in to the allure of a steady paycheck. (Laughs) Having said that, judging by how he made indie games by himself, I can see that he has a stubborn side to him, too. One day, the topic of SP2 came up naturally in our conversation, only this time, he actually showed an interest. I think he was drawn to the fact that it was pioneering a new field. It’s certainly an attractive proposal from a developer’s standpoint, not to mention KRAFTON’s company-wide support for the project. Above all, being able to freely work on whatever we want in such a free environment is an unparalleled advantage.
Kim: It’s true. Kwak made various propositions before I joined Team E3. Even if he was half-joking, I was really grateful he asked. He never forgot about me and kept knocking on the door. I’ve known Kwak and Lee for a long time since we were all junior recruits and we’d become good friends, so I decided to accept his offer.
So, the three of you were hired for the same company at the same time, but Kwak now leads Team E3. Is there a reason for that?
Lee: Even when Kwak and I were developing indie games, Kwak was always the team leader. At least on my part, I thought that Kwak was a better fit to lead the team. There are bound to be differences in opinions in the game development process, but I felt that I could trust him in such cases.
Kim: First of all, I didn’t want the responsibility. (Laughs) Kwak’s always been better at thinking on his feet. Plus, he’s a better communicator than I am.
Speaking of communication, I imagine the three of you communicate very effectively, since you’ve known each other since long before you joined SP2. Could you tell us about Team E3’s decision-making and communication style?
Kim: All three of us have distinct personalities and like different things. We’re also quite stubborn. That’s why we think that any prototype that all three of us are satisfied with must be fun.
Lee: We speak frankly with each other. When people come together to make games, they’re sometimes overly cautious when they share negative feedback, but we don’t do that. In fact, we hit each other right in the gut. (Laughs) In fact, we try to review every idea from a negative standpoint first. Is this really based on deep learning? Is this really going to be fun? Sometimes, when we ask these questions at our meetings, it feels like we’re not making much progress. However, I think that’s a necessary evil.
Kwak: I believe that communication shouldn’t consist of a series of meaningless platitudes. In some ways, I think I lack that kind of respect and consideration. (Laughs) I usually try to be sharp with my criticism, but that’s not possible without an underlying level of solidarity. When we trust each other to have that kind of solidarity, consideration, and capacity for tolerance, that’s when we can share such sharp criticisms. Since our team already has these aspects, we can communicate directly with incisive criticisms.
Kim and Lee, do you often successfully persuade your team leader?
Lee: Not at all (Laughs)
Kwak: Hey, I say lots of nice things to you, too… (Laughs) Of course, it’s never easy for someone to change my mind.
I’d like to ask you about your project. Please tell us what kind of game you’re making and the current stage of the E3 project.
Kwak: We’re making a game that uses various physical interactions using webcams. Think of it as a virtual reality (VR) game that doesn’t require you to wear a headset. Webcam distribution has increased exponentially since the pandemic began and many apps allow smartphones to be used as webcams, which has opened the door for VR-like but distinct experiences without the need for expensive headsets. Currently, we’re looking into what kind of positive experiences we can create by combining webcams with various poses and hand signs.
Is there a reason why you focused on motion-tracking technology?
Kwak: We looked into various technologies and frameworks, while contemplating the kinds of experiences we can generate, but we felt that this is the best technology to start using immediately to create a video game product within the given timeframe.
Lee: There aren’t many games that utilize webcams and computer vision. In fact, there are hardly any, which meant that we didn’t have any reference points to study. That’s one of the reasons why we chose this technology.
Kwak: Today, teenagers and twentysomethings are already accustomed to cameras, including selfies and short-form videos, which are closely linked to webcams. We haven’t decided whether to only apply webcams in the game or to expand it into an online game that incorporates broader communication, but we felt that it was a highly marketable concept either way.
Please tell us more about the areas that you’re focusing on with regard to deep learning technology, as well as your daily routine.
Kwak: Among the currently available deep learning-based technologies that have been researched to date, we chose those that could allow us to develop the game to production level within the limited time and resources. Of course, we did our own R&D for about four to five months to start with. Then, we set our direction to quickly incorporate available technologies into our game. Lee and Kim are focusing on demonstrating the fun in our game, while I’m working on ways to organically approach players.
Kim: As we used deep learning to create the base mechanism that serves as the underlying skeleton of the game, we encountered unprecedented experiences that we’d never seen before. Now, we’re focusing on incorporating such experiences into our game in a fun way.
Lee: During the early R&D stage, we used to leave the learning model running overnight, check the results when we got home, check it again in the morning, then check it again when we came into work in the morning. Our routine was generating the data ourselves, processing it, teaching the AI, and reporting on the results. However, we felt that this method would make it too difficult to estimate development costs. So, these days, we build deep learning models for each function as needed, then combine them to incorporate them into our game.
Deep Learning seems like a field that requires constant learning. How do you each make sure to keep learning about the field as you advance the project?
Kwak: There was a time when I encountered a problem during the project and sought help by attending a lecture organized at KRAFTON’s Deep Learning Division. By talking to those who were taking on similar challenges with similar issues in other, non-game-related domains, I discovered clues to resolving my own problems.
Kim: I’m focusing on a mechanism that utilizes a lot of hand movements. Since hand gestures are closely linked to fatigue in terms of ordinary postures and gestures that human beings make, as well as our individual experiences, I’m often contemplating the subject of human interactions. That’s why I’m often reading up on academic papers on the topic. In some ways, it might not seem like an area that has much to do with video games, but our project requires us to consider these things in order to create positive gaming experiences.
Lee: When we first thought about using webcams, I studied eye-tracking technology with the idea that we could make something fun by using players’ eyes to predict their point of sight. Ultimately, I came to the conclusion that it would be too difficult to use in practice, but I still like to study certain areas that I become interested in while conducting the project. I’m aiming to create better experiences by learning more about one area after another as we advance the project.
Some say that deep learning is an area that remains somewhat obscure and not clearly defined. Incorporating such an underexplored element in a new game to generate fun seems like a striking challenge. What kind of mindset or personality do you think you need to take on such a challenge?
Kim: Usually, when we develop video games, all of the algorithms stem from the developer’s mind. However, in deep-learning-based development, the deep learning model creates the algorithms by itself. As such, I think it’s important to have a creative mindset.
Lee: I’m quite an anxious type of person. But, when we’re using deep learning to develop games, it’s difficult to guess whether something might work or not. It’s only when you try it in practice that you find out. This kind of working method can often create a sense of pressure. Having said that, even if the deep learning technology doesn’t yield results that precisely match our expectations, I would like us to maintain a positive mindset that can actually turn these mistakes into a source of fun for our game.
Kwak: I think the most important thing is to have flexible convictions. Of course, it goes without saying that we need to believe that we can create something using deep learning, in line with SP2’s overarching aim. However, we each have different ways of realizing that ambition. Each team at SP2 has its own direction and beliefs, but I feel that it’s dangerous to try and push ahead with our own beliefs no matter what. Instead, we should try to be flexible and explore more possibilities.
Furthermore, I think it’s important to work efficiently. We’re not the absolute authority on deep learning, so we need to consult as many experts as possible.
Are there times when you feel especially grateful that you joined SP2?
Lee: I feel that the experience we’re creating right now wouldn’t be possible anywhere else. This is the only place where each team was given overall autonomy. We set our own milestones and make adjustments ourselves at every stage. Obviously, that means we also take responsibility for the outcome ourselves.
Kim: I feel exactly the same as Lee. I love that our team is given full autonomy.
Kwak: On the other hand, I think there are some people who are suited for this kind of autonomous system, as well as those who aren’t. I remember watching a promo video before joining SP2. In it, KRAFTON Chairman BG Chang commented, “We’re looking for people who like to learn and grow, while being able to enjoy the process of exploring new ways of having fun.” I think that’s also a crucial part.
When you play video games, what kind of experiences do you enjoy the most?
Kwak: I always call it a “pointed” feeling. A game with a point. When we talk about creativity, I think something’s only truly creative when it has a point to make within a frame that can be universally recognized and empathized with. In this respect, I enjoy games that showcase some new, original point inside a universal frame.
Kim: I have the most fun with games whose base mechanisms are solid and well-implemented.
Lee: Actually, I don’t really have such clear-cut definitions. But I think a game is good enough when you play it and think of someone that you want to play it with.
In SP2’s Batch 1 and Batch 2, there are several teams that began their projects before Team E3. What were some lessons you learned by listening to the previous trials and errors of your colleagues from the other teams?
Lee: I think that we can derive a sense of our path ahead based on the prior experiences of the challenges taken on by the teams in Batch 1 and Batch 2. In return, I hope that the other teams find success in the end. No matter what kind of outcome they produce, I’m sure it will help us all.
The upcoming Batch 4 will allow new teams to take on new challenges. Do you have any advice for your incoming colleagues?
Kwak: Firstly, I hope that Batch 4 is joined by those who enjoy adventures, challenges, and growth. I also hope that they’re good communicators. Although there are numerous communication tools available, talking face to face is ultimately a hugely helpful method. At least, it was for us. In addition, I would like them to set a reasonable expectation for the scale of their challenge within the given timeframe and a goal for a deadline to demonstrate their ideas. Of course, predicting all of those things is much easier said than done, but thinking about them in advance will make it easier to choose or research technologies to incorporate into your games, as well as to live with your failures. Either way, you’re going to end up making plenty of mistakes… (Laughs)
Final question. I was told that you were asked what your favorite games were during your interviews when you first joined SP2. What were your answers?
Kim: Mine were “Dark Souls” and “GTA.”
Kwak: I think I said I like hack-and-slash games…
Lee: I like “Overcooked.”
Kwak: Everyone gave a different answer. I guess that shows you how different we all are. (Laughs)
