You log into PUBG: BATTLEGROUNDS. Unfortunately, all your friends you usually play with are offline. You still don’t feel like playing in a random-matched squad. You wish for a friend who understands your play and listens to your briefing and orders. KRAFTON is studying the deep learning technologies aiming at creating an artificial intelligence (AI) it calls the “Virtual Friend.” Read onto the following interview with Kangwook Lee and Hyeongjin Lim of Deep Learning Div. and find out the reason why KRAFTON concentrates on deep learning among other AI-related technologies.
Nice to meet you! Could you introduce yourself to our readers?
Kangwook Lee (Kangwook): Hi. I’m Kangwook Lee, the head of Deep Learning Div.
Hyeongjin Lim (Hyeongjin): I’m Hyeongjin Lim of Deep Learning PM Team (DL PM Team). I used to be a developer before joining KRAFTON. I’m now a project manager (PM).
AI and deep learning contain diverse subsectors. In which field do you have your expertise?
Kangwook: While I’m working as the head of Deep Learning Div. in KRAFTON, I’m also a professor of electrical and computer engineering department. I’ve been focusing on mathematical theories about machine learning and deep learning at the university. And I’ve been looking into more practical deep learning project since I joined KRAFTON. While working on such projects, I could find some intersections between the theories I’ve studied and the applied technologies that KRAFTON wants. I’m trying to help KRAFTON develop advanced deep learning technologies, tapping into my theoretical expertise.
Speaking of PUBG: BATTLEGROUNDS, do you play game often? Are you good at it?
Kangwook: Honestly, I’ve been playing “StarCraft” a lot, long before I started playing PUBG: BATTLEGROUNDS. I was and still is a huge fan of games. But the only game I can say I’m actually good at is StarCraft. I still play it. I couldn’t get a professional player certificate for StarCraft and ended up as an amateur. For “StarCraft II,” on the other hand, I used to be a pro player in the U.S. for about half a year. Now I come to think of it, I’ve won an amateur StarCraft competition. (Laughter)
There are various kinds of PMs with diverse roles in a game company. What’s your role as a PM in Deep Learning Div.?
Hyeongjin: Deep Learning Div. works on various research projects in a wide range of subjects, so as a project manager, my role to oversee if the human resources are distributed properly according to the subjects. In short, my job is to determine if the projects are well-balanced in terms of the range of subject, manpower and time. Most of the people in Deep Learning Div., except for those in my team, are researchers. Among them are who joined KRAFTON right out of graduate school or who had been working in other companies. The DL PM Team works on filling up the “gray zone” that occurs while the researchers concentrate on their respective field of expertise.
You sound like an all-rounded player although what you do seems not easy at all. In order to keep up with what’s going on in all those AI and deep learning-related projects as a PM, you must be studying very hard. Did you study or experience related stuffs before joining KRAFTON?
Hyeongjin: I majored in statistics. I studied machine learning briefly in my last semester at the university. Although it wasn’t enough to jump into the professional level, the experience motivated me to work in the machine learning sector. So, I’m continuing to study it while I work.
Could you tell me more about Deep Learning Div. as an organization and about the people working in it?
Kangwook: Deep Learning Div. largely consists of Deep Learning Dept. and Project Beluga Dept. Hwaeium Yeom is leading Deep Learning Dept., and I’m engaging more in this department. It contains four teams – DL Research Team, Applied DL Team, DL Ops Team and DL PM Team. DL Research Team works on general and basic research while Applied DL Team study and develop technologies that are required to realize new services. We need DL Ops Team to upload what we make to the server and provide it as a service. Of course, we have DL PM Team, led by Hyeongjin here. Recently, a new methodology called “Data-centric AI,” which diverges from the traditional model-centric views and approaches to deep learning and AI from the data-centric perspective, is on the rise. We’re preparing to organize a new team, Data-centric Deep Learning Team, to keep up with this significant new method.
At Deep Learning Div., we’re working with colleagues with various backgrounds. Most of new members are who recently completed their master’s or doctorate programs in deep learning. They have done various deep learning research projects in their postgraduate courses, accumulating experiences and theoretical knowledge in this sector. On the other hand, most of the senior members have experienced deep learning while they were working in the industry. It’s because only few universities taught deep learning at that time. They have abundant practical experiences in deep learning as well as a wide range of knowledge in computer engineering and statistics in general. These people with a variety of backgrounds are harmoniously working together, actively engaging in research projects.
Before deeply diving into what Deep Learning Div. does, could you brief us about what “deep learning” is?
Kangwook: The highest concept here is AI, which you might have already heard quite often. It’s actually a very old concept, but it was in the 1950s when the researchers started to study and materialize the concept with the invention of the computing machines. Since then, there were a plenty of attempts to realize AI, but none of them worked. Then in the 1980s, a new method called “machine learning” came into the limelight. The researchers, however, ran up against the limits all over again. In the mid-2000s, what we call deep learning, a new groundbreaking technology that has stemmed from the machine learning technology, was developed. And since then, deep learning is towing the advancement of AI technologies until now.
I think there’s no methodological limit on how an AI is realized. All kinds of methods that leverage data to “learn” is included in machine learning. On the contrary, if we can realize an AI without leveraging data to learn, we can call it a non-machine learning AI. Among the machine learning algorithms, there are some that we can classify as deep learning depending on the way they leverage data. If an algorithm is designed to learn data based on what is called the “deep neural network,” it can be considered as one of the deep learning algorithms.
According to what you said, deep learning is a subordinate concept of machine learning which is one of various elements of AI. Then, why did KRAFTON choose deep learning to name the organization that works on AI?
Kangwook: Machine learning and deep learning has a big difference. It’s difficult to apply other machine learning methods except deep learning to various media. One example that came into the public spotlight between the late 1990s and the early 2000s was the technology used to detect junk mails. Another example was the ones used for search engines. They are based on machine learning methods. But they were used to search letters or analyze very simple patterns at most.
Before deep learning, one of the biggest problems that many machine learning researchers faced was to get language, speech and image data recognized. What made the problem even more complex was that there were separate subordinate fields of study for each sector. Thankfully, as deep learning was created and developed fast, it became possible to tear down the borders between these sectors and solve the problem at once. This happened only in the last decade. Before that, there were computer-based studies of image or speech that don’t use machine learning methods. Of course, there were studies on languages, too. But now, almost all of them are increasingly getting involved in machine learning or deep learning. These areas that have long been separated from each other are now merged with various media, thanks to deep learning.
This leads to the goal of KRAFTON’s Deep Learning Div. KRAFTON’s trying hard to achieve the ultimate goal of realizing what we call the “Virtual Friend.” A Virtual Friend should have a face and a body, of course, but it also should be able to talk fluently and understand human languages. Since it should have different facial expressions and body postures as it speaks, we also need imaging technologies to make them possible. We need to be fully capable of handling image, speech, and language technologies. If KRAFTON had tried this kind of project a decade ago, it would have needed three different divisions. Not to mention the hideous expense, it would have been almost impossible to achieve the goals in each sector at that time. Now, as the deep learning technology advances fast, we have a key to cover them all at once.
In a previous interview, we talked to those who are working on the virtual human project at KRAFTON’s Creative Center. Whereas Creative Center focuses on making the exterior of virtual human, Deep Learning Div. seems to be researching and developing the internal functions of it, such as listening, watching, speaking, and reading, am I right?
Kangwook: You’re right. Deep learning researchers are very active to open and share what they have discovered. They are opening codes, research results, and even data, accelerating the progress of development. I believe this will result in a gradual diminution of commercial values of making a new deep learning technology faster and of making better methods than others. What’s far more important than that is to be able to tap into the universalized deep learning technologies to quickly grasp what we can do with them, and to quickly come up with further technologies required for our goal.
Considering our goal of creating a virtual human, I believe KRAFTON, a game company, is better prepared than many non-game businesses. No matter how good one can be in handling language, speech, and image deep learning technologies, it’s not easy to create appealing exterior and character, and to produce contents and platform to provide as a service. KRAFTON already has experiences in platforms, and in creating characters to make them as contents. That’s why I think game companies like us are in a more favorable spot to create a virtual human.
Will the Virtual Friend that Deep Learning Div. is working on be more likely to serve for specific functions, not like an artificial general intelligence?
Kangwook: Although it’s hard to distinct them clearly, I think our Virtual Friend is more likely to be an AI with a special purpose. We’ll need specific technologies for a specific goal, and we develop accordingly. For example, we can think of a friend who plays PUBG: BATTLEGROUNDS with me around the clock like a teammate who follows the orders I give. We’d like to create a virtual friend who understand the game, listens to my orders, and plays alongside me whenever I need, although this friend is not perfectly like a human being.
Considering Deep Learning Div.’s biggest goal in the long-term perspective is to create the Virtual Friend, what will be your short-term goals in the process?
Kangwook: Like you said, our long-term goal is to be able to use our virtual human technologies in various areas. Our medium-term goal is to develop various deep learning element technologies to offer them to our colleagues in KRAFTON, and to make them as new services for the markets.
In a relatively shorter-term perspective, we should keep releasing the element technologies. The best way to do that as a group of researchers is to produce good research papers, which would make our products and service, as well as the virtual human more trustworthy. And in turn, we would be able to attract more talents who agree with our values to further accelerate the progress. Since we have high standards in determining “good” research papers, we won’t be able to produce lots of them. Still, we’d like to surprise people both at home and abroad in the foreseeable future.
I’m curious how the deep learning researchers work. Do they have a unique routine at work?
Hyeongjin: Compared to my previous experience as a developer, I think the main task for our researchers is to analyze the results of experiments. What every researcher in Deep Learning Div. does in every morning is to check the results of GPU computing (Note: Graphics Processing Unit is mainly used to process image data. As it can perform massive volume of calculation very fast, it’s widely used by machine learning researchers and developers.). We sometimes jokingly say to ourselves it’s the GPU that does all the work, not us. The researchers set an experiment on the GPU before leaving and check the result when they came back to work in the morning. Honestly, it doesn’t go well in most cases, so they usually start a day with debugging.
We have many researchers working on specific fields such as speech, image, and text, and they spend a lot of time to share what they’ve done. They also keep doing experiments and modifying the codes. If they face a problem that cannot be solved just by modifying the code, then they go back to the beginning and examine data. They repeat this kind of cycle over and over again and it’s their routine.
Do you have any principle of working and communicating that you want everyone in Deep Learning Div. to follow?
Kangwook: We need more proactive communication. I think developing is like architecture. Of course, it requires enormous time and effort, but we can find a way anyhow. However, this doesn’t always happen in researching. Researchers sometimes don’t get any result even after a month long of hard working. When it happens, we feel very disappointed and exhausted. Some researchers may worry about achievement and performances, but I always tell them facing lots of failures itself can be a big achievement. For example, if you tried only once and failed, when others failed 10 times in 10 challenges, you definitely didn’t work hard. But it’s never wrong that you failed much and didn’t gain any desired result. Nobody knows a right answer now, so it’s very crucial for the researchers to have active communication whenever they come up with new ideas.
When you make a game and release it in the market, the possibility to make a hit is extremely low, and of course, there are lots of failed attempts in this industry. For this reason, KRAFTON believes that the game developers should know how to learn from their failures to move forward. For deep learning researchers, since there aren’t many successful references, they must have their own know-how in dealing with failures. How can we learn from our failures or bad results?
Kangwook: I’ve focused more on mathematical theories than on applied deep learning in my research, I try to figure out the cause of failures in principle. Likewise, even when I succeed, I thoroughly analyze why it worked. I never think that happened out of pure luck. For instance, when I gain a successful result with 100 units of data, I don’t finish the experiment there. Instead, I reduce the volume of data on purpose and run a follow-up experiment. If I can get the same result only with 20 units of data, whereas it doesn’t work with 10 units, then I can confirm that I don’t need to spend 100 units of data next time. The same is true for when I fail.
You said researchers in the AI and deep learning research sector are actively opening what they’ve discovered. In that sense, university-industry collaborations can be very important to researchers, too. Do you have any plan regarding such collaborations?
Kangwook: As I told you, my background lies on theories. Thus, one of big goals is to be able to work with professors who are specialized in applied fields. To this end, we’re preparing for several projects with the world’s best professors in the speech and image deep learning fields starting this year. And I often ask the best experts in this sector to participate in seminars and give us some consultations.
We’ll also run the first research internship program for graduate students from home and abroad this summer. I hope remarkable talents join this program to work on papers together with our researchers, and then spread their experiences here to their colleagues. As they study in a graduate school writing research paper, I believe it would be a nice opportunity for them to experience a big project based on a massive data, which is not easy to do in the university.
You must be looking for talents from diverse fields. In which positions are you hiring?
Hyeongjin: We’re currently looking for video engineers and natural language processing engineers. As we believe that these once-separated fields are increasingly getting united, we’re thinking of hiring people based on their experiences and expertise in deep learning and then assigning them to more detailed sectors later.
For example, even if you don’t have experiences in the 3D video sector, you still can apply for the video engineer position. Since most of the necessary technologies and resources are open and available, we believe that you’ll be able to quickly adjust to the position should you have experiences in deep learning. I’d like to stress that everyone who has studied deep learning can apply for any position, not limiting oneself in a certain position.
Apart from major, skillset or career, is there anything you want from your future colleagues?
Hyeongjin: The reason why I chose to join KRAFTON is because it weighs on the value of “creativity.” It’s nothing strange to seek something “interesting” based on deep learning here. Therefore, I hope to work with colleagues who value creativity. One more thing I anticipate from them is to be emotionally resilient. Since researchers in Deep Learning Div. face failures quite often, they should be emotionally resilient get over and learn from the failure. It’s natural to feel depressed when you fail, but I think you should be resilient and overcome it soon.
Kangwook: I think I’m doing on the most interesting research among the studies I can do in a company in Korea. And it’s absolutely top notch. Therefore, if you want to do interesting research in the top level, you should come here. We actually didn’t open any source yet and we are ensuring stability, so I know that not many people would know what my division is doing now. But, I’m sure that they would soon recognize why we are so confident.
What is your personal aim as a deep learning researcher?
Kangwook: I’ve thought about it much since I joined KRAFTON. I love that I can now figure out how much the theories I’ve studied are working in the practical fields. There is a gap between the theory and the reality that I cannot analyze with the theoretical insights. I’d like to get inspiration from this to expand and change the existing theoretical research and make a new one. When I only did research at the university, I was prone to do experiments that are apart from the reality. In KRAFTON, I feel I have a wider view now that I’m doing my job based on more effective goals.
Lastly, how would you describe deep learning in one word?
Hyeongjin: I think the core concept of deep learning is “dimension reduction,” which means a process of extracting the essence. I think deep learning is eventually one of the tools that helps see the world in a simple way.
Kangwook: I think deep learning is like “deep sea.” We still don’t know much about deep learning and we’re not sure what we can do with it yet. When you try to study deep sea, some might say you have your head in the clouds. But it’s still possible that we could find many useful things from it. I think the same goes for deep learning. Some may wonder why a game company like KRAFTON researches it, but I believe we would be able to find something big and new. So, keep your eyes on us.
KRAFTON AI utilizes deep learning technology to enhance the immersion and enjoyment of games
Our goal is to innovate the paradigm of game production, publishing, and operation.
For more details on research, benefits and hiring, please visit our website.
KRAFTON official website : https://www.krafton.com/more-experience/ai/
KRAFTON AI website : https://www.krafton.ai/
