게임 개발에 플레이 테스트를 도입한 밸브의 사례

Valve의 상징적인 게임 포털은 플레이 테스트 피드백을 통해 재구성되어 GLaDOS를 탄생시키고 게임 디자인의 모든 측면을 개선했습니다.

### 게임 개발에서 플레이 테스트의 중요성
* 🎮  밸브는 플레이 테스터들이 실제 게임으로 인식하지 못하자 포탈에 GLaDOS를 적대자로 등장시켜 상징적인 존재로 만들었습니다.
* 🧪  Valve가 포털의 거의 모든 측면을 개선하는 데 플레이 테스트를 사용한 사례에서 알 수 있듯이, 플레이 테스트는 게임 디자인의 변화를 이끌어내는 데 필수적입니다.
* 👥  하프라이프부터 포탈 2에 이르기까지 밸브의 플레이 테스트 방식과 게임 개발에 미친 영향은 성공적인 게임을 만드는 데 있어 플레이 테스트가 얼마나 중요한지 잘 보여줍니다.
* 🔑  효과적인 플레이 테스트를 위한 Valve의 팁: 일찍 테스트하고, 자주 테스트하고, 조용히 관찰하고, 디자이너를 참여시키고, 적합한 사람을 타겟팅하고, 가정에 도전하고, 피드백을 데이터로 해석하세요.

About a year into the development of Portal, playtesters kept giving 
Valve very similar feedback. Something along the lines of that was a great 
tutorial, I can't wait to play the actual game. *VCR Pause Sound Effect* Uh. Slight problem. That was the actual game. Despite playing through 14 or so 
hand-crafted puzzles, players were clearly missing some vital element that would 
tell them this was, indeed, a video game. And so after a lot of discussion, Valve 
decided that the game needed an antagonist: someone to push back against the player, to provide motivation to keep moving, 
and to put these puzzles into context: this was all training in order to defeat a boss. The end result was, of course, GLaDOS: a loopy AI overlord with a 
biting, passive-aggressive wit. She teases you, taunts you, tricks you. And the whole game leads up to a climactic 
one-on-one showdown in her central chamber.

GLaDOS: Well, you found me. Congratulations. Was it worth it? Because despite your violent behaviour, the only thing you've managed to break so far is my heart. Thanks to sharp writing and killer voice acting, she has become one of the most iconic video game villains of all time. But, says Valve's Robin Walker, her genesis begins with a straightforward process of us trying to solve the core gameplay problem in Portal. Now if you ask me, the fact that GLaDOS may never have existed if it wasn't for this feedback goes to show the value of playtesting. Not to be confused with quality assurance - that's for rooting out bugs. Or focus testing - that's for market research. No: playtesting is simply watching people play through a chunk of the game - sometimes with a questionnaire or interview at the end - and then using what you see and hear to drive changes in the game's design.

For example, if Valve sees that players are routinely getting themselves killed when trying to redirect these energy balls, perhaps they should make a change like having it so you can only place portals on walls that are way above the player's height. And so on Portal, this approach was used to touch up almost every aspect of the game. Valve used playtesting to make sure the learning curve was perfect. They used it to remove moments of frustration, to make sure players noticed key objects, to improve the pacing, to tweak the difficulty, and to ensure the storyline was coherent. Playtesting even led to the game's iconic visual style with those sterile white walls and floor. Originally, the game featured cluttered and grungy environments, but playtesters found it hard to identify the key puzzle elements. In one test, a player spent half an hour trying to push a shelf onto a button while completely ignoring a nearby box.

Kim Swift - one of the key developers 
on the project - calls playtesting, the most important thing that we 
learned since coming to Valve. You see, if you didn't know, Portal 
started life as Narbacular Drop: a student project made at 
Digipen University in Washington. They were invited to show the game off at 
Valve's headquarters and about halfway through the demo, CEO Gabe Newell stopped the 
presentation to offer the entire team a job. Their goal was to remake the game in the Source 
engine, and within the Half-Life universe. And to help, Valve introduced the small 
student team to its game development process. Here's how it works. You start with a goal: perhaps, in this case, to make a puzzle that is clearly 
readable and satisfying to solve. You then take a stab at reaching that goal by designing something - in 
this case, a test chamber. Next, you evaluate whether your design 
reached that goal by doing a playtest.

And if it doesn't meet the grade, 
change the design and repeat. Valve keeps going with this 
iterative process until it is no longer excruciatingly painful to watch 
the playtests, says developer David Speyrer. GLaDOS: Fantastic! You remained 
resolute and resourceful in an atmosphere of extreme pessimism. Perhaps, given the subjects 
of its most famous games, its apt that Valve sees game development as 
an engineering problem or a scientific study. Valve's former in-house psychologist, 
Mike Ambinder, says, we see our game designs as hypotheses and our playtests as 
experiments to validate these hypotheses. Now, it should be said that playtesting 
is by no means exclusive to Valve. And indeed, every game developer sources 
player feedback at some point in the process. But the difference is that Valve is obsessed 
with it, to an almost religious degree. Ambinder calls playtesting the most important 
part of the game development process.

Newell calls it our secret weapon. And a designer who joined the studio in 2018 said, I always heard that Valve does a lot of playtesting before I came to work here, and I was not prepared for the amount of playtesting this company does. Valve does playtesting early. Swift and her team started running playtests of Portal after just one week of starting to build the game at Valve, even though they only had one half-finished room to show people. And then do it often. Portal was playtested practically every week. They'd do a playtest on Friday, discuss the results on Monday, apply the lessons the rest of the week, and test again on Friday. It sounds extreme. But I think this devotion to playtesting makes a lot more sense when you learn about the disastrous development of Valve's very first game, Half-Life. About two months before Half-Life was supposed to ship,

Valve realised that the game... well, sucked. The developer said, You couldn't play the 
game all the way through. None of the levels tied together well, and there were serious 
technical problems with most of the game. As a whole, the game just wasn't working. The only real option on the table was to 
scrap the whole thing and start from scratch. But this time, they would do things 
differently - which lead to two game development philosophies that are 
still present in the studio today. One is the idea of the cabal: where 
the staff is split up into tiny, multi-disciplined teams who each take complete 
ownership of a small chunk of the game. But the other is frequent playtesting, from 
the very earliest point of development. This time, if the game sucked, 
they wanted to know right away. So, about three months into development of the 
restarted Half-Life, they started playtesting.

They found random gamers in video game shops, and contacted people who filled in those little registration cards you got in old PC boxes. They'd sit them down in front of the game and just watch them play. Each test would result in dozens of things that needed to be fixed, changed, added, or deleted from the game. Like, after watching some playtesters break every crate in a level, Valve realised it probably needed to stuff some of those boxes with goodies like ammo and health packs. And this new process obviously worked: this version of Half-Life was a huge success, and is now seen as one of the most important and influential games ever made. And, as such, Valve would use playtesting extensively in all of its future games. On Half-Life 2, Valve planned to give out the gravity gun towards the end of the game, but players loved using it so much, the team decided to make it available much sooner.

On Left 4 Dead, playtesters found it hard to find teammates who were in trouble, which led to the x-ray outlines of your fellow survivors. And on Portal 2, a paint type that let you walk on walls was scrapped when it made multiple playtesters feel queasy. The arrival of Steam allowed Valve to go further - converting millions of online players into post-launch playtesters. When Steam's data showed that lots of players were getting stuck in Episode One, they released a patch to reduce the difficulty of a tricky siege battle. Plus, playtesting was instrumental when exploring an all-new technology, virtual reality, for Half-Life: Alyx. For instance, Valve learned that a player's tolerance for standing around watching people talk was significantly lower in virtual reality, so had to speed up the game's pace. Player behaviour helped us navigate VR development, says Valve's Christine Phelan.

You could almost think of the player as another designer. There is barely a moment in the game that wasn't improved by what playtesters told us and showed us through their play behaviour. So, from Half-Life 1 to Half-Life: Alyx, playtesting has been an invaluable tool for Valve. So I want to share some tips from the developers to show you how to playtest more effectively. Though it should be stated: Valve is not like other companies. From its unorthodox organizational structure to its near infinite resources, there are very few companies on Earth who can copy what Valve does. And also, this approach may work best on super linear, extremely hand-crafted story games, and other techniques like heat maps may work better for other types of games. But, with those caveats out of the way, perhaps their experience can still provide some useful guidance. So, tip number one is test early.

Valve says, playtesting is where we make the vast majority of our most important changes to our game. So we try to do it as early as possible on the project, to get the most value out of it. When you identify a problem early on, you have time to go back to the drawing board and develop an effective solution to the problem. But do it too late, and sometimes the only way to fix a problem is with a flimsy band-aid solution. Like, uh, I dunno, having the characters just tell you the solution to a crummy puzzle. Valve may start testing within a few days of prototyping a mechanic or designing a level. Even if it looks hideously ugly, with programmer art or bright orange textures on all the walls. But this actually helps them avoid wasting time on other aspects: there's no point investing heavily in art or audio if the game mechanic

is gonna be completely changed - or cut from the game entirely. Number two is test often. Valve typically tests every single week, to make sure that they are constantly iterating on player feedback. This also leads to a huge amount of data. Each Half-Life 2 chapter had about 100 playtesters, for example. And that mass of information can be invaluable. By having a huge number of people look at the game, Valve can look for common trends in the feedback, and avoid tweaking the game based on a few weird outliers. And the more experience you have seeing people interact with your designs, the better you'll get at preempting this feedback in the future. Gabe Newell says after you've watched a couple hundred playtests, you start to develop a much better sense of what are successful and unsuccessful design strategies. So, some famous lessons learnt at Valve include players don't learn when stressed and players don't look up.

Number three is shut up and watch. A playtest should try to simulate the 
actual experience of playing the game. Which means the observers need to stay 
quiet for the duration of the test: no hints, no guidance, no 
answers to burning questions. Nothing is quite so humbling as being forced 
to watch in silence as some poor playtester stumbles around your level for 20 minutes, 
unable to figure out the 'obvious' answer that you now realise is completely arbitrary and 
impossible to figure out, the studio has said. Post-game interviews and 
questionnaires can certainly happen - those were instrumental in solving 
the GLaDOS problem after all - but you'll often learn more by watching 
players than by talking to them. Swift says they may tell you later 
that they like the game but you'll really tell by their body language whether 
or not they actually enjoyed themselves. It's also a common game design trope that

playtesters like to offer potential solutions to the problems they face - but as they don't have insider knowledge of your vision or constraints, those solutions are usually better left ignored. Number four is designers should run playtests. Valve does not outsource testing to a special department or leave it with a publisher: playtesting is done by the people who are actually responsible for the design and execution of the level or mechanic in question. The cabal, from earlier. This shows the developers exactly what needs fixing, and can give much-needed motivation to improve the game. And player behaviour might inspire new puzzle solutions or ideas for the designers watching. Like, in Half-Life Alyx, players would instinctively cover their real-life mouth to stop Alyx from coughing, and alerting a gigantic blind zombie named Jeff. So Valve turned covering your mouth into an actual game mechanic. Number five is get the right people.

Valve gets feedback from a huge variety of different playtesters, from fellow staff members to little kids to pro gamers. But it's learned to always have a target audience in mind when making changes. Like, when Valve was figuring out how to make a climactic boss fight against GLaDOS, they initially got feedback from hardcore FPS players, who said the level needed more action, more challenge, and more skill. But that idea was a dud: when facing this tricky boss fight, says Erik Wolpaw, the vast majority of playtesters who had gotten used to the slower-paced, cerebral nature of Portal were just frustrated, confused, and dissatisfied. They still wanted to satisfy those hardcore FPS players, but did so through optional content, like Portal's advanced chambers and challenge maps. Number six is to challenge your assumptions. Okay, so maybe the GLaDOS boss fight didn't need to be a test of skill.

So, perhaps it needed to be the most 
complex puzzle in the entire game? Surely that would be a suitable climax. But, here's the thing - playtesters 
thought the game's mid-point escape, that bit where you use a portal to 
avoid falling into a pit of fire - players thought that was 
incredibly climactic and satisfying. Even though it's basically 
the easiest puzzle in Portal. But Valve realised the time 
pressure, the visual impact, and the high drama all made it 
way more exciting to players. So maybe they were overthinking 
the final boss, too. We've been holding on to this idea that 
we need a complex puzzle at the end and it simply wasn't true, says Swift, and so 
they settled on a pretty simple sequence. It's very easy for game designers to make 
wonky assumptions about how players will act - I recently made a whole video about 
doing this in my own puzzle game.

I'll drop a link to that one in the end screen of this video. Perhaps the most important tip, though, is that playtesting feedback is just data, and it's up to the designer how that data is interpreted, filtered, and applied. For example, Half-Life 2 originally had a very short introduction before Gordon Freeman grabbed a gun and started shooting. Playtesters loved it. Who wouldn't be excited when jumping into the action? And so, given this good feedback, it would be easy to keep the game exactly like that. But, despite the positive sentiment, writer Marc Laidlaw says Valve decided to keep working. They thought it would be better to delay combat. We wanted you to witness the cops doing something horrible, and feel like what you were doing was a response to that - not that you were just a killing machine, says Laidlaw. And they thought it would lead to a better emotional pay-off if you

had to wait a much longer time before finally getting Gordon's iconic crowbar. BARNEY: Oh, and before I forget. I think you dropped this back in Black Mesa. Good luck out there, buddy. Here's the thing: playtesting is as useful as you make it. If you just bend the game to the whims and desires of every playtester who comes through, you'll end up with a dumbed-down game, or bland, design-by-committee sludge. But if you go in with a clear goal, a specific game you want to make, for a specific audience you want to entertain, and then use playtesting to validate whether you are hitting that goal, you can use feedback to unlock the incredible game you are trying to make. Oh, don't click off. I've got one last Portal story while the Patreon credits roll. So, the invention of GlaDOS is actually not the most dramatic change that Valve has made.

in response to player feedback. During an internal game jam, held 
shortly after the release of Portal 1, a bunch of Valve developers came up with 
an experimental puzzle game called F-Stop. In this game, you use a camera 
to take photos of objects. Then, you can spawn that object elsewhere in the world 
- perhaps at a completely different scale. Gabe Newell loved the concept, and said it 
should be developed as a follow-up to Portal - meaning each game in the series would feature 
a different piece of technology developed at Aperture Science. But, after nearly a year of development, 
playtesters were very vocal in their feedback: 
Portal without portals just didn't work. And so Valve decided to scrap the game 
and start Portal 2 all over again. In recent years, we've been able to 
see what F-Stop might have looked like, as fans have used leaked assets and code 
to make various recreations of the game.

But Valve never returned to the concept. If you want to play something similar, check out Superliminal, or wait for the upcoming photography puzzler, Viewfinder. Thanks for watching.

Valve's "Secret Weapon"

Game Maker's Toolkit is a deep dive into game design, level design, and game production, hosted by Mark Brown.

Want to contact Mark? Visit the GMTK website below to find the contact form.


Game Maker's Toolkit

플레이 테스트가 포털 개발에 미치는 영향

플레이 테스터들은 처음에 Portal에 비디오 게임의 중요한 요소가 부족하다고 느꼈고, Valve는 이러한 피드백을 반영하여 상징적인 적대자인 GLaDOS를 만들었습니다.

Valve는 플레이 테스트를 통해 게임플레이 문제를 해결했을 뿐만 아니라 게임의 디자인, 학습 곡선, 시각적 스타일, 스토리라인의 일관성에도 영향을 미쳤습니다.

포탈의 개발은 플레이 테스트의 영향을 많이 받았으며, Valve 팀은 플레이 테스트를 통해 얻은 가장 중요한 교훈으로 꼽았습니다.

게임 개발에서 플레이 테스트에 대한 Valve의 헌신

Valve의 반복적인 플레이 테스트 프로세스는 게임 개발 철학의 핵심입니다.

개발 초기 단계부터 플레이 테스트에 집착하는 Valve의 집념은 첫 번째 게임인 하프라이프의 참담한 경험에서 비롯되었습니다.

플레이어의 피드백을 광범위하게 수집하려는 이러한 노력은 Valve 게임 성공의 핵심 요소였습니다.

Valve의 효과적인 플레이 테스트 팁

Valve는 플레이 테스트를 통해 Left 4 Dead에서 팀원의 엑스레이 윤곽선을 도입하고 Portal 2에서 벽을 걷는 페인트를 제거하는 등 중요한 게임 개선 사항을 도출했습니다.

Valve는 개발 프로세스 초기에 테스트를 실시하여 영향력 있는 변경을 위한 시간을 확보하는 것이 중요하다고 강조합니다.

플레이어의 행동과 피드백을 기반으로 하는 Valve의 독특한 플레이 테스트 방식은 선형적인 스토리 중심의 게임과 가상 현실 경험을 개발하는 데 중요한 역할을 해왔습니다.

Valve의 플레이 테스트 모범 사례

Valve의 플레이 테스트에는 플레이어 피드백을 수집하고 분석하기 위한 빈번하고 광범위한 테스트가 포함됩니다.

Valve는 게임 후 인터뷰에만 의존하지 않고 플레이어를 관찰하고 게임 중 플레이어의 행동에서 배우는 것이 중요하다고 강조합니다.

또한 Valve는 게임 디자이너가 직접 플레이 테스트를 진행하여 개선할 부분을 효과적으로 파악하고 플레이어의 행동에서 영감을 얻는 것의 중요성을 강조합니다.

플레이 테스트 및 피드백에 대한 Valve의 접근 방식

Valve는 다양한 플레이 테스터 그룹으로부터 피드백을 수집하지만, 궁극적으로 변경할 때는 타겟 고객을 고려합니다.

Valve는 GLaDOS 보스전 개발과 하프라이프 2의 도입 과정에서 보았듯이 가설에 도전하고 플레이 테스트 데이터를 해석하는 것이 중요하다는 것을 배웠습니다.

Valve는 게임 시퀀스 디자인에서 알 수 있듯이 단순히 하드코어 게이머를 만족시키는 것보다 정서적 영향과 플레이어 경험을 우선시합니다.

플레이 테스트가 게임 개발에 미치는 영향

플레이 테스트는 게임의 목표와 타겟 고객을 검증하는 데 유용할 수 있습니다.

Valve의 F-Stop에 대한 경험은 플레이어 피드백이 어떻게 Portal 2를 위해 게임을 폐기하게 되었는지를 보여줍니다.

F-Stop에서 퍼즐을 풀기 위해 카메라를 사용한다는 콘셉트는 플레이테스터들의 강력한 부정적 피드백으로 인해 포기되었습니다.