Jun 302010
 

Today The Escapist published an issue that’s all about games journalism, and there are a number of fascinating articles. I particularly liked “1984 out of 10”, a piece by Peter Parrish that examined the current game-reviewing scene through the lens of George Orwell’s 1936 essay “In Defence of the Novel”. It’s a sharp take, and shows how the inflation of praise (not just scores) results in the apotheosis of mediocrity, and thousands of crummy games walking around with Metascores in the high 70s.

Most of the commenters and linkers have focused on the idea of score inflation, which is ironic since the article is critical of readers’ obsession with and passionate screaming about scores. When I write a review, of course, I generally think about the score for a little less than a minute, and with few exceptions this has served me well. It’s almost amusing to see how obsessed people can be with the 5 I gave Shattered Memories or the 4 I gave Kingdom Hearts 358/2 Days, without devoting a moment’s concern to the actual content of the review. Fortunately, the comment policy at GameCritics.com allows me to give these kinds of comments a much-deserved summary deletion.

Seeing as Parrish’s article concludes with his hope that intelligent writers can rescue game reviews from the dustbin of advertorial content, I thought it might be time to look back over my own stuff and ask how I’ve been doing. So, I went back to GameCritics.com and reviewed my reviews. I’m intelligent, and I’m a writer, but am I writing intelligently about games? And how inflated are those scores I don’t think about?

Nostalgia
I feel like this was generally a success. Nostalgia was out to mine older games and stories to produce a familiar-feeling RPG and it succeeded at that, which I think I conveyed. I also managed to get across that I didn’t think this was a particularly interesting or memorable thing for a game to be doing. I got bogged down discussing the uneven difficulty, though. That’s important information that needs to be in the review, but it needed some lens so it would fit in with the discussion of the bland story elements. This is a pretty good effort.
My score: 6.0 / 10
Metascore: 72 / 100
Review grade: B

Silent Hill: Shattered Memories
This is a passable review. I think I got a bit too caught up in particular shortcomings, when my real problem with the game was that the exploration was too boring and the nightmares were too frenetic. I failed to adequately convey my feeling that Climax had really strong and worthwhile ideas for the gameplay and the story, but that their execution was too far off the mark. I also didn’t really get enough into what the game was about.
My score: 5.0
Metascore: 79
Review grade: C+

Kingdom Hearts 358/2 Days
Well, this game isn’t really about anything, so it’s no sin that I didn’t get into that. I’m pretty successful here at conveying my core problem with the story, which is that it feels like it’s just filling up time (the title is very appropriate). I simply failed to explain that the game always feels like it’s about to start going somewhere interesting, and then lets you down. The paragraph about the panel system feels like a digression. I needed to do more with the lens about how this game isn’t really something that will appeal to anyone.
My score: 4.0
Metascore: 75
Review grade: B-

The Saboteur
Well, The Saboteur is a game about blowing up Nazis, which I think I managed to get across. I should have done more to emphasize that the game is more about play than about immersivity. I got a touch too bogged down in details and didn’t sufficiently convey how the game felt in the process of play. This is the only game where my score is significantly higher than the Metascore, but I feel secure that I’m right about this.
My score: 8.5
Metascore: 73
Review grade: B

No More Heroes 2
This was not one of my better efforts, on any level. I clearly got hung up on comparing this game to its predecessor. That may have been unavoidable, given how much I loved No More Heroes, but the end result speaks for itself. I didn’t discuss this game enough on its own merits, and I forgave it too much for its difficulty and execution issues. This is one case where, in retrospect, I scored a game too highly. Were I to try again it would probably land in the 6-7 range.
My score: 8.0
Metascore: 85
Review grade: D

BioShock 2
I think I nailed it. Either this or my Fable II review from this blog is the best review I’ve written. I got hold of everything the game made me think of, what the game seems to be about, what it’s really about, and why it succeeded at the latter. I wish every review I wrote was this good.
My score: 9.0
Metascore: 88
Review grade: A

Deadly Premonition
I think I did a pretty good job here, too, although I didn’t really succeed in conveying what the game is about or how empty it felt. I should have done more to discuss the tension problem this game had, which was much like the problem for Shattered Memories. I think I succeeded, though, in explaining what didn’t work in this game on a level more fundamental than just saying that the shooting wasn’t any fun.
My score: 4.0
Metascore: 66
Review grade: B+

Infinite Space
I’m not satisfied with this one. I mention a whole host of different problems and successes, but I never really build them into a coherent picture of the game’s ideas or themes. Of course, the game itself is somewhat incoherent in this regard. In particular I failed to really deal with whether and how the gameplay related to the narrative. I recall that the early drafts of this essay were really bogged down in details. The final form still has too many, and is a bit lost in the weeds as a result.
My score: 7.5
Metascore: 75
Review grade: C-

Metro 2033
This game is about desperation and privation, and I managed to get that across. I got bogged down in some details about the plot and the mechanics, and I didn’t talk enough about the strange transition from FPS to first-person platformer in the last parts of the game. A stronger lens around the Artyom-as-ghost idea could have really tied this review together strongly.
My score: 6.5
Metascore: 77
Review grade:B-

Resonance of Fate
This is a lame, workaday review for a lame game. I probably scored this one too high. There’s not much else to say, except maybe that I didn’t fully capture my loathing for these characters.
My score: 3.5
Metascore: 72
Review grade: C-

Red Dead Redemption
I’m slightly ambivalent about this one. I feel like I nailed what the game was about at its core. I’m a bit less certain that I properly conveyed the experience of playing the game. The information is in there, but I think it might be pushed a bit too much into the background. If I’m going to miss the mark, though, I’d much rather miss in this direction, swinging too far towards conceptual contemplation, than by clinging too close to detail-oriented reviewing.
My score: 8.5
Metascore: 95
Review grade: A-

ModNation Racers
This is not my best. It feels very close to traditional software review where I take various features and just examine them in isolation. I really should have lensed this whole thing around the question of the game’s identity crisis and examined its competence from that angle.
My score: 6.5
Metascore: 82
Review grade: D+

Alpha Protocol
I feel like I did a good job on this one, because I really thought the game was a failure in every way, and “every” involves a lot in a game this complex. So the criticism comes in pretty densely, but I needed to address that this was a bad idea that Obsidian did a crummy job of turning into an poorly-designed game. Still, late in the review it starts to turn into a bit of a hit list, which it probably shouldn’t have done.
My score: 1.0
Metascore: 63
Review grade: B-

Converting the scales so they match, my reviews come to an average score of 60 (median of 65) with a standard deviation of 24. That’s a pretty decent spread. For the same group of games, the average Metascore was 77 (median 75) and the standard deviation was 8.8, which makes the point about score inflation better than I could hope to. The games I’ve reviewed vary wildly in every aspect of quality, from narrative to design to basic coding, and they simply should not have landed so closely together on any scale. That these games averaged nearly an 80 is also a sign of trouble, to my mind.

From my perspective, the thing I need to work on most in the writing is formulating a high-level view of the games I’m reviewing, rather than getting caught up in their details. I’m tending to miss the mark on the side of getting too far into the details. Like I said above, if I have to miss, I’d rather screw up by being too concept-oriented, rather than the other way around. There are about 8000 sites on the internet where you can learn about a game’s graphics or mechanics in the review. What’s needed, and what I want to provide, is discussion of a game’s ideas and how its design informs, develops, and supports those ideas. The goal is to convey in a review what a game is about and how it is about it.

APB’s blue wall of silence

 marketing, video games  Comments Off on APB’s blue wall of silence
Jun 192010
 

Well, we’ve just come to the end of what will probably be the world’s least productive week this year. Sports fans had enormous distractions: the NBA finals, the beginning of the US Open, and of course the World Cup, which is something I can’t avoid hearing about since I work with a Korean, two Germans, and an Italian. And that’s just the soccer-mad people whose teams are actually in the tournament! Fortunately, most of the lab computers do not have speakers, so I am spared the god-forsaken noise of the vuvuzelas while at work. If you’ve no taste for athletics, however, this week offered an alluring diversion in the form of E3. Between the various webcast events and the constant impact of game and hardware impressions, there was plenty to divert people’s attention from work. Apparently, Realtime Worlds took note of the deluge and tried to slip something in under it — a review embargo on their upcoming MMO shooter, APB. Now, there’s nothing unusual about a review embargo, but this one is notable because it ends on July 2, a full week after the game is released.

I heard about this at roughly the same time from Nukezilla (née Negative Gamer) and Rock, Paper Shotgun. Many of the responses to these posts indicate that the open beta has revealed a core game with some great customization but less-than-ideal gameplay (a widespread problem among games that allow a lot of user creativity). The natural conclusion from this starting point has been that this is a move by RTW’s PR staff to keep negative reviews of a marginal game off the air long enough to make a significant profit off the first wave of suckers. Obviously RTW doesn’t see things this way, and explained their decision to the RPS crew as being about making sure that reviewers get a veridical impression of the game, something that’s apparently impossible in the present open beta for whatever reason. This explanation strikes me as being only mostly bull.

I don’t generally play MMO games; I don’t have the time for it. But I do play games, and in most cases you can tell that a game is crap within thirty minutes of inserting the disc. Sometimes it takes a few hours, but for the most part a game that is really bad lets you know this early on. It doesn’t take long to figure out that a game has got painful graphics, or janky physics, or that the shooting is terrible. Once you’ve realized these things, more time with the game won’t help, unless you’ve got a serious case of choice-supportive bias. If a game is just plain bad on a mechanical level, or even sometimes on a writing level (everyone with any sense knew Warrior Within was crap before the tutorial ended), it’s obvious early on, and an additional week with the game won’t help things. I’m not aware of any game that offers a graphic upgrade once a player hits level 10.

But, a game isn’t just gears and scenery. The quality also depends on what it demands of you and the context it provides for those demands. Is the level design solid and varied? Does the story stay consistently interesting, or does it go off the rails? These questions take longer to resolve; we’ve all played games that go from a promising beginning to an abysmal end. More rarely, a game begins and ends well but takes a long detour through the doldrums (Red Dead Redemption, for example). Repetitive gameplay or uninspired design can easily remold a beautiful and fun game into a tedious and frustrating mess (Muramasa). And then there’s “Indigo Prophecy syndrome”, the tendency of a game with a good story to take a bizarre turn for the worst towards the end, that is the reason Yahtzee never gives a positive review unless he’s finished a game. Extending play sometimes reveals the hidden weaknesses of a game that seemed good, but the opposite rarely happens.

Of course it’s possible, especially with an MMO game, that the content available early on is less interesting than what the player can do later. Frankly, though, we should be well past the point where “kill ten rats” is considered acceptable design. I can do crappy little fetch quests in practically any game, and at this point many of them are free. If you’re asking me to pay for your game then it had better start popping within the first hour. If I can read an entire novel in the time it takes for your game to really get interesting then you are a professional failure.

So, the good folks at Nukezilla and RPS should be able to assess the quality and breadth of play within less than, say, 12 hours. What they cannot assess in that period of time is the game’s community. First of all, it takes time for a game’s player base to take shape, considerably longer than a week. Will APB tend to draw people who are just out to make random mayhem, or will it draw people who are interested in forming tight-knit crews? Perhaps it will draw both groups initially and only retain the latter after a few months. Even if the player base is established, timing plays a role. The weekend population of a game may not be the same as the one you find on weeknights. In any event, just one week won’t be enough to inform the reviewer (or by extension, potential players) just what it will be like to play the MMO game in question.

From the standpoint of writing a review this means there are two problems with the embargo. First, it’s not going to improve reviewers’ reports of the early experience. If the driving and shooting really are excruciating, then hanging around for seven days provides no benefit in terms of accurate description of the gameplay itself. Second, the only thing that can be improved by requiring a longer wait, i.e. the assessment of the game’s population, can’t actually be judged in a week anyway. For determining the quality of gameplay, a week is too long to wait; for determining the qualities of the player base, a week (especially the first week) isn’t long enough.

From a PR standpoint the embargo has a different danger. This embargo only applies to people you’ve given the gift of some free play, who presumably will at least be somewhat favorably disposed towards you as a result. Now, because they agreed to review your game for free, they have been put at a disadvantage relative to their competitors, who can get the scoop by just buying the game. Amateur reviewers and sites that didn’t play ball with you will produce all the early word on the street, meaning that you have completely ceded the message to people you don’t know. Granted, all those guys will have to pay you $60 for the right to produce their review, but is that $60 worth the loss of goodwill from sites you have a relationship with, plus the risk of handing over your message to sites you have no relationship with? Frankly, this approach strikes me as being insane. This is not to mention the perception that is created by doing the gaming equivalent of refusing to screen a film for critics.

Of course, RTW can respond to unfavorable early reviews by the now-standard mechanism of digging up a player log and complaining that the reviewer only played the game for X hours, where X is some number that a PR flack can argue is unjustly small. But this kind of protest is generally specious, for reasons I’ve outlined above. As a player, I don’t owe your game anything. I gave you $60, which means you owe me. If you’re not going to deliver on that investment unless I additionally spend hours doing things I don’t like, then I deserve to know about it. My time isn’t free, and if I’ve got to spend hours of it running stupid quests or fighting an awful control system just to get to marginally tolerable content, then I’m going to have to insist you at least pay me minimum wage for my trouble. If a reviewer has five horrible first hours with your game, then, as long as he’s honest about it, that is important information to me.

RTW has a valid point in that a review of their (or any) MMO game will not accurately capture its spirit if it’s based on a reviewer’s experience during a beta or in the first few days of the game being online. That being said, reviews sampling that period can at least let players know if the game, on a purely mechanical level, might be for them. Moreover, a delay of just a week will not suffice to let reviewers accurately assess the game’s society, as that will still be in a process of rapid evolution. In addition, this kind of a restriction will always be anathema to the first-to-publish tier of big-name review sites. Given the marginal benefits and significant costs, it’s difficult to understand why Realtime Worlds chose to take this approach rather than, say, offering sites additional free playtime after a few months to support a follow-up review focusing on the game’s community. Embargoing reviews after the release date can only serve to support negative perceptions and word of mouth.