Working Up A Scoring System

UPDATE: In over 45 years, I’ve written a few things that I wasn’t happy with later. This article may beat them all. It’s poorly organized, wordy, and I’m really not liking it much now that I’ve read it with fresher eyes. I’m currently reworking it, and the permanent version that will be placed here should be far superior. I ought to engage in a cover-up and remove this one, but I think I’ll let it stay as an object lesson to me that when I don’t feel the muse speaking, don’t force the bitch. Or something like that.

I’ve been mulling around trying to formalize my system of reviews for some time. It’s been difficult, because I’m not an organized person to start with, and I tend to write about a series as I’m watching it, or just after I finished it. I’ve been prone to tossing off statements like “I’ll give it a C,” or “This is at least a B+.” Scoring right after I see the series is difficult, because it’s so subjective — I’m grading on how I feel about the series.

Recent discussion of Shingu and Misaki Chronicles led me to decide that the minimum qualification for a B was that I’d re-watch at least my favorite parts of the show immediately. Doing that doesn’t mean it is a B, but if my reaction to the show upon finishing it is to shelve it and not even go back to look at the “good parts,” then obviously the show didn’t have many.

Once I decided that much, the rest of the ratings started falling into place over the last few days. Pretty soon, I had a system, and here it is.

[Table=8]

“+” or “-” may be assigned if I feel the base grade didn’t really fit the show fairly.

I’ve decided to take that subjective “feel” and run with it. The entertainer’s core job is to entertain me: I pay my money (assuming I purchased instead of downloaded a fansub) and I expect to be entertained in return. (Exactly what I mean by entertainer could be the subject of of a long-winded digression, but let’s not go there. Just substitute “studio”, “director”, or “writer” if it fits better.) It can be base entertainment (any harem or fanservice show), it can be thoughtful entertainment (Simoun, Lain), emotional entertainment (Simoun agan, Misaki, Angelic Layer), intrigue (Noir), fun and adventure (Outlaw Star) … well you get the idea. Whatever the hook, it’s got to capture my attention and make me feel that the time I spent in the entertainer’s world was not wasted. This table summarizes how well I feel the entertainers did their job.

Since it is subjective, the final score is not the average of the scores for individual categories, and might not even be closely related to them. The process of breaking down the series and scoring individual parts only influences the final overall score I give the show. It does not determine it directly. The overall grade is a highly subjective rating of how the series impacted me emotionally. Was I entertained? Did it rivet my attention, and draw me into the entertainer’s world? Was it something I want to watch again? Or was it total crap that I wish I’d never bought? These are the questions I ask myself for the overall rating, and that’s what I give the overall score on. A series that receives objective category scores that average much lower than the subjective overall score may be a case of the direction, voice acting, and writing coming together to create something that really is “greater than the sum of its parts.” Or maybe I was just easily entertained that week, who knows?

As I note, it’s highly subjective, but it’s also subject to change, which happens through the process of breaking the show down and analyzing it. A lot of things just don’t survive the trip to the refrigerator. When I have just finished a show, I tend to be very positive about it, because I enjoyed it — while I was watching it. Later, the emotional “afterglow” fades, and the rational, analytic side kicks in. Sometimes, negative aspects of the show that I overlooked (sometimes deliberately) may come back to bug me. I start dissecting the series, breaking down the individual categories, noting which ones did a good job and which ones were bad. This is the point where I start engaging the left-brain, and it’s a bad sign if this phase starts before the series has ended; it means that I’m not caught up in the world the storyteller is trying to create — in other words, I’m not being entertained — I’m just being diverted. (Shows like this might get a C from me.)

This means that the emotional side is stronger at that point, and a lot of my early reviews got messed up by my writing them just as I finished the series. Of course the problem is, if I wait, I’m likely to lose interest in writing about it at all. That’s one reason why less than half of the series I’ve watched have been reviewed.

When I do review them, the format has varied a lot, but I generally look at the following things:

Art and Animation
Voice Acting/Dubbing
Plotting and Logic
Storytelling and Pacing
Characterization
Fanservice
Music
DVD/Packaging & Extras (or Fansub Quality)

Lately, I’ve not explicitly broken these categories out like I did when I first started writing about animé, although I’ve often mentioned what I thought of those things. I hope to get back to that, and eventually review (or re-review) all the series I have on the summary page, with a bit more in the way of analysis. The problem is, going back and looking at the series now, I know that the objective view is going to contaminate the subjective scoring. For instance, right after watching Yumeria, I’d probably have given it a C+ or a B. Looking back a year later, I just can’t. Part of it is that I’m a bit more knowledgeable now, and I know the animation was sub-par. A lot of it is the fact that the most memorable character, the one that stuck in my head, is a pedophile I didn’t like. That definitely affects the show negatively for me, and impacts the scoring, whether I want it to or not. I’m not sure what I’m going to do about this overall, since what will happen is that series I go back and re-score will tend to have lower grades than series I score going forward. To prevent “grade inflation,” I might have to write up a review right away, let it sit a week or two, and then see what I think.

Here’s some examples of series, and the overall grades I give them (as of now). Note that this doesn’t include any + or -. This is just a general example.

[Table=9]

I’ve created a new page that will be the permanent home of the system outlined above, as well as a summary of the score I give each series, although right now, it’s just a placeholder with no information.

Update: And I’ll re-write the above when I do. I posted it, didn’t like it, edited, it, and still don’t like it. But it’s late and I need to go to bed.

This entry was posted in Administrative, Episode Reviews, Series Reviews. Bookmark the permalink.

7 Responses to Working Up A Scoring System

  1. AvatarADV says:

    This is why I’m not bothering with ratings when I review something. Shows are good (or not good) in different ways – I can really appreciate a show from a technical perspective, but think it’s a crummy story packed full of stupid, and I can really enjoy a good bit of writing and characterization even when my eyeballs are bleeding. I’m susceptible to a bit of gee-whiz factor, so something genuinely interesting is going to get breaks in other departments. I’ve got a lot of old shows I have a soft spot for, even though by modern standards they’re crap – there’s really no reason to recommend Soul Hunter, for example, but I still get a big kick out of it. I dislike other shows for completely non-rational reasons (Sailor Moon!) So… how do I distill all that into a grade?

    The way I figure it, I’m not really capable of doing a thorough logical analysis of each show (or rather, that’d be highly against the point), so in the end there’d be plenty of inconsistencies in my grading system, ones which people could only work out if they knew what sort of shows I enjoyed or not… and if they’re putting that much time into figuring out my likes and dislikes, well, they can bloody read the part in the review where I say whether I liked the show or not. ;p I don’t need to say A+ to say “wow, really, Nanoha A’s was great!” (or “Divergence Eve was great!”, heh). I don’t need to say F- to say “Apocalypse Zero is the worst thing I have ever watched, ever.” And saying “I give E’s Otherwise a solid C-” isn’t really any more informative than saying “it’s so utterly average that you should probably watch something else,” no?

  2. AvatarADV says:

    (Also, rating stuff means that people can look at your rating and decide you’re not worth consulting about whether a show is any good or not before they ever -get- to your reviews. Really, Yumeria rated higher than Those Who Hunt Elves? ;p)

  3. Ubu Roi says:

    Yep. Yumeria was “ok” except that they tried to tack on a dramatic ending and lost both logic and the feel of what made the series good. The pedo aspect of it was also something that gave me the heebee-jeebees.

    But TWHE was just one long, annoying trip through “somebody beat these idiots over the head” country. And I had a hugely negative reaction to the amount of violence perpetrated by Junpei against Celecia.

    Celecia and her temper were meant to be funny, but she is turned into a big joke by the writers — then spends the whole series being abused by Junpei. I ended up hating him for it. He lives to fight, eat curry, and strip elves. In that order. His only redeeming feature is his sense of honor, which doesn’t extend to allowing elf women to disrobe in modest privacy and be inspected by the women with him. No, his usual method is to just yank their clothes off in public.

    Junpei was the least likeable out of four characters, none of whom I would pick to be stranded on a desert island with.

  4. I dunno. The redhead with the gun fetish could be fun.

  5. Ubu Roi says:

    She lost me when she started snickering at Celecia, when the latter got upset over how Junpei was disrespecting her. I think I’d rather take the tank. It wouldn’t be as much use, doesn’t look nearly as nice, and is a lot harder. But it has much bigger guns and never really does anything mean either. Well, except for shooting at that mouse it was chasing, and how was the poor thing to understand it wasn’t a little kitty anymore, but a 75 ton tank?

  6. AvatarADV says:

    Elves was quite brainless fun, I thought. If anything, it was a series pretending to be a good bit more scandalous than it was – for something with a premise straight out of porn, it’s awfully tame. You can’t take it seriously – it’d be hideous – but that’s not what it’s -for-.

    Yumeria… I remember when we got it, one of my co-workers (the biggest otaku of us all, and mind yez, that’s saying a LOT) blurted out, “Oh god, we got the bad touch anime!” Seems Yumeria’s roots are in a PC game where you, er, touched little girls for encouragement – so the pedo aspect that’s giving you the heebies is actually toned down considerably, and isn’t that a pleasant thought? I do know that the people working on it would gradually grow despondent, people who you could throw a hundred episodes of Gatchaman at and they could roll with it. “Oh god, please kill me, so I don’t have to work on Yumeria any more!” The only bright moment was looking up the Hilbert Effect for liner notes (caught the Xenosaga ref, but didn’t know the science behind it.) And that happened in the first minute or so!

    Of course, a lot of this depends on the kind of mood you’re in when you catch it, especially when you’re working and don’t exactly have choice of show; Elves tended to catch me in the proper mood, just sort of happened that way. Never watched Yumeria myself, not about to. ;p

    But that’s kind of against the point. I mean, obviously it’s worth reading your opinions on a show, but if all I knew about you was the grades you’d assigned shows, I wouldn’t come away with that. And a lot of shows, well, they’re good if you like that kind of show, and only if; I mean, Shin Getter Robo was fantastic if you were looking for a stupid kill ’em all robot show, but if you weren’t specifically looking for that, it’s terrible by any sort of rational analysis.

  7. Ubu Roi says:

    TWHE just got on my bad side because of huge dangling plot loopholes and hitting one of my hotbuttons (abuse towards women… don’t ask. I’m not going there). Yumeria was actually pretty likable–except the holes in the backstory were huge, the loli sex-object angle was WAY too overt, and that teacher…. he’s actually responsible for the biggest laugh of the dub (for me), but plainly put, he’s a child molester, and that’s another hotbutton.

    Your last point is well taken, and thats why I decided on a subjectively based rating system with modifiers for objective viewing. I am still kicking it around though, and I’m not sure even double-plus and double-minus modifiers work. I may have to split the rating into two separate grades entirely.

Leave a Reply