Note: This essay was written a little over a week and a half ago. Between the time I wrote it and today (as I post it), Wizards of the Coast has released errata for 4th Edition which corrects some (but not all) of the problems described below.
Since this essay still accurately describes my playtesting experience and serves as an apt critique of the rules as they were published, I have chosen not to rewrite it. However, I have added an Errata Addendum to the end of the essay discussing the changes that were made in the errata.
I’ve also talked about skill challenges before. Having completed my playtesting, here are my current thoughts on the matter:
(1) Skill challenges in their most general form are unusable as written because they’re so heavily dissociative. They are fundamentally disconnected from the game world (caring not about what the PCs have done, but merely how much they have done) and create strangely skewing probabilities, among other problems.
(2) Skill challenges in the specific form described in the 4th Edition Dungeon Master’s Guide are unusable because they mandate railroading. If you follow the rules in the DMG you are supposed to (a) write a script for the PCs to follow; (b) tell them the script; and (c) if they try to deviate from the script, punish them for it with more difficult skill checks.
(3) Skill challenges are unusable because their probability is wacky. I’m not going to delve into the maths, but basically what it boils down to is that a 65% chance of success on a skill check is the watershed: If you have less than a 65% chance of succeeding at the skill checks making up a skill challenge, your chance of success on any skill challenge is very small and shrinks rapidly towards the essentially non-existent as you increase the complexity of the skill challenge.
If your chance of success is exactly 65%, then increasing the complexity of the skill challenge is virtually irrelevant (even though it’s supposed to be getting more difficult). And if your chance of success is larger than 65%, then skill challenges actually get easier as the complexity increases (when it’s supposed to be getting harder).
This is obviously not working properly. And, when you run the actual numbers of the system, you discover that the PCs generally have about a 10-20% chance of succeeding on a skill challenge designed for their level.
(Here’s a fix for the probability issues that looks pretty good to me as I glance over it. The author has also done some interesting things in terms of adding some depth to the gameplay of skill challenges. I haven’t fully delved into it, but it looks like it’s worth checking out. Note that, while this fixes the wacky probabilities of WotC’s skill challenges, it doesn’t address the emergent probability skewing which is an inherent characteristic of the dissociation arising from open skill challenges.)
(4) Even if you fix the probability, skill challenges are surprisingly boring in actual gameplay.
In the best case scenario, skill challenges simply duplicate the gameplay of previous editions: The players propose a course of action, the DM determines the skill and the DC, and then a check is made to determine success. In this scenario you’re tracking a bunch of extra numbers and suffering from the inherent dissociation of the system, but you’re not actually gaining any sort of reward for your effort.
In the worst case scenario, skill challenges turn one interesting die roll into six to ten monotonous die rolls. (And you’re still tracking the extra numbers and suffering from the inherent dissociation of the system.)
(5) The only potential benefit you gain from using the skill challenge system is that it gives you a structure for rewarding XP. But the wacky probabilities alone assures that this “system” is just as likely to erroneously give you a larger reward for an easier challenge.
Here’s another example of this “system” in action, from pg. 73 of the DMG: “If you use easy DCs, reduce the level of the challenge by one. If you use hard DCs, increase the levelof the challenge by two.”
When we look at the table for DCs by Level on pg. 42 of the DMG, we can quickly see that this is complete nonsense. For example, at 10th level the values are easy DC 17, moderate DC 21, hard DC 25. The guideline is claiming that if you take a 10th level challenge with moderate DCs and redesign it with easy DCs, you should end up with something equivalent to a 9th level challenge with moderate DCs. But you don’t. At 9th level, the moderate DC is 19, not 17. A 10th level challenge with easy DCs is, in fact, equivalent to a 6th level challenge.
Similarly, a 10th level challenge with hard DC 25 is not equivalent to a 12th level challenge with moderate DCs. It’s actually the equivalent of a 20th level skill challenge.
The “difference” between a 10th level skill challenge and a 12th level skill challenge actually reveals the complete absurdity of this “system”. That’s because there isn’t one. The DCs by Level table on pg. 42 of the DMG assigns the same values to every 3 levels. So levels 10-12 are all grouped together and have the same DCs for skill checks. Despite the fact that they’re identical in every way, a 10th level skill challenge with complexity 3 only rewards 1500 XP whereas a 12th level skill challenge with complexity 3 rewards 2100 XP.
This “system” is worse than useless. It’s literally just generating random noise and isn’t worth the paper it’s printed on.
(6) I still think there’s some real potential in the basic concept of social skill challenges. But the extant system isn’t even a stepping stone towards achieving that: You basically want to throw out everything the DMG has to say about skill challenges and start over from scratch.
(7) I also discovered some interesting uses for the basic concept of skill challenges in structuring cooperative disabling of dynamic traps. Once again, however, this requires you to scrap everything the DMG has to say about using traps and skill challenges before rebuilding the system from scratch.
What it really boils down in the final analysis is that complex skill checks are a useful mechanic. In other words, when you have a specific task defined by a concrete goal and a single method of success — such as disabling a trap, disarming a bomb, or playing a game of Chess — that is best modeled as a sequence of discrete actions, the basic formula of X successes before Y failures is a useful way of representing that mechanically. Even the S-curve probability distribution works well for these types of scenarios (it becomes a feature instead of a bug as skill trumps luck in larger and more complex tasks).
You can even get away with generalizing this to some extent: For example, you can use this structure to say that you can disable a magical trap by making Arcana checks, Thievery checks, or by dealing damage to the structure of the trap. By allowing these disparate checks to all feed into a single complex skill check, you facilitate cooperation in a way that’s far more dynamic and interesting than just using the Aid Another action.
But the skill challenge “system” as it’s presented in the DMG? Dissociative, broken, and useless. Don’t waste your time.
ERRATA ADDENDUM
In response to the general public outcry over the shoddy and unusable skill challenge mechanics published in the DMG, WotC responded in mid-July with errata aimed at correcting some of the more egregious problems with skill challenges. I’m going to take a few moments here to take a second look at the problems with the skill challenge mechanics and analyze how they were (or weren’t) corrected.
DISSOCIATION: Nothing was done to correct the heavily dissociated nature of the skill challenge system.
PROBLEMS WITH PROBABILITY: The errata corrected the most egregious and obvious of the probability problems with skill challenges. Notably, more complex skill challenges no longer become easier for people with higher skill modifiers. However, the probability of success still varies radically as you move away from the baseline values assumed at each level. This means that min-maxing is heavily rewarded. It also means that, rather than encouraging the participation of everyone at the gaming table (the purported design goal of skill challenges), the system instead rewards the group for figuring out whoever has the highest applicable skill modifier and then having that character roll all the checks.
(This means that skill challenges are yet another example of 4th Edition providing a “solution” to a “problem” which actually ends up making the problem worse rather than better. Brilliant.)
This probability pattern also means that tackling a skill challenge a couple levels higher than your current level is much more difficult than tackling a combat encounter a couple levels higher than your current level (and vice versa).
However, with all that being said, the emergent probability skews of the system (which result from the possibility of multiple paths of succcess and the dissociated nature of the mechanic) still remain.
EXPERIENCE AWARDS: They partially fixed their inability to perform simple arithmetic by removing the XP guidelines based on using Easy vs. Moderate vs. Hard DCs. Instead, you just vary the level of the challenge to make it easier or harder. However, this ignores the fact that there remains a significant difference between a skill challenge which features Easy DCs for a given level versus a skill challenge which features Hard DCs for a given level. (Nor are any solid guidelines given for the proporion of Easy vs. Moderate vs. Hard DCs you should be using.)
They also fixed the discrepancy where, for example, 10th level and 12th level skill challenges were statistically identical but had significantly different rewards by simply limiting skill challenges to the mid-point of each level range. (So, for you example, you can have 11th level skill challenges, but not 10th or 12th level skill challenges.)
RAILROADING: They have removed all of the rules requiring the DM to railroad their players. This is excellent news, and since I was (AFAIK) the first person to post these concerns online (both here and at WotC’s messageboards) I feel like I actively contributed to having these pernicious passages removed from the rules.
SLOPPY DESIGN: Skill challenges are essentially one of the core mechanics of 4th Edition. And they royally screwed them up. I’m glad to see that they’re issuing corrections in a timely fashion, but it doesn’t exactly instill a lot of confidence in me that they so fundamentally screwed up the most basic balancing of a core mechanic like this. What does their complete failure here say about any kind of complex interactions in the system?
CASCADING EFFECTS: Because skill challenges are a core mechanic, they’re used extensively throughout the system. For example, they’re a major element in the design of many traps. Despite this fact, the current errata doesn’t correct the design of these traps to match the revised skill challenge guidelines.
DESIGN DISCONNECTS: On June 14th, Mike Mearls stated: “The system went through several permutations as we worked on it, and I think there are some disconnects between the final text, our intentions, and how playtesters and internal designers use skill challenges.”
Clearly.
What I find interesting is the evidence of this disconnect that we have now seen strewn around the handful of books WotC has published for 4th Edition to date. For example, the skill challenges presented in H1: Keep on the Shadowfell don’t match the guidelines found in the DMG nor in the errata. And the skill challenges in H2: Thunderspire Labyrinth? They don’t match the DMG, the errata, or the skill challenges found in Keep on the Shadowfell.
That means that we have seen literally four different iterations of the skill challenge mechanics coming out of WotC.
This is, frankly, bizarre. And it speaks, again, to the fundamentally (and inexplicably) sloppy design of 4th Edition.
USABILITY: It should be noted that the errata itself is fairly unusable in its published form. I know it’s standard practice in WotC’s errata to simply include the relevant changes, but in this case the changes are of a nature which makes neither the rulebook nor the errata usable.
Notably, the revision of the skill challenge mechanics also included a revision of the Difficulty Class and Damage by Level table on pg. 42 of the DMG. For those of you unfamiliar with 4th Edition, this table is the heart and soul of the system. I don’t think there’s been a table so crucial to the playing of D&D since the hit tables in AD&D1 were replaced with THAC0. And it’s been rendered unusable by the errata… which only replicates the three key columns which have been altered (without the other columns which give the information in those columns any relevance).
And since they didn’t get this problem fixed before they printed the Dungeon Master’s Screen for 4th Edition… well, that won’t fix your problem, either. You’ll need to recreate the table yourself by combining the information from the DMG and the errata.
Sloppy, sloppy, sloppy.
THE BOTTOM LINE: For me, the bottom line hasn’t changed much. Skill challenges are still dissociative, (slightly less) broken, and useless.
