It is still 80% even if the pool gets larger.

80% of a 10 person pool gives you an 80% chance to be picked

1 - 9/10 * 8/9 * 7/8 * 6/7 * 5/6 * 4/5 *3/4 * 2/3 = 0.80

80% of a 100 person pool gives you an 80% chance to be picked

1- 99/100 * 98/99…20/21 = 0.80

Maybe it was never explicitly stated, but the Exemplar program had been treated as a substitute for GP foils. When it was announced that GP foils were being discontinued, the general reception was “your pay is being decreased”. When exemplar was announced it appeared that “your pay will not be decreased if you continue doing exemplary work”. Now it seems that “as long as you continue doing exemplary work, you can enter a lottery in which your pay may or may not be decreased”. Am I wrong in perceiving it this way?

Originally posted by Justin Purcell:

Am I wrong in perceiving it this way?

Yes, because it's not pay, nor was it ever presented as such. Perception is powerful, to be sure, but the idea that Judge Foils are “pay” is a root cause of many challenges.

d:^D

I think that in general, these changes are fine. They are fair, while being philosophically appropriate and logically considered.

Despite this, however, I think that the plan to determine foil recipients randomly is going to cause a lot of feel-bads, and may not be the best way to achieve the Exemplar team's goals. I will echo others' desire in this thread for more transparency around this randomization process—guaranteeing first-time nominees foils and other stipulations that may or may not already be planned as part of the process should reduce those feel-bads.


Without knowing the specifics, it strikes me that two other options could have merit:

1) Reduce the number of nominations available to judges so that it is lower than the number of foil packets to be awarded. All nominees would be treated equally, and the same number of foils would be awarded as if there were more nominations with only a percentage of them receiving foils. Admittedly, this does not address the goal of Exemplar not guaranteeing foils.

2) (Increase the number of waves per year, and) select a random set of those waves to send foils to the nominees. Judges would still be encouraged to do exemplary things year-round, since they wouldn't know which waves would award foils until after they close; all nominees would receive the same things at the same times; and foils would not be guaranteed for anyone.

Perhaps these ideas have already been debated, or incorporated in the plan? If so, it might be comforting to many to hear some more details around distribution, even though WotC and the Judge Programme have no obligation to do so.

My biggest problem with the randomization is that I don't understand how it's solving the problem.

Over the last several waves, we have drifted from that assertion, and it has encouraged some undesirable behaviors, as too much emphasis is placed on foils, instead of the exceptional actions of Judges.

The emphasis will still be on the foils. Just with a whole lot more salt than before.
Much better to do away with exemplar foils altogether as others have said, or

3. Distribute packets randomly to ALL judges, with Exemplar merely increasing your chances.
Even if a recognition no longer guarantees foils, no recognition still guarantees no foils. Getting a recognition is still the only way to be eligible in the first place. To de-emphasize the foils we should do away with that as well!
The foils are no longer Exemplar Foils but Random Foils. All judges have some chance to obtain them. Recognitions happen to increase that chance.
A recognition does not guarantee foils. No recognition does not guarantee no foils.

I'll address some of the questions regarding the randomization process.
First, I wont be sharing what the process is, or what the percentages are. As I said in the blog post, we want to de-emphasize the importance of foils within Exemplar, and giving judges a puzzle to figure out how to maximize the ‘effectiveness’ of their Exemplar nominations works against that objective.

However, to share a bit more.

A lot of eyes have been on the rules. The RCs, the PCs, some of the Judge Apps developers, and an Exemplar tiger team that helped flush them out originally. These are smart people who think a lot like you do and can identify the same issues you can. Nearly* every concern I've seen brought up in this thread was either already addressed prior to the review or addressed during the review.
The Apps developers are setting up to test runs of the rules so we can confirm that nothing whacky happens either, so we can get unpredicted problems fixed before we start applying the rules.

Do I realize that I'm essentially saying “trust me”? Yes. Is that a big ask? I hope not. But I realize that's a bridge too far for some people, and that's why I asked Damian to look over my shoulder challenge the results.

Hope this helps.
-Bryan

*Nearly is a CYA. I would say ‘all’ but there's a chance I missed one.

Why wouldn't you just do the randomized removal of packets AFTER the first? If you get an exemplar, you should always get some foils.

Originally posted by Steve Guillerm:

The foils are in fact the award, and it's disappointing that we have to dance around this self-evident fact.

This is absolutely the case. I understand that the recognition is meant to be the main thing but the truth of the matter is that that recognition isn't (from what I've seen) even what the people writing the exemplars see it as. It's a way to reward people being a good judge with a prize - namely, the foils. (Note that what the judges receiving the foils think isn't even the issue there)

If recognition was really what the program was about then people saying “Good job” at a GP or something would be enough.

Edited Derek Heinbach (Dec. 2, 2017 09:28:01 AM)

I agree, exemplar is and always has been about the foils.

We already had a system for favourable reviews, but it was felt that nobody cared enough about it. Now we’ve got two review systems for no reason, to pretend otherwise is disingenuous.

after thorough read here are some of my thoughts about the changes and EP:
1) the “token of gratitude” is an amazing idea of giving out something worthy and personal at the same time. Thanks for that!
2) the randomization idea connected with distributing judge foils is not good imo. please either remove foils from EP entirely or hire more people to read and evaluate the nominations more carefully. what REALLY creates a bad feeling is to read something like “thanks for driving me home after the tournament”, knowing this has BEEN ACCEPTED as a exemplar behaviour. also judges abusing EP by repeatedly sending “problematic” nominations shall be warned and then removed from the program.
3) not really sure why you are still omitting L1 judges from the EP (as nominating people). My distribution would be following: Each wave L1 gets two slots, L2 gets four slots. Or similar. Forcing L1s “you have to try to push to L2, if you want to nominate someone” feels wrong to me. Many L1s are TO's and spot a lot of exemplar behaviour. Asking them to either Advance or write to RC looks in my eyes as if they were disabled. They aren't.
Thanks for reading!

Hey guys

Just my 2 cents - let us assume that there are a certain number of
foils allocated to the EP. Let us also assume they are fixed at a
certain amount*

Lets also trust that this number is not decreasing overall

You can either give a judge a foil pack or not, which is a problem for
a number of reasons.

you are already working with forced whole numbers on recognitions.
making this match up with whole numbers on foil packs is just crazy,
and realistically the only way you can achieve it is by under
allocating foils to make the maths work

That is BAD

Also it's going to place artificial limits on how you have to allocate
recognition slots

They are simply smoothing out the curve to fix the whole number
problem and make it less of a binary yes/no answer. And too be honest
I think the whole number problem is what has lead to the problems
perceived in exemplar in the first place


TL:dnr whole number maths really doesn't work in a something like EP -
you need to smooth it out and this is a pretty good way of doing it


Roger :)

Originally posted by Bryan Prillaman:

First, I wont be sharing what the process is, or what the percentages are. As I said in the blog post, we want to de-emphasize the importance of foils within Exemplar, and giving judges a puzzle to figure out how to maximize the ‘effectiveness’ of their Exemplar nominations works against that objective.

Hi Bryan,

Are you permitted/willing to say whether the process is intended to remain largely the same wave to wave (i.e. restricted to updating/improving), or change deliberately from wave to wave?

- If the process remains largely the same wave to wave, do you believe that it will be sufficiently complex yet not too specific so that zealous judges will be unable to “figure out the puzzle”?
- If the process intentionally changes from wave to wave, will the process for each wave be revealed after it closes?

Also, does the amount of criticism/concern you are currently seeing from judges approach what was anticipated?

<edited verb tense to agree>

Edited David Poon (Dec. 2, 2017 02:44:26 PM)

The only way to de-emphasise foils from Exemplar is to remove them completely, anything else just inserts more problems.

We can bleat about foils until we're blue in the face, but anyone that says they are not used as a marker for the ‘value’ of an Exemplar recognition is being disingenuous and simply missing the point that for most judges Exemplar foils make up the vast majority of their judge income (whether Scott, the PCs, WotC etc say otherwise is frankly irrelevant).

What this iteration of the Exemplar Program does is add in the feel-bad moment of one person's Exemplary behaviour being ‘valued’ less than others - judge A, your contribution to the development of the program and yourself is exemplary and here's $500 as a token of our appreciation. Judge B, you did the same thing in a different place to judge B but we in the program would like to say here's a token and some fuzzy feelings from being nominated. Does judge B have a reason to be disappointed?

It baffles me that those with decision making power in this area continue to think that repeating mantras about foils and value and recognitions being about the feel-goods and not about the money will actually make a difference to the real life interpretation of the situation.

I'm not quite sure how randomly removing foils from certain people will remove “too much emphasis is placed on foils”. It feels like it'll just add even more feelbad and if the fear is people recognizing others to give foils to their friends, it'll keep contributing to that as each recognition still is a higher % chance for them to get some (and if it isn't, it definitely won't be clear enough without showing the actual algorithm).

The idea of offering only one pack no matter the amount of recognitions sure seems more attracting. To balance that sphere leaders, program coordinators, regional coordinators and conferences could offer more support to those participating in each corresponding field.

Obviously I don't expect any change for Wave 12, as a matter of fact I'm all in for a wave of tests to see what happens. But, if the results aren't as good as expected, I hope the feedback might help the Exemplar Team and those involved to consider a change for Wave 13.

Originally posted by Justin Purcell:

Maybe it was never explicitly stated, but the Exemplar program had been treated as a substitute for GP foils. When it was announced that GP foils were being discontinued, the general reception was “your pay is being decreased”. When exemplar was announced it appeared that “your pay will not be decreased if you continue doing exemplary work”. Now it seems that “as long as you continue doing exemplary work, you can enter a lottery in which your pay may or may not be decreased”. Am I wrong in perceiving it this way?

You are perceiving it wrong, or at least, if it was the general perception in some areas, it wasn't based on facts. When Foils were discontinued from GPs, compensation from the TOs increased to compensate. In the time immediately following, many GPs were the highest compensated ever. There were a few GPs where I came home with multiple cases of product. Then TOs started pulling back the compensation. Some gradually, some abrubtly.

But, GP compensation was GP compensation, and Exemplar was something new and different and unrelated to GPs at all. It was ‘extra’.

Originally posted by Bryan Prillaman:

You are perceiving it wrong, or at least, if it was the general perception in some areas, it wasn't based on facts. When Foils were discontinued from GPs, compensation from the TOs increased to compensate. In the time immediately following, many GPs were the highest compensated ever. There were a few GPs where I came home with multiple cases of product. Then TOs started pulling back the compensation. Some gradually, some abrubtly.

But, GP compensation was GP compensation, and Exemplar was something new and different and unrelated to GPs at all. It was ‘extra’.

I understand that's how you see it, but for a lot of judges it's relevant. I live in a country where judges have almost no work, and when they do, the compensation is poor. Not all of our judges are L2s (we have no L3), and for those who are, not everyone can go to GPs.

For most of my judges, one exemplar pack is worth more than a year's compensation.
As far as we're concerned, you just cut our lifeline.