Controversial post - points are a useful lie.

Topics and polls that cover the overall marching band activity

Moderators: Trumpet Man 05, malletphreak, Hostrauser, instrumental director

PSM
New Recruit
New Recruit
Posts: 33
Joined: Tue Oct 27, 2015 12:47 pm

Controversial post - points are a useful lie.

Post by PSM » Sun Nov 01, 2015 10:37 pm

So, I’ll just go ahead and put this out there. There’s SO much talk about scores and points and numbers in this activity. I have to wonder – why does anyone care what these judges think?

I visited one of the more successful SoCal bands a few years ago – I guess it was on a Monday following a performance, and the director seemed very pleased with, in fact almost obsessed with, the numbers his band was putting up. In fact it was all he could talk about to his kids – this number, that number, all the scores and rankings and all that junk. Like, dude. You’re going to be at championships, and you’re going to be in the finals, and you’re going to be in the top 5. Shut up about your numbers already. And it really wasn’t all that much different at a follow-up visit several months later.

Nowhere was there any mention of how the performance could have been improved, or where they did well and where they did less so, or how to address these things and why to approach certain performance challenges or practice/rehearsal objectives in particular ways – just numbers. It was all he could talk about, and I thought it was really bizarre. I know he did the work, and so did his band, because they play and move really, really well, but everything seemed to be about getting more points.

These scores. How on earth can anyone take these things seriously? If you’ve got two bands, and one wins by 0.1 points, or 0.5 points, or even 2 points, how is that not just statistical uncertainty? How is an 85.6 any different – qualitatively – from an 85.7? Hell, how is an 86 any different from an 88? Not only that, but a single judge can deep-six a band if he wanted to, just because of the way the point system works – maybe it's unlikely to happen, but the fact that this is possible shows that this system of ranking is broken.

If you don't know how it works, the judges basically evaluate the band and then assign a number that describes how well the band did within some particular category - like individual musicianship (which apparently means, are individual players handling their parts well). A bunch of these numbers for different categories get added up and that determines your score, which then determines your ranking. But in an entirely subjective field, and judging panels with zero or minimal knowledge of the works they are evaluating, is a fine-grained evaluation system like this really the way to go?

The fact is, this ranking system is basically nothing more than throwing numbers at a wall within some very loose parameters and then crunching them in some way. I wouldn't mind so much if they didn't then pretend that the numbers imply some sort of objectivity and therefore validity to the process.

The whole BAND XYZ WINS SWEEPSTAKES BY 0.05 POINTS! thing is misleading at best and a lie at worst. I think we need to rethink the way we evaluate bands and rank them in competition, and we can start by eliminating the concept of scoring.

It’s a detriment to the activity that people chain themselves to this sort of thing.

wjs05
Rookie
Rookie
Posts: 201
Joined: Fri Jun 10, 2005 3:52 pm

Re: Controversial post - points are a useful lie.

Post by wjs05 » Mon Nov 02, 2015 1:08 am

I really loved reading your post. People get caught up in the numbers game. It's sort of a human thing to rank stuff.

A crazy fact about me: I used to arrange my Blu-ray collection by years and the RANKING in which I liked them. I've since scrambled everything to get rid of my numbers obsession and just enjoy the art of film.

Blue Devils 2014: felliniesque: 99.65. I've seen it dozens of times and cannot really tell what a 99 score actually means in the context of the other corps.

Anyways, I hope you stick around this forum. It's been a while since anybody attempted to bring some form of deep thought about the activity without the old school moderation police that reduced this place to what it is now.

User avatar
Bandmaster
Site Admin
Site Admin
Posts: 1716
Joined: Tue Oct 08, 2002 12:41 am
Location: Upland, CA
Contact:

Re: Controversial post - points are a useful lie.

Post by Bandmaster » Mon Nov 02, 2015 11:56 am

wjs05 wrote: Anyways, I hope you stick around this forum. It's been a while since anybody attempted to bring some form of deep thought about the activity without the old school moderation police that reduced this place to what it is now.
Hey... I take exception with that! :shock: :wink: :lol:

We used to get many great discussions here, and for the most part the moderators only jumped in when things got too emotional and feelings were getting bruised. The biggest thing to hurt our traffic has been Facebook... But there you only get to read what your "friends" post, here you can see what everybody writes. 8-)

BTW, the site stats show that we still getting about the same number of visits as we did 5 to 7 years ago or more. Only people just don't post as much. But the readership is still strong, so don't be shy... post away!
Dave Schaafsma
Pageantry Webmaster

PSM
New Recruit
New Recruit
Posts: 33
Joined: Tue Oct 27, 2015 12:47 pm

Re: Controversial post - points are a useful lie.

Post by PSM » Mon Nov 02, 2015 12:24 pm

Hey, glad you liked it.

It does seem a bit slow around here. I guess I wasn't controversial enough with this post - it hasn't really provoked much of a response. As to Blue Devils 2014... well, DCI judging has always been screwy. All they have to do is ditch points entirely, set up the panel with an instant runoff voting system for any local show. You use the season results to set up a tournament bracket for finals, and it'll lead to the top four groups or whatever.

I don't see what's so hard about this. The point system's only useful function, IMO, is that it serves as a convenient justification for the results that were probably going to happen anyway. In the end, the reality is that this only about whether the judges collectively like Group A's performance more than they like Group B's performance. You can keep your judging captions in place and have them evaluate different aspects of the performances, but this thing with scoring is an impossible task because it pretends to be what it is not - OBJECTIVE. Really, guys, you can't imagine that the Blue Devils 2014 performance couldn't be any better? Come on.

Points need to go away - just give the groups a ranking based on who you think who did the better job in your particular caption for this particular band class. What's the downside?

I guess it takes away sweepstakes sub-trophies from large enough high school bands who can get away with murder on the field because it's easier to put out a good sound when you have 250 players and don't ever need anyone to play above poco forte and specifically discourage them from doing so, ever (SCSBOA is notorious for this and I deeply hate the fact that the judging/ranking causes directors to neuter their students in this way). Or that it might take away a visual sweeps trophy because it's much easier to hide the visual dirt when you've got 250 players because, let's face it, that's a lot of people, and you judges can't see all of it.

Meanwhile, bands with 20-100 players typically get crushed on both visual and music scores because everything that happens is way more obvious. I mean, really, isn't it pretty clear that, 99.9% of the time, the 6A band that gets "first place" really is just getting second place to whichever other 6A band wins sweeps, with a possible and extremely rare upset by a 3A guard or, more commonly, a 4A or 5A percussion ensemble? How is this a useful thing at all? And even if it is useful to indicate that a smaller band just won a caption overall, why does that have to be determined by points? Can't the judges just discuss these things afterwards to award outstanding achievement in a particular caption if it's warranted? How are these points helping anything, again?

Ranking in this activity needs a complete overhaul. And I'm not sure that I should even talk about judges, because if I haven't gotten myself into trouble yet, I certainly will do if I get into that particular conversation.

magicsax22
Rookie
Rookie
Posts: 244
Joined: Sat Nov 20, 2010 8:11 pm

Re: Controversial post - points are a useful lie.

Post by magicsax22 » Mon Nov 02, 2015 1:19 pm

PSM wrote:I know he did the work, and so did his band, because they play and move really, really well, but everything seemed to be about getting more points.

The fact is, this ranking system is basically nothing more than throwing numbers at a wall within some very loose parameters and then crunching them in some way. I wouldn't mind so much if they didn't then pretend that the numbers imply some sort of objectivity and therefore validity to the process.

It’s a detriment to the activity that people chain themselves to this sort of thing.
A few thoughts:

First, I do not disagree that getting overly attached to scores and numbers is detrimental. But most directors I know focus on what did the judges say, not just what numbers did they give. Sure numbers are a nice indicator of how well you did in comparison to other groups (a few points in a caption means really close, a few more means a decent difference, etc) but it is also important to note if the judges keep saying the same things are wrong, or if you are showing improvement.

Second thought, in most associations a 2 point score difference in the overall band score is actually a pretty significant difference. That's 20 points within the subcaptions split over multiple judges. 2 points within each other in music performance or visual effect or whatever? Barely a difference between the groups, but band x did better in that caption than band y.

Third thought, if we assume that the judges are skilled instructors in their own right (which is a requirement in all associations I know of_ and have a reference criteria they are judging against (which all associations do have...though they are slightly different) then working to get more points implies they are working to find ways to continue to improve their performance. There is no inherent problem with this. People who play sports like to improve their stats as an indicator of better performance, so do bands. Judges aren't just throwing darts and looking to give points to whomever has the prettiest uniforms...and I suggest that anyone who thinks that should download a copy of a judge form for a certain caption and sit at a show and actually try to apply the criteria and see how things would turn out. Obviously there are differences of opinion sometimes in how clean something is etc as music is subjective to a point, but if you think judges just throw darts try doing it yourself first...get deep into one of the sheets and see how it makes you look at things differently.

Just saw you put up a question about using points versus not. Some areas in other states do this, not sure how it works. I'd say that points are useful because marching band involves many different aspects and while the music judges may think one group should be first over another, that doesn't mean the visual judges do or the effect judges do, and as a result you have to have a way to weight the different captions. Different associations use different weighting based on their perception of what is more or less important. How would you do this without numbers?

User avatar
DG95
New Recruit
New Recruit
Posts: 21
Joined: Tue Nov 10, 2009 8:35 pm
Location: Shingle Springs, CA

Re: Controversial post - points are a useful lie.

Post by DG95 » Mon Nov 02, 2015 1:48 pm

To be honest, I think we can have a never ending discussion when it comes to scoring, judging, and placement. But I feel that it should be left at membership meetings for the various organizations that are bold enough to engage in the conversation.

As a music educator, I believe that the sole purpose of a band director is to ensure that every student can read music and perform well together, not to win trophies. This may come as a shocker, but not everyone can take first. And to put such emphasis behind the scoring and placing only lead to the devaluation of the actual reason why we are here: for the music.

I have had my fair share of victories, thanks to judges and the scoring system. But what really stuck with me were the fond memories of being with friends, who I still talk to. It was about being in an environment in which I felt that I belonged. These are the same memories I wish to allow future band students to develop.

In sports there are clear winners and losers, but in band, there's the opportunity for everyone to walk away feeling like a champion, after putting on a performance they worked hard to pull off. And always remember, in the end the trophy stays in the band room, but memories are what students take home with them.

User avatar
gipsonation
New Recruit
New Recruit
Posts: 20
Joined: Thu Jan 17, 2013 10:45 am
Location: Reedley, CA

Re: Controversial post - points are a useful lie.

Post by gipsonation » Mon Nov 02, 2015 2:23 pm

Specific scores help a band track progress over the course of a season and even year to year. It is also necessary for any sort of Championship event. If Championships were determined simply by place at a tournament you would have bands going to specific tournaments just because they won't have as tough of competition.

That being said, I am not of fan of what competition and scoring has done to marching band. It has led to bands playing for the judges, not the audience. Selecting music and a show for points, not for the education or entertainment that can come from it. Many modern marching band performances are well-executed, but devoid of any real music.

It would be neat to see marching band festivals in California that used a festival rating system (Superior, Excellent, etc) rather than points and rankings.

teacherken
Veteran
Veteran
Posts: 422
Joined: Wed Nov 13, 2002 9:29 am
Location: Ventura, CA
Contact:

Re: Controversial post - points are a useful lie.

Post by teacherken » Mon Nov 02, 2015 3:04 pm

[quote="gipsonation"]It would be neat to see marching band festivals in California that used a festival rating system (Superior, Excellent, etc) rather than points and rankings.[/quote]

A band can request that now in most circuits. Of course, the judges assign numbers to the performance, and then the numbers are translated to a rating (85-100 = Superior, etc.) so I'm not sure that's really going to help anything.

And I agree, the "winning" can easily overshadow the performances. This was a topic of discussion with my students after watching DCI quarterfinals. It started with the question, which composer do you like better Bach, Mozart, Tchaikovsky, or Berlin? They're all fantastic. But which one's better? That's the question I ask as the top-rated corps and bands perform. The performances in many cases are extraordinary. One being better than another is irrelevant, at least to me. But to a sports minded society, there has to be a winner, one better than another.

My students perform for the sake of performance. We do it at competitions because 1) we get feedback, 2) we're performing for a knowledgeable audience, 3) we get to see and experience other great programs. It would be far cheaper to buy a trophy than spend what we do to compete. So if that's what's important, I'll order one.

Ken

JLGORMAN
Section Leader
Section Leader
Posts: 757
Joined: Sat Jun 28, 2003 7:35 pm
Location: Killeen, TX

Re: Controversial post - points are a useful lie.

Post by JLGORMAN » Mon Nov 02, 2015 3:45 pm

as someone who comes from a Area where Ratings are the basic determination for Bands Advancing in their state Tournaments I see valid points for both Ratings and Rankings(Scores)

One advantage that Ratings have is that Bands that do not march Corps Style drill are not judged Region, District, marching Assessment, ETC under a criteria that favors primarily corps style Bands. The use of Ratings determine if a Band has met a standard of musical performance and drill no matter what type of Band they are. Texas has approximately 80 HS Bands that march military precision drills shows. They do not have pit crews and are constantly marching in precision movement. There are also in the South a great many SHOW BANDS that also do not march like Corps style Bands, but perform much like the Bands in the movie Drumline. Using ratings in these contests allow Bands to compete against a Standard and their performance is judged against that Standard. This allows Bands from different styles to perform for a Rating that shows just how well that Band meets the standards of what a Superior Band should be, be it corps Style, Show Band or Military Precision Drill Band.

One of thing that Ratings have is that in states like AR, both the Score and the Bands Ratings are announced. I have been in many events in parts of the country were ten or twelve very MEDICORE Bands performed in an event, yet one was the Champion because they had the highest score. With Ratings present, you could win a contest a be the best of a bunch of medicore Bands which would allow those selecting Bands to advance to further Contest to know exactly what the quality of the Bands were that competed in that event.

We could talk for hours about this, but it depends on what we want from our Bands. Having Bands at Festivals tends to increase the number of HS Bands performing in Marching Events. In TX 98% of the 1200 HS Band compete at District Marching Contest. The Percentage is the same in OK and AR. Any way we can get more HS Bands marching and performing we are doing what should be done
Deep in the Heart of Texas. Home of the III Mobile Armored Corps and the 1st Cavalry Division, serving America in Iraq

NAdrian7
New Recruit
New Recruit
Posts: 16
Joined: Wed Oct 07, 2015 8:36 pm

Re: Controversial post - points are a useful lie.

Post by NAdrian7 » Mon Nov 02, 2015 5:15 pm

As a musician, I advocate one thing above all others, improvement. As a musician, you want to improve upon your skills and even affinity for interpreting a composer's intended expression. I feel this whole field show era is sort of eliminating the musical mindset, rather than wanting to become better musicians, bands are trying to go for better scores. In SoCal we have a new show writing phenomenon that I like to call "Percussion and Chords" in which bands to very little playing and instead march their drill to heavy percussion.

wjs05
Rookie
Rookie
Posts: 201
Joined: Fri Jun 10, 2005 3:52 pm

Re: Controversial post - points are a useful lie.

Post by wjs05 » Mon Nov 02, 2015 11:15 pm

Bandmaster wrote:
wjs05 wrote: Anyways, I hope you stick around this forum. It's been a while since anybody attempted to bring some form of deep thought about the activity without the old school moderation police that reduced this place to what it is now.
Hey... I take exception with that! :shock: :wink: :lol:

We used to get many great discussions here, and for the most part the moderators only jumped in when things got too emotional and feelings were getting bruised. The biggest thing to hurt our traffic has been Facebook... But there you only get to read what your "friends" post, here you can see what everybody writes. 8-)

BTW, the site stats show that we still getting about the same number of visits as we did 5 to 7 years ago or more. Only people just don't post as much. But the readership is still strong, so don't be shy... post away!

I still love you and this site. :yeah: :bow: :bow: :bow: :bow: :bow:

PSM
New Recruit
New Recruit
Posts: 33
Joined: Tue Oct 27, 2015 12:47 pm

Re: Controversial post - points are a useful lie.

Post by PSM » Tue Nov 03, 2015 9:48 am

magicsax22 wrote: Just saw you put up a question about using points versus not. Some areas in other states do this, not sure how it works. I'd say that points are useful because marching band involves many different aspects and while the music judges may think one group should be first over another, that doesn't mean the visual judges do or the effect judges do, and as a result you have to have a way to weight the different captions. Different associations use different weighting based on their perception of what is more or less important. How would you do this without numbers?
Warning: wall of text to follow:

We'll use something called an Instant Runoff - sometimes called Ranked Voting, or Single Transferable Vote to determine the rankings.

Let's assume we have four judges for music - two for ensemble, one for individual, one for effect. Then three judges for visual - one each for ensemble, effect, and individual. Two more judges, one for aux and one for percussion. That's 9 judges, so we're going to get rid of the possibility of a tie. As you can see, we have 4 music judges, three for visual, and 2 for other. Each judge gets a single vote (meaning, for first place) and they rank the bands in their caption, based on their opinion or preference of which bands most successfully met or exceeded the standards of the judge's particular caption.

Let's say there are five bands in this division - Bands A, B, C, D, and E. The judges will rank them in order of preference, highest to lowest, and the votes will be published after the competition just like scores are now. This will allow bands to see where they are in comparison to others, and the comment sheets will become much more important:

Judge 1 (Mus Ens): A, C, B, E, D
Judge 2 (Mus Ens): C, A, E, B, D
Judge 3 (Mus Eff): D, E, B, A, C
Judge 4 (Mus Ind): A, B, C, E, D
Judge 5 (Vis Ens): C, B, A, D, E
Judge 6 (Vis Eff): A, C, E, D, B
Judge 7 (Vis Ens): C, A, B, D, E
Judge 8 (Perc): B, A, C, D, E
Judge 9 (Aux): C, A, E, B, D

Breaking this down into votes for bands, it looks like this:

Band: 1st, 2nd, 3rd, 4th, 5th..

A: 3, 4, 1, 1, 0
B: 1, 2, 3, 2, 1
C: 4, 2, 2, 0, 1
D: 1, 0, 0, 4, 4
E: 0, 1, 3, 2, 3

First question: did anyone get MORE THAN 50% of the judges? If any band gets a majority, then they have won the competition outright. If not, as above, then we have to go through a runoff to determine the winner. A plurality is not enough. We need a majority of judges to agree here, and we'll do it through the rankings.

It looks like there's a couple of oddball judges who are looking for something completely different here - Bands B and D only got one vote to win. This is sometimes called the "spoiler effect" (or the "Ross Perot effect") and it happens when votes that otherwise should have gone to a particular candidate wind up going elsewhere. But we're trying to eliminate this effect by requiring that to win, you need more than 50% of the committee's votes. A plurality is not enough - we want the band that the most judges can agree on.

So, what we need to do is start eliminating candidates. Who did the worst of all the bands - who got the most votes for last place? Band D got 4 votes for last place, so Band D is eliminated, and the votes are recalculated as if Band D was never in the competition at all. This is the "transferable" part of STV.

Judge 5's rankings looked like this: C, B, A, D, E.
and now they look like this: C, B, A, E.

Judge 3 was the oddball who voted for Band D to win: D, E, B, A, C
And we want to make sure his input still matters by redistributing his votes: E, B, A, C.

Once this is done, the grid looks like this:

Judge 1: A, C, B, E
Judge 2: C, A, E, B
Judge 3: E, B, A, C
Judge 4: A, B, C, E
Judge 5: C, B, A, E
Judge 6: A, C, E, B
Judge 7: C, A, B, E
Judge 8: B, A, C, E
Judge 9: C, A, E, B

And the vote tally looks like this:

Band A: 3, 4, 2, 0
Band B: 1, 3, 2, 3
Band C: 4, 2, 2, 1
Band D: 5th place
Band E: 1, 0, 3, 5 (eliminated to 4th place)

There is still not majority of voters going for band C, which is starting to look like a somewhat controversial choice among the voters - it has more votes for 3rd and 4th place than Band A, even though Band C has more first place votes than Band A. But remember, we're trying to get the band that has the most agreement among the committee.

Moving on, Band E is eliminated because it has 5 votes for last place. It is ranked 4th and the process continues:

Judge 1: A, C, B
Judge 2: C, A, B
Judge 3: B, A, C
Judge 4: A, B, C
Judge 5: C, B, A
Judge 6: A, C, B
Judge 7: C, A, B
Judge 8: B, A, C
Judge 9: C, A, B

And the vote tally looks like this:

Band A: 3, 5, 1
Band B: 2, 2, 5 (eliminated to 3rd)
Band C: 4, 2, 3
Band D: 5th place
Band E: 4th place

It's more clear here that Band C is a controversial choice. By the third round of runoffs, it's not so clear that the majority of the panel likes it more than band A. Continuing on:

Judge 1: A, C
Judge 2: C, A
Judge 3: A, C
Judge 4: A, C
Judge 5: C, A
Judge 6: A, C
Judge 7: C, A
Judge 8: A, C
Judge 9: C, A

Band A: 5, 4 (1st place)
Band B: eliminated to 3rd place
Band C: 4, 5 (eliminated to 2nd place)
Band D: eliminated to 5th place
Band E: eliminated to 4th place

Now, this all probably seems more complicated than the points system I was railing against earlier. But it's really easy to come up with an Excel spreadsheet that does the calculations for you - and ultimately, this far more clear cut. You weight committee preferences by putting more or fewer judges on a given caption, not by giving individual judges more or less voting power. You wind up with an overall winner that more judges can agree upon - as we saw here, people either loved or hated band C, and more people agreed that band A was a better choice than C.

By eliminating points, we also force judges to improve their use of tapes and comment sheets as evaluative tools, and to stop viewing numbers of points as per some arbitrary standard as being enough of an indicator to a director as to how a band is doing - 5A and 6A sheets tend to have a whole lot of nothing on them in terms of comment quality, probably because the judges often have little to say that isn't completely obvious. The point system allows them to just put down a number that will indicate how often a band meets some particular component of a caption (the box 1-5 thing, ugh.). Eliminating that crutch forces judges to do better on the important things - the comments.

By designing a voting system that accurately determines overall committee preference, we've given the results much more legitimacy because we're at one acknowledging that this is, ultimately, actually about preference. Points-based systems imply an objectivity that they cannot realistically provide. Additionally, we give no single judge overriding authority over the outcome, while ensuring that all judges have a voice at the final vote. By publishing the voting preferences, we ensure that people know the process is fairly applied, while also letting people know where they stand in particular captions when compared with a given set of competitors. We can't realistically compare scores from two different competitions at two different locations with different sets of bands and different sets of judges, but we all try to draw comparisons with this kind of stuff when there's no connection there at all, no matter how hard people try to pretend that there's consistency there. This system gets rid of all of the unfortunate side effects of the points-based approach.

Most importantly, by eliminating points, we can start designing for audiences and students again - rather than trying to gather more points by doing those things that we think judges want to see and hear.

The next thing to kill off will be captions, but that's another story for another time =)

magicsax22
Rookie
Rookie
Posts: 244
Joined: Sat Nov 20, 2010 8:11 pm

Re: Controversial post - points are a useful lie.

Post by magicsax22 » Tue Nov 03, 2015 10:22 am

First I will freely admit I have not had time to read the entirety of this proposed system. However, I feel the following point needs to be made in this thread somewhere:

Using a rubric as a criteria to assign a numeric assessment to work by a student is a standard part of being a teacher in the US. I find it immensely disappointing that so many fellow educators seem to feel that using a rubric and numbers to evaluate students is inappropriate or impossible. My question then becomes, how do you grade your classes? Do students simply get good grades for showing up? Does it matter if the kid can play or not as long as they show up? Yes there is some manner of subjectivity, just as there might be in an English class when grading essays, but that doesn't mean that it cannot be done well. Part of the reason we as music teachers (and other similar courses like Art or PE) are often seen as not mattering as much as other classes, is because of the general perception of these classes not having "real" grades. I wrote a research paper when working on my PE credential (whole other story) compiling research that suggested that not only do student skills improve when real skill assessment is used in PE (instead of getting points for "dressing out" or "participation") but so does student involvement and community perception of the subject. I can't imagine how this would be different for music.

Numbers should not be the end all be all of competing, but they are a useful tool for tracking progress and evaluating quality. The same arguments about competition lowering the quality of groups because they "play to the judges" could be made of concert festivals as well....plenty of people out there are going to play safe music because they want to make sure to get that superior rating. But what is more educational for the students, playing grade 3 music at a Superior level with little effort, or working hard to play grade 4 music at the Excellent level? We as educators have to decide what we want for our students and how we will "play the game"....but that doesn't necessarily mean the game is broken.

PSM
New Recruit
New Recruit
Posts: 33
Joined: Tue Oct 27, 2015 12:47 pm

Re: Controversial post - points are a useful lie.

Post by PSM » Tue Nov 03, 2015 11:06 am

magicsax22 wrote:First I will freely admit I have not had time to read the entirety of this proposed system. However, I feel the following point needs to be made in this thread somewhere:

Using a rubric as a criteria to assign a numeric assessment to work by a student is a standard part of being a teacher in the US. I find it immensely disappointing that so many fellow educators seem to feel that using a rubric and numbers to evaluate students is inappropriate or impossible...

Numbers should not be the end all be all of competing, but they are a useful tool for tracking progress and evaluating quality. The same arguments about competition lowering the quality of groups because they "play to the judges" could be made of concert festivals as well....plenty of people out there are going to play safe music because they want to make sure to get that superior rating. But what is more educational for the students, playing grade 3 music at a Superior level with little effort, or working hard to play grade 4 music at the Excellent level? We as educators have to decide what we want for our students and how we will "play the game"....but that doesn't necessarily mean the game is broken.
You make a good point here, and I definitely see what you're saying. But this isn't the same as a teacher giving a grade - which does need some kind of rubric to grade against, yes. As a director, you'll know your band and its strengths and weaknesses far better than any judge will. I'm not sure the comparison holds.

This is about eliminating the major problem with committee evaluation and ranking - my argument is that, broadly speaking, point systems cannot be consistent.

I'd love to run an experiment - a high quality video recording of a round of four or five bands, all in the same division in the same competitive circuit, and all recorded on the same day at the same show, will be shown to two separate committees (and ideally, committee members will never know anything about other committee members, but they should all be from the same circuit and judge along the same sheets they always use). They won't be familiar with the bands, they won't know anything about when these sets were performed or what the rankings were, nothing. And then we see how similar the scores are. We'll use those scores as a control against three other committees - and we'll tell those groups that this is an early season show, or a late season show, or a mid-season show.

I have no doubt that we'd find a very wide spread of scores for the same series of performances, simply because the committees have different information. This is what I mean by lack of consistency, and it's why the points system should be replaced with something else.

airons0678
Veteran
Veteran
Posts: 691
Joined: Wed Jul 09, 2003 9:10 pm

Re: Controversial post - points are a useful lie.

Post by airons0678 » Tue Nov 03, 2015 12:26 pm

Well, it seems like Bands of America (Music for All) has experimented with scores, no scores, and rankings, no rankings. In the swing dancing community contests are not determined by scores, but by relative placement scoring. I like the idea of ranking without numbers, but I still prefer the numbers. BOA used to announce Division, Festival-like Ratings (I, II, III, and IV) which I found highly useful, and rewarding...however, I have seen that the "rewarding" could also easily translate to "demeaning." (i.e. "Band A with a Division IV rating!" "Band B with a Division I rating!" etc.).

As for scores, I don't buy into the scores much, and I think that most "mature" band directors (those who have marched DCI or have been around for a while) "get it." Of course the science isn't perfect, particularly when you add in the subjective. I've seen time and time again Band A losing to Band B by 10 points one week in one circuit, only to beat Band B in another circuit, by who knows (who cares?) points.

I think someone else pointed it out--numbers are more useful for the judge, especially if there is a criteria reference system. BOA is very, VERY rigid about this. A score of 75 (Division I rating) at BOA is pretty darn good, because it is a score of 75 at the national standard. How do we get the CONSISTENCY of the "national standard," you may ask? Well, the judges watch tapes, they apprentice to other judges, they go through all the training. Are they perfect in a subjective world? No. But if I see a band score a 75 at a BOA contest, chances are that band is a band I would say is "good." But drilling down a little more, when you have a contest of 70-80 bands, scoring comes in handy when you have numbers to the decimal points to play with, and gauge so-and-so's band who is close to or above-and-beyond better than such-and-such's band. Sure you can do the ranking-only thing...but I know a lot of directors want feedback on how well their program stacks up not just band-to-band, but musically to standard, or visually to standard. (FYI, BOA Division ratings have not gone away. They are just not announced. You can find them in the recaps.)

In summary: Points: do they matter? Yes and no. Better to have than not to have.

Alan

Post Reply