Return to Writings
What Else? The Endless List of Flaws in the IJS
by Sonia Bianchetti
The results of the men's event at the 2008 US Nationals generated a lot of
discussion on the internet and the media. The topic was: Who really won
the men's title? Was the men's title decided by a faulty interpretation
of the calculation method?
More importantly, is the software of ICECALC—the software studied and
imposed by the International Skating Union to calculate the results in all
championships and international competitions—the best one from a
mathematical point of view? According to George Rossano, Jack Curtis, Ed
Russell and a few other mathematicians that I contacted, it would not seem
Rossano is of the opinion that "intermediate rounding steps used in the
ISU calculation method, is incorrect and indicates a basic lack of
understanding of significant digits and how they affect the precision of
the calculation for final results" and that "ISU rule 353 should be
revised to require an exact calculation to at least three decimal places,
with rounding of the total score only as the last step of the method"
Jack Curtis wrote: "Good math practice does not round off during
calculations, doing so only at the end number after all intermediate steps
are accomplished using all the places. That principle is violated in the
ISU software and in the Canadian proposal. It should be evident that the
more often you round off during intermediate steps of calculation, the
less accurate the final result will be. In IJS at present, this can
sometimes change the outcome placements, which occurred at US Nationals in
the men's event."
According to Russell: "It appears that IceCalc is the standard and the
rules don't matter. The rules are clear: if you are NOT instructed to
round, you SHOULD NOT ROUND. It's not an ambiguity of instructions.
I'm not instructed to put water in my gas tank, being intelligent, I
don't. IceCalc put water in their gas tank—they should have to FIX
Of course, whatever the outcome of this analysis may be, it will not
affect the results of the U.S. Nationals because the title to Evan Lysacek
was correctly awarded based on the existing tie-break rules, good or bad
as they may be. Still, the issue is very intriguing and perhaps some
clarification should be provided by the ISU experts themselves!
Mathematics—to my knowledge—is recognised as being an "exact
science." There cannot be different opinions on which is the system
that provides the fairest results for the competitors. If even math is
questionable in the IJS, what is left? This controversy only added one
more question mark about the validity of the IJS.
The new judging system introduced in 2004—as is well known—was
presented as "the solution to all figure skating judging problems."
Its main purpose was to make judging more fair and objective. Whether it
reached its goal is still to be proven. What is sure is that it made the
judging incomprehensible and inconsistent while killing the artistry, the
creativity and the popularity of the sport. And this is not a personal
opinion, it is supported by facts!
After the 2002 Olympic scandal in which the French judge, Marie Reine Le
Gougne, said she fixed the results in the pairs competition in a deal with
the Russians, the International Skating Union leaders decided to remedy
the situation by giving the judges anonymity. It cannot be just a
coincidence that TV ratings have gone down hill ever since. The sport's
new judging system is perhaps fairer to the skaters—at least as far as
the technical score is concerned—but remains indecipherable to the
viewing public. Whether there's a cause-and-effect here is only a guess,
although it wouldn't be a bad idea for the ISU leadership to at least
consider that possibility before the last figure skating fan turns off the
TV and leaves the room.
All over the world, the arenas where the most prestigious events are held
are dramatically empty, and—even worse—today TV money is disappearing.
In 1999, ABC signed a five-year contract with the International Skating
Union for $22 million a year. Today, they can't find a U.S. network
willing to come close to the current $5 million a year contract with ESPN
which expires after the 2008 Worlds. The International Skating Union's
search for a new contract has so far been unsuccessful.
Ottavio Cinquanta—the ISU President—must really be desperate if he
told the Chicago Tribune last month he would give away the rights if
necessary to assure a U.S. telecast of the 2009 World Championships in Los
Angeles! What will happen from next year? How will they face all the
expenses, subsidize the ISU Championships both in figure and speed skating
and balance their budget? Without income, the Union very likely will have
to dip into its reserve money. How long will it last?
In the United States the skating federation is faced with the same
problem. Once one of the most popular sports for American audiences,
skating ratings have been sinking from double figures into the 6.0 range.
After making $12 million a year with its old TV deal, the USFSA is now
receiving absolutely no rights fee with its new TV deal, and State Farm—
their main sponsor during the past 12 years—decided not to renew its
Many are the reasons which caused this decline in the fans' interest.
Surely one is that the sport has become boring, totally uninteresting,
with all the programs looking alike. Johnny Weir—in a recent
interview—said: "I like to make an impression, I like to make
art. With the new
judging system, it's taking away the individuality of the sport. At the
moment, it's looking a little bit like tennis, just hitting back and forth
and nothing changes. That's a shame to me." Exactly the same feelings
were expressed by Stephane Lambiel and Evan Lysacek after the Grand Prix
Final in Torino.
Technique in our sport is essential. Without technique there is no way a
skater can express himself, can make "art", but if one has nothing to say
or is not allowed to express his "art", then even the best technique is
worthless. As Philip Hersh wrote in an article published in The Los
Angeles Times—February 3 issue, "Figure skating sadly has become a
mind-numbing mathematical exercise, where artistry is only paint by
I have addressed this matter many times and I do not intend to do it once
more here. What I will do is to deal with the problems related to judging
If after four years of intense training through judges' seminars, judging
is still considered "unsatisfactory", perhaps the blame should not be
thrown just on the judges. I cannot believe that several hundred judges,
all over the world, have—all of a sudden—become incompetent, ignorant
OR STUPID! I do sympathize with the judges who are blamed for their
unsatisfactory judging because it is not their fault. It is the fault of
Besides the mathematics of the calculating system, the ISU should consider
whether secret judging, the random draw of the judges or the replacement
of "relative judging" with "absolute judging" were the best ideas. It
remains to be proven whether the judging system has produced more
objective and reproducible results in a competition and if it really
prevents collusion or other forms of dishonest judging, which was its
One of the greatest harms done by the ISU to their sport was adopting an
anonymous judging system as a response to the 2002 Salt Lake City Olympic
judging scandal, and eliminating the universally recognised trade mark of
the sport: the 5.8's, 5.9's and 6.0's.
In a recent interview quoted in the January 24 issue of USA TODAY, Brian
Boitano said: "The 6.0 needs to come back. People have to be able to
root, they have to be able to understand the scoring system. Round up, add
it up, multiply by something, do whatever it takes—but it all needs to
get down to a number people know, either a 6.0 or a 10."
The sport also must once again allow to identify the judges with their
scores. Audrey Weisiger, a top U.S. coach, added, "Transparency is
essential, but the audience also needs to feel that it is participating,
too. You used to be able to boo the judge from Bulgaria. Now the audience
feels cheated. You can't have a good-guy, bad-guy feeling anymore. I kind
of miss it myself."
Every single judge could be cheating now and the public will never know.
This is unfair or an "insult" to the skaters and to the public, as Dick
Pound—the then Vice President of the IOC—declared, but it is also
unfair to the honourable judges who are lumped together with those who
cheat. As a matter of fact, secret judging is perceived as a way to hide
intrigues or deals among the judges and is detrimental to the credibility
of the sport. Judges must be held accountable not only by the ISU Council
or Technical Committees but also to the athletes for their results.
And what about ethics? "Transparency" is considered fundamental to the
ethical functioning of any institution. All decisions must be open and
subject to public scrutiny. The anonymity of the new judging system is an
evident violation of this principle.
Besides, the practice of penalizing a judge whose marks diverge too widely
from those of other judges is just outrageous! There is a well documented
tendency for people to try to bring their views, opinions, decisions in
line with what they feel is expected of them. Not only does the ISU
not minimize the tendency to self-censorship, they actually
maximize it through the practice of penalizing judges for failing
to go along with the crowd. This practice is widely recognized by
professional ethicists as one of the main sources of unethical behavior.
The random draw of the judges
Studies have proved that there is a wide spread of marks and consequent
placements among the judges, even among the top five competitors. In each
event, depending on which judges have been selected, the result could
easily vary from first to fifth. For instance, it's quite likely that
in close competitions there may be two or more different potential winners
depending on which judges are dropped at random. Is it right that the
winner should be determined by a coin toss? That is not sport, it's
gambling and is most unfair to the skaters. It reduces the whole results
system to farce.
The random draw is another flaw of the system, especially when it does not
guarantee a fairer result but is only used to make secret judging even
more secret! The only way to compensate for that is by using the marks of
all the judges on the panels, deleting the highest and the lowest.
At the heart of the new judging system there was a fundamental change in
the method of evaluating skating performances. The former ordinal method
of scoring was based on the recognition that humans can make relative
judgements with greater precision than absolute judgements.
Under the 6.0 system the judges evaluated the performances of the skaters
by comparing one with another, using the so-called "relative marking."
This skater is better, the mark must go up. How much? The only way to be
consistent through the whole event is to be thinking through the whole
event whether the marks given now make sense compared to the marks given
before—and that is a comparison.
Under the IJS the judges are now asked to evaluate performances on an
absolute point scale without comparison to any other performance. It is
questionable if this point base approach to scoring can ever work
If you are expecting judges to focus on evaluating individual elements of
a program, you cannot expect them to evaluate the composition and
presentation of that program. These are entirely separate ways of
thinking. It is another flaw of the system.
A great deal of research has been conducted into the marking skills of
human beings in general. It has been found that when marking on an
absolute standard, over a period of time, marking will drift quite
considerably. Absolute marking, in general—but even more so in a sport
like figure skating—is considered inappropriate. It is practically
impossible to quantify objectively the quality of any element of a
skater's performance. Marking by comparison tends to be more stable.
This is even more true with the Program Components scores where the
nebulous nature of their terminology allows for a wide range of
interpretation from the judges. What the judges do not have is a procedure
to insure that a consistent standard is applied throughout an event. What
we used to say and to hear in every judges' school and at every
competition—"Don't forget the first warm-up group!"—has now
been replaced by "Judge in the moment!" If a person is asked to
judge in the moment, and no guidance is included to insure consistency of
judgment, obviously the judging will suffer.
Each of the program components scores has a range of possible values from
zero to ten. To decide which mark to assign to each of these elements one
should know exactly what is to be considered as "perfect" deserving ten!
Does anybody know for instance how "Performance/Execution" or "Skating
Skills" should look to deserve 10? Has anyone ever seen a precise and
detailed definition of these or other elements?
Or, as George Rossano points out in his excellent article "Scoring
Drift," in ice dancing what does "Pattern and Ice Coverage" mean? There
are no explicit tables of examples, nor pictures or diagrams for what
constitutes good or bad ice coverage. There are no tables of values to
say what to do with your marks. The judges have very little definite
guidance for what marks to give, and are forbidden from comparing the
marks they gave at the beginning of an event or to a previous skater. How
possibly can a judge assess a correct mark? And on which basis can a
certain mark be considered right or wrong?
The purpose of a figure skating competition is to determine which skater
gave the best performance on a given day. How can one accurately rank
skaters without seeing what he gave a previous skater? The way our minds
are, we can't tell if something is 2 inches long or 3 inches long, but we
can compare two things and say which one is longer. So "absolute" judging
makes no sense, especially in program components, both because it's
impossible for the human mind to know whether something is worth 4 or a 5,
and because a competition is supposed to compare things—and we should
take advantage of that.
Without the help of explicit guidelines and a self consistent process it
is no wonder the Program Components marks increase significantly during an
event and reflect more the reputation of the competitor than the actual
We can expect there will be additional problems in judging if the next
Ordinary Congress approves proposals submitted by the ISU Sport
Directorate concerning a new way of determining the starting order, both
in free skating and the short program, in all Championships.
For the short program it is proposed that the competitors be divided in
two groups according to their ranking. The competitors in the group
"skating earlier" will then be divided into two parts: "competitors
with and without ranking, so that competitors with ranking will draw for
the later starting numbers." The reason for this reads: "To avoid
the situation where competitors without ranking skate before competitors
Why such discrimination?
And for free skating: "Each group shall skate in the reverse order of
the previous segment with the best placed competitor/pair skating
This sounds like an invitation to a wedding party for the judges! They
cannot help but be influenced by the order in which the competitors
skated. How badly will this affect the final results?
The "personal best performance" scores that are so highly publicized are
laughable when you look back at tapes of far better performances by those
skaters. Is this due to rule changes? Marking drift? Different judges and
specialists? The inconsistency in the judging makes the new judging
system a farce as well as incomprehensible.
The bottom line is that the judges have no ability to consistently score
the skaters because the system puts insurmountable obstacles in their way
and makes it impossible for them to do a good job. The judges may be able
to accurately tell that a certain skater should be ranked in a certain
way, but their ability to reflect that knowledge in their marks is
completely taken away from them. That is probably why at the Olympics and
the Worlds last year the ordinals were all over the place.