fbpx

Judging Process

Our judging process is more involved than most competitions. The goal was to eliminate some common issues that arise during photo competition judging. Often, with the common process of ‘score averaging’, images that are well rated by several judges (i.e. all given average to good scores) rise to the top, and images that create excitement and discussion (i.e. that individual judges give top scores to, but are more polarising) are left behind. As such, some of our winning images may have been rated lower by one or more judges; but were loved and rated the highest score by other judges.

Additionally, for many competitions, the judges are not even aware of the winner nor are they included in discussion to lobby for winning images and photographers. That was not the case for our competition, and we wanted to take the time to outline our process below so that entrants can understand the process as well as decipher the scoring that can be found on their certificates.

Our judging process was completed in several stages, with the goal of filtering work into a manageable pool of images for our judges. Additionally, judges were given criteria that the founders of the competition wanted the judges to focus on, including originality, composition, and complementary editing. As such, many “very good” photographs of iconic scenes or copies of other photographers’ work did not score as well or make it as far into the competition.

It should be noted that we received a tremendous amount of incredible photography in our first year and your scores and placing should be assessed in the context of the large pool of amazing photography it sits alongside. We also want to reiterate that we will be offering free opportunities (such as webinars) for entrants to learn ways to improve their work.

Stage 1: First Pre-Judging

9,947 photographs and 368 projects

 

The four organizers of the competition, Tim Parkin, Matt Payne, Alex Nail, and Rajesh Jyothiswaran spent a week individually ranking and reassessing all of the submissions on a scale from 0 to 5, with 0 being poor and 5 being “the best of the best.” Every image that scored a single 5 and any image that scored two 4’s also went through to make sure that the selection wasn’t purely based on averages, and that subjectivity was accounted for. The goal of this phase was to eliminate photographs that we knew would not be well assessed by our main judges, mostly due to technical issues or due to not meeting the submission criteria . These individual scores were then averaged across all four organizers and any photographs that were either rated highly by at least one organizer or rated an average score of 3 or more were kept for the next stage. Of the 9,947 images submitted, 6,142 (or 62%) made it to the next stage. The same process was used for the Project submissions, and 240 or 65% made it to the next stage.

 

Stage 2: Second Pre-Judging

6,142 or 62% of photographs and 240 or 65% of projects

 

In the second stage of the competition, all four founders reassessed the remaining images to ensure that any images that stood even a small chance of winning a category or overall winner went through to the main judges. If any of the four founders thought an image should progress, it was passed onto the next category. After a week of deliberation, we had a second look through all of those that failed the cut to check if we had missed anything.

 

Stage 3: First Main Judging

3,431 or 33% of photographs and 110 or 30% of projects

 

Images and projects that made it to this stage of the competition were divided into two categories: (primary images) Images or projects having received an average rating of 4 or 5 (out of 5) in the pre-judging process; and (secondary images) Images or projects that received an average rating of 3 (out of 5) in the prejudging process. Judges were instructed to focus their attention on the batch that received an average rating of 4 or 5. However, the judges we asked to check all of the secondary images to see if any warranted upgrading to the next stage as well.

 

Stage 4: Second Main Judging and Verification Round

1,000 or 10% of photographs and 40 or 11% of projects

 

Objectively speaking, if your entry made it this far in the process, you should be very proud indeed! All of these images were fully evaluated and scored by every judge on the panel. Judges were instructed to use the full range of scoring. These images and projects were rated on a scale of 0 to 5 by each judge:

  • 0 – A good image or project that does not meet the high standard of the other images and projects left at this stage.
  • 1 – A  very good image or project but not good enough for the finals
  • 2 – An excellent image or project but not good enough for the finals.
                Approx. 65% of images received scores from 0 to 2
  • 3 – A fantastic image or project that you would be happy to see in the final round
                Approx 25% of images received a rating of 3
  • 4 – An exceptional image or project that could win and should definitely be in the book. 
    Approx 10% of images received at rating of 4
  • 5 – A judges’ absolute favorites from each category – limited per judge as such:
                Grand Landscape: Up to 5 images
                Intimate and Abstract: Up to 5 images
                Aerial: Up to 3 images
                Nightscape: Up to 3 images
                Project: Up to 5 projects

In order to embrace the subjectivity of the judging process, i.e. each judge having their own tastes and preferences, we decided that if a judge scored an image 5, it would automatically go through to the final judging process. There were some images that weren’t judge favourites but scored consistently highly by all judges. Any image that had an average score of more than 3 stars across all judges also went through to the next round. We also ensured that any images from this round were assessed against the RAW files submitted by the entrants.

After this checking process, we ended up selecting 120 images (1.2%) and 10 (3%) of projects. 

Combined Portfolio

Each entrant had their six highest scoring images selected and a combined score calculated. The highest scoring portfolios were passed onto the judges for the final round in order to select the Natural Landscape Photographer of the Year

Stage 5: Final Live Judging

120 photographs ,10 projects and 10 portfolios

In this stage of judging, 120 images were brought forward for the category awards, 10 projects, as well as the top 10 overall portfolios of six images from photographers to be evaluated for the “Natural Landscape Photographer of the Year” Award. All eight judges and all four founders met for a 6-hour meeting via Zoom to rigorously discuss each entry, portfolio, and project that made it to this stage of the competition.

Main Categories

Any image that was voted as a favourite by more than one judge was automatically passed through to the final vote.Then, each judge in turn showed their own selection of favourite images to the other judges in the group. After a discussion about these images, the other judges voted for their own favourites out of that selection. The judge could then had to decide which of their images to put forward to the final vote. If it was clear there was a single group favourite, it made sense for the judge to select that but often there were two or more very close and the judge could decide from these.

In summary, each judge was asked to nominate a single one of their favourite images for the final vote, based on the feedback obtained by the rest of the judging panel. After this, each judge voted on their top choice for the category winner and the highest three scoring images were passed through to a final vote which gave us a first, second and third place image.This was repeated for each category. Then, a discussion was had about the four category winners and a final vote selected the “Photograph of the Year Award”.

How to read your Award Certificate

Every photograph and project that made it past Stage 3 was placed onto a certificate, showing how far your entry made it into the judging process. Additionally, entries making it to Stage 4 and beyond have the judging scores embedded into the certificate. Judge scores are shown on a normalised scale of 1 to 10, which was arrived at using a formula to show more scoring granularity for the entrant and also represents the stage to which an image has proceeded. See the below example for more detail.