Judging Process
Updated November 15, 2023
Our judging process is more involved than most competitions. The goal was to eliminate some common issues that arise during photo competition judging. Often, with the common process of ‘score averaging’, images that are well rated by several judges (i.e. all given average to good scores) rise to the top, and images that create excitement and discussion (i.e. that individual judges give top scores to, but are more polarising) are left behind. As such, some of our winning images may have been rated lower by one or more judges; but were loved and rated the highest score by other judges.
Additionally, for many competitions, the judges are not even aware of the winner nor are they included in discussion to lobby for winning images and photographers. That was not the case for our competition, and we wanted to take the time to outline our process below so that entrants can understand the process as well as decipher the scoring that can be found on their certificates.
Our judging process was completed in several stages, with the goal of filtering work into a manageable pool of images for our judges. Additionally, judges were given criteria that the founders of the competition wanted the judges to focus on, including originality, composition, and complementary editing. As such, many “very good” photographs of iconic scenes or copies of other photographers’ work did not score as well or make it as far into the competition.
It should be noted that we received a tremendous amount of incredible photography in our first year and your scores and placing should be assessed in the context of the large pool of amazing photography it sits alongside. We also want to reiterate that we will be offering free opportunities (such as webinars) for entrants to learn ways to improve their work. The numbers below represent what happened in Year 1 of the competition.
Stage 1: First Pre-Judging
The four organizers of the competition, Tim Parkin, Matt Payne, Alex Nail, and Rajesh Jyothiswaran spent a week individually ranking and reassessing all of the submissions on a scale from 0 to 5, with 0 being poor and 5 being “the best of the best.” Every image that scored a single 5 and any image that scored two 4’s also went through to make sure that the selection wasn’t purely based on averages, and that subjectivity was accounted for. The goal of this phase was to eliminate photographs that we knew would not be well assessed by our main judges, mostly due to technical issues or due to not meeting the submission criteria . These individual scores were then averaged across all four organizers and any photographs that were either rated highly by at least one organizer or rated an average score of 3 or more were kept for the next stage.
Stage 2: Second Pre-Judging
In the second stage of the competition, all four founders reassessed the remaining images to ensure that any images that stood even a small chance of winning a category or overall winner went through to the main judges. If any of the four founders thought an image should progress, it was passed onto the next category. After a week of deliberation, we had a second look through all of those that failed the cut to check if we had missed anything.
Stage 3: Verification, Main Judging and Live Pre-Judging
Objectively speaking, if your entry made it this far in the process, you should be very proud indeed! All of these images were fully evaluated and scored by every judge on the panel. Judges were instructed to use the full range of scoring. These images and projects were rated on a scale of 0 to 5 by each judge:
- 0 – A good image or project that does not meet the high standard of the other images and projects left at this stage.
- 1 – A very good image or project but not good enough for the finals.
- 2 – An excellent image or project but not good enough for the finals.
Approx. 65% of images received scores from 0 to 2 - 3 – A fantastic image or project that you would be happy to see in the final round
Approx 25% of images received a rating of 3 - 4 – An exceptional image or project that could win and should definitely be in the book.
Approx 10% of images received at rating of 4 - 5 – A judges’ absolute favorites from each category – limited per judge as such:
Grand Scenic: Up to 20 images
Intimate Landscapes: Up to 20 images
Abstracts and Details: Up to 20 images
Project: Up to 5 projects
In order to embrace the subjectivity of the judging process, i.e. each judge having their own tastes and preferences, we decided that if a judge scored an image or project a 5, it would automatically go through to the final judging process. There were some images that weren’t judge favourites but scored consistently highly by all judges. Any image that had an average score of more than 3 stars across all judges also went through to the next round. We also ensured that any images from this round were assessed against the RAW files submitted by the entrants.
Combined Portfolio
Each entrant had their six highest scoring images selected and a combined score calculated. The highest scoring portfolios were passed onto the judges for the final round in order to select the Natural Landscape Photographer of the Year.
Stage 4: Final Live Judging
In this stage of judging, we evaluated the combined scores of images scored highly by the judges and narrowed the selections down from there based on the highest scoring images. 48 were brought forward for the category awards, 16 projects, as well as the top 23 overall portfolios of six images from photographers to be evaluated for the “Natural Landscape Photographer of the Year” Award. All five judges and all four founders met for a 6-hour meeting via Zoom to rigorously discuss each entry, portfolio, and project that made it to this stage of the competition.
Live Voting
In order to make the final judging process more efficient, we decided to develop a live web-based voting system that allows. This allows each judge to see an independent view of the images to vote on and enables them to see high-resolution versions and then vote for their first, second, third and fourth choices. The system adds up the scores by weighting and allows the admins to see and sort the results. Using this method, we were able to reduce the number of images in the main categories down to six and then make a final vote.
This was repeated for each category. Then, a discussion was had about the four category winners and a final vote selected the “Photograph of the Year Award”.
How to read your Award Certificate
Every photograph and project was placed onto a certificate, showing how far your entry made it into the judging process. Additionally, entries making it to Stage 2 and beyond have the judging scores embedded into the certificate. Judge scores are shown on a normalised scale of 1 to 10, which was arrived at using a formula to show more scoring granularity for the entrant and also represents the stage to which an image has proceeded. See the below example for more detail. Certificates will be issued before the end of December.
