Judging Plan

Judging has always been the biggest pain point for every hackathon organizer. This is due to the fact that while planning out the event, other than recruiting and finalizing the judges, and assigning a certain time for judging, organizing teams do not plan anything further, assuming everything can be managed on the day-of. But, in reality, if planned right, judging can be one of the easiest and the most smooth aspect of any hackathon.

Our recommendation is always towards implementing a science-fair type of judging plan.

Structurally, we recommend having hackers submit 2 minute videos on your submission platform for the judges to review. Make sure they put a lot of love and care into their submission. Their video should be a demo of their hack, not a presentation. This is not needed for judging when science fair style, but can be a reference during final deliberations, and is always a good future reference if they want to include it on their resumes.

To avoid confusion, the key is to make sure hackers know exactly where you are in the schedule for judging. Tell them you’re advancing to the next stage of judging when you do, or if judging is taking longer than expected, tell them how far along you are. Giving quick updates like “Our judges have reviewed about 50% of the projects!” goes a long way in keeping hackers engaged.

Science Fair Style Judging

In this particular judging style, assume your hackathon to be a science fair and every hacker team setting up their stall at assigned table numbers. Hackers present their hackathon project in front of the judges one by one as they arrive at their stalls, and judges score them accordingly. This way every hackathon attendee is getting a chance to demo in front of the judges and have the experience of presenting at the hackathon.

Time Requirement

Towards the end of the hackathon, take out some buffer time for the judging. This usually takes around 2 - 3 hours, depending on the number of submissions and judges.

Calculating the required number of Judges

A typical calculation for the number of judges looks like follows:

$J = ⌈(P * n * t )/ T⌉$

Where, $J =$ Number of Judges $P =$ Number of Submitted Projects $n =$ Number of Judges for each project (or number of rounds of judging per project) (MLH recommendation: 3 rounds per project) $t =$ Time taken by a Judge per project (MLH recommendation: 3 mins per project) $T =$ Total time allocated for Judging (in minutes)

For eg.: Consider you have 150 projects submitted at your hackathon and you have allocated 2 hours for judging. As per our recommendation, we are having 3 rounds of judging per project and 3 minutes are allocated per project for a judge to take their decision.

Now the calculation looks like:

J= (150*3*3)/120 = 11.25 = 12

Hence the number of judges required is 12.

Allocating Projects

Allocating projects is more logistically heavy in a science fair style of judging because each judge cannot see every project.

Check out the Readme on our Judging Example Sheet for allocating judges to projects! This example process makes sure each team is seen by at least 3 different judges.

Choosing Winners:

There're plenty of options with finalizing winners, but at MLH, we love to use Stack Ranking because it reduces a judges’ previous biases.

Stack Ranking: To run stacking ranking, you'll be needing to the following steps:

• Allocate projects to every judging according to what we discussed above.

• Ask each judge to view their projects and report back their top 3 best/favorite projects.

• Assign the top three following points:

  • 3 points (First Place)

  • 2 points (Second Place)

  • 1 point (Third Place)

Now, you'll have a list of the project that appeared across multiple judges' top three favorites!

After all of the scores have come in, you’ll sort all of the scores in descending order. Then, the projects at the top will be your winning projects!

A go to table to estimate the number of judges required:

	Number of attendees
Time Allocated for Judging (mins)		45	60	90	120	150	180
	100	9	7	5	5	4	4
	200	15	12	9	7	6	5
	300	22	17	12	10	8	7
	400	29	22	15	12	10	9
	500	35	27	19	15	12	10
	600	42	32	22	17	14	12
	700	49	37	25	20	16	14
	800	55	42	29	22	18	15
	900	62	47	32	25	20	17
	1000	69	52	35	27	22	19
	1100	75	57	39	30	24	20
	1200	82	62	42	32	26	22

Resources:

• Template: Judging Expo Process

Any questions? Feel free to shoot an email to [e-mail] to discuss your Judging Plans!