A call for transparency of judging in FLL

[Dean Hystad]

Originally Posted by baybots2012

and now for something completely different

http://www.youtube.com/watch?v=zNyln...eature=related

This is NOT in the spirit of friendly coopertition!!! The horror! The horror!

[dnb]

Originally Posted by baybots2012

and now for something completely different

http://www.youtube.com/watch?v=zNyln...eature=related

Oh, man, that's almost like pro wrestling.

[Sacramento Volunteer]

Originally Posted by ISHW

No, we haven't received the rubrics.

It is now more than 6 weeks since the NorCal FLL Capital District Championship and nearly a month since we were told that the Rubrics were coming and Scores would be posted on the Championship web site......

Nothing yet!

If the intent is for the the Rubrics to provide constructive criticism to the teams, this delay defeats that purpose.

[Dean Hystad]

Originally Posted by Sacramento Volunteer

It is now more than 6 weeks since the NorCal FLL Capital District Championship and nearly a month since we were told that the Rubrics were coming and Scores would be posted on the Championship web site......

Nothing yet!

If the intent is for the the Rubrics to provide constructive criticism to the teams, this delay defeats that purpose.

Judges feedback is not constructive criticism. It should be a mixture of praise for what you did right and recommendations for how you can improve. It should not contain little or no criticism, constructive or otherwise.

As for scores, I'm confused as to what scores you refer. The only "scores" are for the robot game, and I don't see much benefit gained from posting those. As for technical, project and core value judging, there may be no scores, and if there were, posting them on a web site is against FLL policy.

[Sacramento Volunteer]

Originally Posted by Dean Hystad

As for scores, I'm confused as to what scores you refer. The only "scores" are for the robot game, and I don't see much benefit gained from posting those. As for technical, project and core value judging, there may be no scores, and if there were, posting them on a web site is against FLL policy.

Here is an abridged version of a posting dated 2/27/12 by one of the officials at Playing at Learning, the NorCal affiliate of FIRST:

"We have received multiple emails over the past few days asking about these items.
With regards to rubrics, they will get mailed.......Most likely after this weekend, we will start to free up and can start getting through each tournament’s paperwork (the rubrics are generally all together with other paperwork that is all thrown together).
With regards to awards and scores, again, these will get posted – on this site. We want to recognize the award winners and want to include the short commentary that went along with each 1st place team that was said at the closing ceremony."

You can read the entire post at http://championship.norcalfll.org/

[timdavid]

I received an email yesterday from FLL that the annual coaches survey was available. I took the survey, and it had a number of questions about judging, the rubrics, and the awards process. Looks like a good opportunity for coaches to give feedback to FLL.

[Nik Bhatt]

Originally Posted by timdavid

I received an email yesterday from FLL that the annual coaches survey was available. I took the survey, and it had a number of questions about judging, the rubrics, and the awards process. Looks like a good opportunity for coaches to give feedback to FLL.

I didn't receive an email about that -- I get various other FLL emails. Is this survey limited in some way?

[timdavid]

Originally Posted by Nik Bhatt

I didn't receive an email about that -- I get various other FLL emails. Is this survey limited in some way?

I don't know. I received the email March 29th from a person at us first.org.
I emailed her to see if I can post a link to the survey on this forum.

[sgridley]

I've been reading this thread for some time, and have been thinking about how best to reply. Many people have spoken very eloquently (thank you Dean and others) about FLL judging, what it represents, how it's done, and numerous other issues. I'd like to invite any additional feedback about ways to improve FLL judging to be sent to me, Skip Gridley, Global Judge Advisor for FIRST LEGO League at flljudge@usfirst.org. We really are interested in your feedback to help make things better.

The mission of FIRST is to inspire kids to get excited about STEM. It does it by using robotics as a hook to get kids interested, having them work with with hopefully good adult role models (coaches, mentors), and indirectly teaching them other skills like teamwork, confidence and the ability to solve problems. It also features exciting competitions to give kids a chance to be recognized for their work in ways that are familiar to them from sporting competitions.

Almost everyone who does FLL does so via choice, as a volunteer, for free, doing the best they can in the time they can with the resources they have. Just like the teams do. Sometimes they get it wrong, but in the vast majority of cases, I would challenge reasonable people to get significantly different results.

Yes, judging is subjective. The rubrics go a long way to minimizing it, as does the deliberations process. I welcome (notice I say welcome) anyone to sumbit more objective criteria that include all the key takeaways from FLL for each of the three areas, and are more objective than the current rubrics. I often hear that the rubrics are subjective, but receive little to no actual suggestions on how to change them.

Accept the fact that full results of all teams will never be made public to all teams. What good does the world knowing which teams finished last do to those teams that already know they have many areas to improve? FLL and FIRST are about getting MORE kids interested in STEM, so we try to celebrate ALL teams in some fashion and focus on the positive and ways to improve. Yes there are awards. They are about Recognition. The "R" in FIRST.

By all means, please feel free to e-mail me. Dean Hystad mentioned a discussion I had with his girls in MN several years ago. I do listen, I'm happy to share as well. In the end everyone benefits from increased understanding, if not always complete satisfaction.

Thank you,
Skip Gridley
Global Judge Advisor, FIRST LEGO League
flljudge@usfirst.org

[LOLComets]

Originally Posted by sgridley

...It also features exciting competitions to give kids a chance to be recognized for their work in ways that are familiar to them from sporting competitions.

The mission of FIRST is to inspire kids to get excited about STEM. It does it by using robotics as a hook to get kids interested...

Yes, judging is subjective. The rubrics go a long way to minimizing it, as does the deliberations process. I welcome (notice I say welcome) anyone to submit more objective criteria that include all the key takeaways from FLL for each of the three areas, and are more objective than the current rubrics.

Thanks for the first post!!!

First, I want to highlight two points:
1. There is a competition
2. Hooking the kids involves robots

Why did FLL remove the ONLY competitive and objective element, i.e., Robot Performance from the overall score considering that it is the hook that attracts kids to the program?

I can only guess that it was done intentionally to remove any objective evaluation.

For robot design / programming / Strategy, there are many quantitative options and here are a couple:

# of missions attempted
# of missions per program
# of programs
Score consistency (average score vs. best score or Std Deviation).