Page 10 of 20 FirstFirst ... 67891011121314 ... LastLast
Results 91 to 100 of 194

Thread: A call for transparency of judging in FLL

  1. #91
    Join Date
    Sep 2008
    Location
    Minnesota
    Posts
    2,946

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by Gary Voshol View Post
    Probably dealing with his or her own life.


    So you'd like to see your entry fees raised to several hundred dollars to cover the professionals? And just how many professional tournament organizers do you know?

    The only way things can improve is if those who are taking part become part of the solution. Maybe that means that parents and coaches become trained volunteers for events - novel concept!


    Maybe they don't have enough volunteers? Did you volunteer?
    I think a lot of folks don't understand how FLL is structured. I'd call FLL a franchise, except none of the local affiliates are making money. What most people think is FLL is likely a local non-profit with maybe a half dozen or less employees, possibly none of which are full time. They spend most of their time trying to raise enough money to rent venues for FLL tournaments. Local affiliates get none of the national registration fee. I pay my local affiliate $60 for all the work they do.

    When your local affiliate isn't turning over every stone looking for sponsors, they are turning over every stone looking for volunteers. The growth of FLL is great, but unfortunately it means we need more and more volunteers every season. I read the FLL judges training and had to laugh at the part about judges having to recuse themselves if they have any relationship, even casual, with a team. I Minnesota we wouldn't have anyone left to judge. Many of our volunteers have been involved in the program for 10+ years and do training or mentoring or many other activities in addition to their judging activities. I have yet to judge a tournament where I don't have some aquaintance with two or more of the teams I judge. Even with our fantastic judge retention, and judges working multiple events each year, we almost had to cancel an event this year because of a lack of judges. That is why you hear so many long time FLLers repeat "If you want things to be better, become part of the solution. Volunteer." Volunteering really is the best way to improve the quality of your FLL program. You can have a big impact.

    So, at the last minute, your local organizers are busy: Trying to raise money for next year. Filling holes left by volunteers who canceled at the last minute. Managing the hundreds of details that have to be right for a tournament to happen. Working their day job.

  2. #92
    Join Date
    Nov 2008
    Posts
    122

    Default Re: A call for transparency of judging in FLL

    I hosted my first qualifying tournament this year and I think it was a great success. HOWEVER, the hardest part was getting qualified people in the areas of judging and table refs. I was lucky enough to be able to have made enough contacts in the "FLL World" to have at least one past judge in each room along with teachers and parents willing to put forth their best effort to assure teams were judge on merit. Additionally, the FRC PINK TEAM had 3 wonderful young ladies travel over 2 hours to come and be our Table Refs--I can't say enough about how amazing, talented, and committed these young girls are.

    Anyway--point is that I had a balance of experience and rookies for judging. And I agree if you think it's not working, step up--part of why I decided to host tournament this season. Now, having first hand experience, I will be more patient waiting for "judging results" at the end of a long day and not make statements like "what is taking so long? or What can they possibly be doing that takes this long!" And, with this understanding, I know that Core Values rule, along with balance in all three aspects (technical, research and CV) the team that shows GP throughout will be the team that the judges also recognize as having "IT".

    Lastly--someone just used this quote at a Volunteer Breakfast I attended:

    "Volunteers are not paid because they are worthless, they aren't paid because they are priceless."

  3. #93
    Join Date
    Feb 2012
    Posts
    4

    Default Re: A call for transparency of judging in FLL

    So you'd like to see your entry fees raised to several hundred dollars to cover the professionals?
    Yes, I don't mind paying more to have great championship tournaments. I am sure there are many parents willing to do the same. If you look at the volume of teams that are participating in FLL, collecting few additional dollors would go long way in helping the local organizers to focus on the other importants tasks of the championship instead of collecting donations.

    BTW: Our team did make donations to our local organizers every time we register for tournament. I understand the financial difficulties fased by the local organizers. We do help them.

    Probably dealing with his or her own life.
    I have hard time making connection to this statement. If you have things to deal in your life, why bother committing to host such a tournament. I rest my case.

    Maybe that means that parents and coaches become trained volunteers for events - novel concept!
    Gary,

    excellent point, I can't agree with you more. FLL should give a serious thought.

    Maybe they don't have enough volunteers? Did you volunteer?
    Yes, I voluteered for events I that I was eligible. I could not volunteer for judging because my team was competing.

    My point is silicon valley has so many talented people. I am surprised they couldnot find voluteers. I know personally, there are organizations in that area have thier engieers and scientists voluteering for community work and the company donating matching funds to the community for the employee volunteered hours. I have reported my FLL coaching hours, so that the local FLL organizers can get the matching funds from my company.

    Let me give you an example, Public schools (elementary/middle/high) in our community conduct scicence fairs every year and they apprach the local tech companies for voluteer judges. You can see the difference in the quality of the judging in these events. I can't understand why same can't be done by the FLL local organizers. I hope they expand their call for jundes beyound thier website.

  4. #94
    Join Date
    Jan 2008
    Location
    Royal Oak, MI
    Posts
    920

    Default Re: A call for transparency of judging in FLL

    Madhala,

    Most events that I have been affiliated with do get volunteers from industry. Here in MI that often means the car companies. There are a number of volunteers and event organizers that come from GM, Ford, Chrysler, Delphi, Nexteer, and several other suppliers. Another source is education and government - we usually have judges and referees who are teachers, school board members and local or state office holders.

    The key is getting the people started. My wife and I became volunteers after watching our daughter's team compete for 4 years. That was 8 years ago now, and we keep on going. This year I will have reffed at 4 FLL events, 2 FTC events, and 3 FRC events (and I'd do more of those if I could get the time off work). But people who don't know about FLL won't be volunteering at FLL. Those of us who are already involved have to bring them in. That's all there is to it.

  5. #95
    Join Date
    Feb 2012
    Posts
    24

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by Dean Hystad View Post
    Yeah, that's just not going to happen unless there are BIG changes in FLL. You can have the one award per team policy or you can have public rankings. You can't have both. If I give you a design award and you find out from the rankings that the Champion's award team also had the highest design ranking, how do you feel about that? A case could be made to get rid of the one award per team policy, but FLL has a lot of love for it as do a majority of coaches I've spoken with about this.

    For quite a while in Minnesota we not only had rankings, but the scores used to calculate the rankings posted on the local FLL website. It was fun to look through the results, but I never gleaned anything all that useful from a bunch of numbers. What does 88 mean? Why did team B get 92? The only time I ever found the numbers useful was my first year when the two teams I coached finished last and second to last. That was informative.

    "If I give you a design award and you find out from the rankings that the Champion's award team also had the highest design ranking, how do you feel about that? "

    So, we are ashmed to publicly standby the policies we came up with! Wow! THis is the best example we can set for our kids!

    If we don't want to give multiple awards to one team. Explain. Don't hide behind lack of transparancy.

  6. #96
    Join Date
    Feb 2012
    Posts
    24

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by timdavid View Post
    The following is the best explanation I've seen for the current FLL policy of keeping rankings and scores a secret. I don't agree with all the rationale, but I can understand the reasoning. It used to be on the FLL web site. I found a copy at
    http://fll.larobotics.org/resources/...onnections.pdf
    ================================================== ======

    Why scores and rankings don't mean much.

    Some of the most common questions I hear about FLL judging concern scores or rankings of teams. Among the questions are “What was my team’s score?” and “Where did my team rank among all the teams?” I always respond to these questions with “The policy of FLL is to not release scores or rankings.” This in itself is not a very explanatory answer, and usually leads to follow-up questions and confusion about the process in general.

    I’d like to tell you why I think disclosure and distribution of judging scores and rankings is not a good idea.

    The main reason can be found in FLL’s Core Values. “What we discover is more important than what we win.” FLL judges are trained to evaluate teams on what they have learned throughout their season using the set of criteria defined in the rubrics. Rubrics are designed to serve as a list of expectations for a learning process that can then be used to evaluate achievement. So in essence, they can be used to evaluate what a team “discovers” throughout the season.

    Judges can use the rubrics to provide teams specific feedback on strengths and areas for improvement by returning feedback to the teams using the rubrics as a template for the feedback. By providing this feedback instead of a score or ranking, the judging process directly reinforces the Core Value.

    Judging is also a subjective process. For most championship tournaments, not all judges get to see all the teams at an event. The judges are human and have variation in how they evaluate teams. The judging process is designed to minimize these differences, but they do occur. The best way to normalize differences between groups of judges in order to compare teams is for judges to talk about the similarities and differences between teams. The time constraints for a typical tournament are so tight that this normalization process is really only possible for teams in consideration for award. So if actual scores or rankings were distributed, a team might feel that they have received a low “score”, but in fact they may have just been evaluated by a “tougher” set of judges.

    Another reason to not provide scores or rankings has to do with the FLL awards distribution philosophy. FLL teams may only win one award at a championship event. Imagine if a team “scored” or “was ranked” number one in more than one judging category or award. The natural inclination is to think that team should be awarded more than one trophy, which runs counter to the FLL awards distribution philosophy.

    Finally, there is a psychological reason for not distributing scores and rankings. FLL tournaments are supposed to be fun celebrations where all teams share what they have achieved. Some teams may feel that if they receive a low “score” or “ranking” that their season and experiences have less value than those of a “higher scoring” team. Or they may simply feel beaten. FLL is about focusing on what has been learned, and no one should feel beaten after a learning experience! In fact, learning quite often has greater impact whenwe fail or have a setback. I want the kids to know that there is something positive that comes out of every experience. It doesn’t have to be a high score or a trophy. It can be satisfaction that your robot can complete just one mission every time. It can be a discovery by a young researcher who has found his or her future in nanotechnology. No score can measure that.
    It all depends on what FIRST wants to do with FLL. Should the tournments stand for excellence? Or, is this an effort to get some feel good results that reward participation in STEM but do not reward real accomplishments in STEM?

  7. #97
    Join Date
    Feb 2012
    Posts
    24

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by Dean Hystad View Post
    That was Skip Gridley's response to my post wondering if Gracious Professionalism means being secretive and lying. This was four years ago, right after INSciTE got their hands slapped for publicly posting scores and ignoring the one award per team policy. The question about transparency has been going on for a while.

    Though I still don't agree with FLL on this issue, I find their current practices neither broken or unfair. The explosive growth in FLL is proof that these policies are not harming the program. FLL and FIRST are robotics competitons, but first and foremost they are experiments in social engineering (my opinion). They are tools to fix a broken world. Does it make sense to embrace the same paradigm that broke the world in the first place? Competition is good, but a short sighted focus on winning is not. How do you teach that lesson? Focusing on ranking and scores probably isn't it.
    "The explosive growth in FLL is proof that these policies are not harming the program. "

    TO me it is looking like the explosive growth is a function of putting evangelism ahead of STEM. This growth will fall flat if the newly coming folks can't make sense of the "results" in the tournments. By then the brand would have lost a considerable amount of value.

  8. #98
    Join Date
    Feb 2012
    Posts
    24

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by Tom Mosher View Post
    Even granted that it's not grammatically correct, or written well enough to be unambiguous, that looks like a very strange rule.

    Say that scores at an event range from 60 to 180, and are uniformly distributed. The average score will be 120. The "2 times" factor becomes 240. No teams scored above 180, so no teams can qualify to advance.

    That just can't be what they really mean.
    That is one of the strangest peices of criteria I have seen. A table that shows how it works would have been useful.

  9. #99
    Join Date
    Feb 2012
    Posts
    24

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by timdavid View Post
    In our state this year, the two champions (one for division 1 and one for division 2), both finished second in robot performance. Probably just coincidence...
    If the winning team places high in performance (like the example you give), the results would be a lot more palatable to teams.
    If the winning team does not rank in the top decile or quartile and still wins then to me that would indicate that lack of emphasis on STEM.

  10. #100
    Join Date
    Feb 2012
    Posts
    24

    Default Re: A call for transparency of judging in FLL

    Quote Originally Posted by LOLComets View Post
    For all those critical of the process and rules (transparency) you have choices:
    • Quit FLL
    • Recruit volunteers and train them to be good judges
    • Volunteer to help Event Organizers and the Operational Partner


    Personally, I don't like the extent to which Judging is subjective and I don't care for lack of transparency. But these are the rules!

    Having been a Robot Design judge at several events, the process of assigning trophies to teams is not simple because of the one-trophy-per-team rule and amount of discussion that takes place during deliberation. There is also the time constraint to arrive at decisions.

    Minnesota has a very good process for ensuring that teams that are nominated for the awards, are reviewed by another call-back panel of judges.

    For those who are not happy about the process, you may want to see what can be done differently next season to improve the process. First step is to become a volunteer.
    THere are other choices - such as trying to change the rules - which is what this particular forum is about.

Page 10 of 20 FirstFirst ... 67891011121314 ... LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. downloading myBlocks for FLL
    By RvRDilemma in forum Programming
    Replies: 12
    Last Post: 01-09-2012, 06:57 PM
  2. Open Judging sessions
    By timdavid in forum Coaching & Team Management
    Replies: 2
    Last Post: 12-08-2011, 03:59 PM
  3. technical vs. nontechnical judging?
    By RvRDilemma in forum Robot Game Strategy
    Replies: 3
    Last Post: 12-07-2011, 02:10 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •