Announcement

Collapse
No announcement yet.

A call for transparency of judging in FLL

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Re: A call for transparency of judging in FLL

    Originally posted by Dean Hystad View Post
    I don't think NorCal is a qualifying tournament, is it? Isn't it one of the four big tournaments in California that may (depending on the lottery) send a team to the World Festival? It's a big tournament, but still one big tournament does not a widespread problem make.
    Perhaps it is just my confusion, but I don't think all the forum members attach the same meaning to terms such as "qualifying" or "regional" tournament.

    My understand is that California is big enough that it is split into 4 different areas - let's call those regions. Those regions are Northern, Southern, Central Valley, and Los Angeles.

    The Northern region (NorCal) was split into 4 "districts" this year. The champion from each district this year receives an invitation to the World Festival in St. Louis.

    The 2/11 Capital district championship seems to be the main one generating controversy, though there have been hints of unrest with the other two district championships already concluded this year. The championship for the 4th district has evidently been postponed due to venue scheduling issues.

    Personally, I wonder if part of the issue with the NorCal region this year is due to the logistics of trying to staff and put on 4 championship tournaments within a month. Holding 4 championship tournaments within a month had to require a lot of effort, and require a large number of volunteers.
    Last edited by timdavid; 03-01-2012, 04:39 PM.

    Comment


    • Re: A call for transparency of judging in FLL

      Originally posted by timdavid View Post
      My understand is that California is big enough that it is split into 4 different areas - let's call those regions. Those regions are Northern, Southern, Central Valley, and Los Angeles.
      My understanding is there are 3 regions in California, (1) Los Angeles, (2) Southern California, and (3) Northern California.

      Originally posted by timdavid View Post
      The Northern region (NorCal) was split into 4 "districts" this year. The champion from each district this year receives an invitation to the World Festival in St. Louis.
      This is accurate. There were (many) qualifiers in November / December, and they fed into 4 district championships: (1) San Francisco - Peninsula, (2) Silicon Valley, (3) East Bay and (4) Capital (= Sacramento area). I understand there are / were 48 teams at each district championship.

      Originally posted by timdavid View Post
      The 2/11 Capital district championship seems to be the main one generating controversy, though there have been hints of unrest with the other two district championships already concluded this year. The championship for the 4th district has evidently been postponed due to venue scheduling issues.
      Not sure which hints of unrest you are referring to. My team attended the SF-Peninsula District Championship in January and the event went very well and was exceptionally well organized.

      However, and this points back to the title of this thread, "A call for transparency of judging in FLL", there are discussions how to increase transparency and improve feed back to teams.

      Originally posted by timdavid View Post
      Personally, I wonder if part of the issue with the NorCal region this year is due to the logistics of trying to staff and put on 4 championship tournaments within a month. Holding 4 championship tournaments within a month had to require a lot of effort, and require a large number of volunteers.
      I was not at any other championship tournament than the San Francisco - Peninsula one, and therefore cannot comment on what might have or not have happened at other tournaments. I think it would be better for everyone to talk about what they know and not make conjectures what might have happened at events they did not participate.
      Claus, Coach "It Should Have Worked", San Bruno 4-H

      Comment


      • Re: A call for transparency of judging in FLL

        Originally posted by timdavid View Post
        Perhaps it is just my confusion, but I don't think all the forum members attach the same meaning to terms such as "qualifying" or "regional" tournament.

        My understand is that California is big enough that it is split into 4 different areas - let's call those regions. Those regions are Northern, Southern, Central Valley, and Los Angeles.

        The Northern region (NorCal) was split into 4 "districts" this year. The champion from each district this year receives an invitation to the World Festival in St. Louis.
        ...and each district championship tournament is fed by several qualifying tournaments. The list of all these tournaments is at

        http://www.norcalfll.org/index.php?o...11ff&Itemid=58

        For comparison, my wee home state of Massachusetts has 6 regional qualifying tournaments that feed into one state tournament. The regional qualifiers are the first tier of competition for teams in MA.
        Last edited by dnb; 03-01-2012, 05:23 PM.

        Comment


        • Re: A call for transparency of judging in FLL

          Originally posted by ISHW View Post
          My understanding is there are 3 regions in California, (1) Los Angeles, (2) Southern California, and (3) Northern California.
          It appears the Central region does exist and held a championship tournament last December.

          Originally posted by ISHW View Post
          Not sure which hints of unrest you are referring to. My team attended the SF-Peninsula District Championship in January and the event went very well and was exceptionally well organized.
          I apologize if I made an unwarranted inference, but I interpreted some of your earlier comments as being unsatisfied with the judging feedback you received. I'm glad to hear your team was happy with the event.

          Comment


          • Re: A call for transparency of judging in FLL

            Originally posted by timdavid View Post
            I apologize if I made an unwarranted inference, but I interpreted some of your earlier comments as being unsatisfied with the judging feedback you received. I'm glad to hear your team was happy with the event.
            No need to apologize. For me there is a huge difference between "being satisfied or unsatisfied with judging feedback" and "unrest".

            To quote myself:
            Originally posted by ISHW View Post
            On the plus side, I see that my team learned a lot, had a lot of fun, worked really hard, and looks back at the season as a big success.
            This thread became a mish-mash of many different things, some of which I disagree with. However I like about this thread that people like you, Dean, Tom and others describe how judging and feedback is handled in other regions. This is valuable, one can learn from it and hopefully improve.
            Claus, Coach "It Should Have Worked", San Bruno 4-H

            Comment


            • Re: A call for transparency of judging in FLL

              Originally posted by BlueCheesyFlamingos View Post
              That is my problem. My team did max out the rubric for the project portion. The kids are now asking what else the other team did that placed them above a perfect score (aka a A++). I have no idea what else they could have done. They went above and beyond any of our expectations.

              I don't care about the awards. Each year we agree as a team NOT to use any of the awards as one of our team goals. I want my kids to be able improve and take a step forward each year. There is no improving and learning in this scenario because there is no feedback.
              And for that there is no answer. At least no useful answer. What another team did to differentiate themselves from other "perfect" teams may only be meaningful in the context of that one tournament. If you max out the rubric there is no way to provide feedback about how you can be better. This is rubric issue, possibly a judging issue, but not a transparency issue. I ran into the same thing with my girls at the first US Open. They were exemplary in every way in every category. A scatter chart showed them at the top in everything. They maxed out the available scoring tool. They received a second place creative solution award. The girls were thrilled because it was their first non-robot award.

              I don't buy into your "How do we know how to improve?" lament. My girls always had a big list of ideas on how they could improve. If you are maxing out the rubric you are obviously smart enough to critique your own work. If you don't care about awards, there is no need to conform to someone elses criteria. You are in the envious position of deciding for yourself what defines success. A real, working device for the research solution? Implementing a real solution out in the real world? Starting two new FLL teams and serving as mentors? Starting a technology program in at a local school? All of the above? You decide. Isn't independent thinkers making real positive change in the world the target goal for FLL?
              Last edited by Dean Hystad; 03-01-2012, 06:17 PM.

              Comment


              • Re: A call for transparency of judging in FLL

                Originally Posted by BlueCheesyFlamingos
                That is my problem. My team did max out the rubric for the project portion. The kids are now asking what else the other team did that placed them above a perfect score (aka a A++). I have no idea what else they could have done. They went above and beyond any of our expectations.

                I don't care about the awards. Each year we agree as a team NOT to use any of the awards as one of our team goals. I want my kids to be able improve and take a step forward each year. There is no improving and learning in this scenario because there is no feedback.

                This is my first response on a very interesting feed! My team is also from MN. We were called back for Programming at the State Tourney. The teams in our grouping were very high level and I'm sure their score sheets had the highest numbers on the ruberic. If I remember right, the team that won the MN state tourney was in our grouping.

                As we were leaving the morning judging session the judge asked if they had anything else to show. They boys have a programming journal and the main programmer pulled it out again to show the judge. In the journal they have everything, including the scraps of paper on which the boys did their original program planning - Seriously scraps of paper with boy scribble - "go forward 20 centimeters, turn to right 90 degress, go forward 5 centimeters". The judge got very excited and said something to the effect of "That's what I was waiting for!" I happened to be standing right there or I would have missed it. He did not get that info written on our judging sheets and you have to wonder how he would have phrased it anyway - "Good job on the Chicken Scratch"? But it seemed to me that that may have been what gave our team the A++ over the other team A+.

                I know that it's been said before, but I think the best feedback is listening to the judges in the room - what do they "Hmm" over. What did the judges see that they Commented "Cool" or asked to see again?

                The above team also had a call back for Strategy and Innovation. The judges had them re-run a mission several times to understand how it worked. It was a cool solution that made the judges laugh once they figured out how it worked - again no place to comment on the ruberic but the Judges laughter let the Boys know that they had done well! To them that was a great affirmation.

                Comment


                • Re: A call for transparency of judging in FLL

                  Originally posted by Dean Hystad View Post
                  I haven't heard about any transgressions, just that some people are unhappy with award results.
                  Yes, but not without reason!

                  I have communicated concerns voiced by the kids in my team about the judging at the recent NorCal Capital District tournament to FLL. We have not posted them on this forum, expecting that it would evoke an automatic “sour grapes” response.

                  Others from the NorCal area have posted responses on this thread, expressing their concerns about the judging process. Several more area FLL and FTC coaches have shared similar sentiments, but since they have “active teams” have chosen to stay silent.

                  Comment


                  • Re: A call for transparency of judging in FLL

                    Originally posted by EmGe View Post
                    Several more area FLL and FTC coaches have shared similar sentiments, but since they have “active teams” have chosen to stay silent.
                    "Active teams" should be the most vocal. They have the biggest stake. I've yet to meet anyone in FLL who is vindictive. Lots of differing opinions for sure, but not much pettiness. The year Minnesota was forced to change some of their policies my girls had a lengthy discussion with Skip Gridley at our state tournament. I don't think any minds were changed, but Skip got some insight into how kids felt about the issue, and the girls were grateful to Skip for hearing them out.

                    I was out wandering the pits as I am prone to do when one of our parents rushed up saying "Some FLL guy is looking for you!" When I got to our pits area Skip and the girls were already well into it so I just hung back and listened. I was very proud of how the girls respectfully presented a logical argument to repeal the policy changes. I was also very proud of how Skip carefully listened to each point and respectfully presented his side. What a wonderful thing is this program that attracts such wonderful people. I don't know if FLL does a good job preparing children for "real life". Participation if FLL may cause children to believe that people behave rationally.

                    You mistook my point about FLL and complaints. I don't know if FLL has a tool to fix the alleged problem in NorCal. It doesn't sound like anything wrong is being done, just that some things aren't being done well. I suppose FLL could tell them to "shape up", but a delegation of FLL teams from the area could do the same thing.

                    Comment


                    • Re: A call for transparency of judging in FLL

                      Originally posted by EmGe View Post
                      Several more area FLL and FTC coaches have shared similar sentiments, but since they have “active teams” have chosen to stay silent.
                      That approach is guaranteed to get no results.
                      FIRST Tech Challenge Judge: 2010, Referee: 2017
                      FIRST LEGO League Mentor, Instructor, and/or Referee/Head Referee since 2011
                      FIRST Robotics Competition judge (Chairman's Award): 2014
                      Dean says I'm an "Oompa Loompa of Science"

                      Comment


                      • Re: A call for transparency of judging in FLL

                        I have skimmed this thread since I first stumbled upon it last week and figured I would add a bit. I would agree with Tom and Dean that "active teams" need to be vocal. I would take it one step further and say that these teams should also get involved in the planning and running of the tournament. Keep in mind that the only tournament that is actually run by FIRST is World Festival (the more senior people on the board will correct me if I am wrong) - the rest of the tournaments are run by state or regional affiliates. These affiliates tend to be underfunded and understaffed and I am sure they would welcome volunteers to help organize, judge and referee - the best volunteers are ones who have actively participated in FLL. I am not sure how much FIRST would actually do to correct any problems in a region; any problems in a region would have to be corrected from within. From my own experience, the best way to do this is to volunteer with a tournament in some capacity. This doesn't mean you should volunteer to ensure your team wins an award, volunteer to be sure the tournament runs as well as possible.

                        I am not sure how there can be much debate about which teams qualify for the champions awards. I can't imagine that your tournament doesn't use electronic scoresheets for the project, core values and robot design judging. Each judged event should have a score for each team. FLL guidelines for the champions award are specific - each judged event is worth 33 1/3%. The master scoresheet (spreadsheet) should add up all the scores for each team and rank them accordingly. I guess I could see how there could be some deliberation over the top 4 or 5 teams, especially if a team does really well in one event and mediocre in the other two events. In the end, though, if teams are ranked for all the judges to see, there shouldn't be a way for an unqualified team to win an award.

                        I have judged at a state competition and helped organize a regional tournament (most states call them qualifiers). My experience has been that judges (at least here in Illinois) are pretty vocal and aren't easily swayed by another judge. They may have strong opinions, but certainly a sub-par team would not even be up for discussion for a champions award.


                        FLL Co-coach 2006-2011
                        FLL judge 2010

                        Comment


                        • Re: A call for transparency of judging in FLL

                          I have to say that some of the criticism here of the Norcal district operation is entirely misdirected. I have been a coach in this region for 7 seasons and every tournament I attended (3 this season) have been extremely well run with the operations and judging appearing to be smooth and fair. The kids at these events are learning, interacting and having fun in the best spirit of FLL. There are 431 teams competing in Norcal this year! That is a measure of success in and of itself.

                          I did not attend the Capital Tournament and even if I had, I wouldn't know how the Champion award winner was chosen. If some coaches were surprised by the choice, so be it. Let it go. You don't get to know exactly how one team outperformed another on a subjective criteria any more than you get transparency in college or job acceptance decisions.

                          Comment


                          • Re: A call for transparency of judging in FLL

                            Originally posted by bananabots View Post
                            The kids at these events are learning, interacting and having fun in the best spirit of FLL.
                            Is the "best spirit of FLL", a member of the Champion's Award team responding, "Go ask my dad.....", when asked asked a question about their project solution?

                            Comment


                            • Re: A call for transparency of judging in FLL

                              Originally posted by bananabots View Post
                              If some coaches were surprised by the choice, so be it. Let it go.
                              Originally posted by bananabots View Post
                              There are 431 teams competing in Norcal this year! That is a measure of success in and of itself.

                              Not so simple! This is not a matter of coaches “letting it go”. What is at stake here is the continuing participation of motivated teams in FLL. The growth of FLL in the NorCal region has in large part been from the outreach efforts of these teams.

                              Selecting a team in which “the girls spent more time shopping than working on robotics”, one whose team member responded "Go ask my dad.....", to receive the Champion’s Award, will only lead to kids who want to challenge themselves and achieve excellence by investing time and effort, abandoning FLL, to pursue alternate STEM activities, those in which their hard work will be duly recognized.

                              Comment


                              • Re: A call for transparency of judging in FLL

                                Originally posted by FLLmom1 View Post
                                Not so simple! This is not a matter of coaches “letting it go”. What is at stake here is the continuing participation of motivated teams in FLL. The growth of FLL in the NorCal region has in large part been from the outreach efforts of these teams.

                                Selecting a team in which “the girls spent more time shopping than working on robotics”, one whose team member responded "Go ask my dad.....", to receive the Champion’s Award, will only lead to kids who want to challenge themselves and achieve excellence by investing time and effort, abandoning FLL, to pursue alternate STEM activities, those in which their hard work will be duly recognized.
                                If a team wants individual recognition, then entering an subjectively judged event with 431 other teams isn't necessarily a successful strategy.
                                FIRST Tech Challenge Judge: 2010, Referee: 2017
                                FIRST LEGO League Mentor, Instructor, and/or Referee/Head Referee since 2011
                                FIRST Robotics Competition judge (Chairman's Award): 2014
                                Dean says I'm an "Oompa Loompa of Science"

                                Comment

                                Working...
                                X