Announcement

Collapse
No announcement yet.

Feedback on FLL Regional competition held in Bangalore, India on 21st January,2018

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Feedback on FLL Regional competition held in Bangalore, India on 21st January,2018

    I am writing to you with regard to the First Lego League Regional competition which happened in Bangalore, India on 21st January,2018


    Our team, Hydronators, had participated in it. I would like to point out a few things that we observed and brought to the notice of India STEM Foundation, organizers of FLL in India, during the course of the event. But they were left completely ignored and unaddressed.


    1. After our teams turn in the game, our team was asked to sign-off on a certain score. When our team disputed, they were threatened to be disqualified. We had recorded a video of our teams' robot game and had proof of certain missions done by our team but the organizers refused the see the video and said that the judges decision was final.Despite requesting them several times, they refused to see the video taken by us. When asked why they hadn't recorded it, the organizer replied that ''We dont even know what is a video'' !!

    Now, STEM Foundation has been posting videos of some games on their facebook page for the sake of marketing and publicity. I am also aware that there was a professional photographer and a videographer at the event. Then why is STEM foundation refusing to re look at the recorded videos and resolve the dispute raised by the team?


    2. During the announcement of the results, the organizers made no mention of the scores obtained by the qualifying teams for the national Level competition. We had keenly observed every teams performance and were very sure about a certain teams' performance which was absolutely not upto the mark. But on seeing them qualify for the nationals, a couple of other teams raised a dispute, but the organizers simply refused to address the issue. We asked them to show the score sheet but neither were they ready to show the score sheets nor did they even tell the score verbally.

    They insisted that they will share the scanned copies on email after a few days, but will not share immediately despite having the copies in their hand.

    Why are the scores kept a secret? When the game is being conducted openly and everyone can watch it, why aren't the scores also displayed openly OR at the very least announced in the end?

    3. We also observed that Coaches of certain teams are judges for Jr FLL/ Robot design /Core Values/ Project. Isnt that a conflict of interest? Why cant STEM foundation have some neutral parties coming in for judging ?

    4. During the announcement of results,the organizers- India STEM Foundation, just announced the winning teams. There was no effort made to even spend 2 minutes to share with the audience, what the winning project was all about OR why was a certain team awarded the best Robot design award. We need not see the entire project presentation but at the very least some mention and a brief description of the same needs to be made. After all this has to be a learning experience. If we know how the other teams performed, we would know what to strive for in our next attempt.

    The way it was organized didn't show any inclination towards gathering learning experiences.

    I run a small institute named STEMdroid Maker Shed ( http://www.stemdroid.in ; http://www.facebook.com/stemdroid) where we impart STEM learning to Children aged 6-16 years of age. Participating in competitions such as FLL is one of the ways that we are able to garner interest from our students into STEM learning. Many of my pupil participated in today's' competition and many parents were also there to witness the event but the turn of events at the competition and lack of transparency that I described above, left them utterly disappointed. I would certainly find it very difficult to motivate my students to participate in such events in future, especially after witnessing the unfair and unprofessional way in which India STEM Foundation conducted the event.

  • #2
    1) In 7 years of participating in FLL, I have *never* seen a team insist on a ref watching a replay video and never seen a ref/organization oblige. In FLL, the robot game is not the primary focus even though it is the most visible part of the contest. The ref decides whether or not the kids should get the benefit of the doubt and the kids need to be gracious and sign off after they have had a chance to state their case. Sure, there are plenty of times when my own kids were probably not in agreement with the call...but they let it go. #MoreThanRobots.

    2) Qualifications to the next level is not about the robot game score if - the other team may have done very well in judging. There are official advancement criteria that FIRST puts out. Also, while it is nice to see all the robot game scores posted (and most often they are), remember that teams are competing with themselves, not other teams. So, ask for your own team's score and rubrics sheet and see if the kids reached their potential. It is not uncommon that the rubrics and the scoresheet the kids signed off on are returned a few days later.

    3) Events are often short on volunteers. While it is far from ideal to have coaches/parents from other teams judging in the same program (FLL), perhaps they were short volunteers and had to. Parents and coaches abide by Gracious Professionalism rules as well and the Judge Advisor can make sure that no bias influenced their scoring.

    4) No tournament I have ever been in spends 2 mins describing each award winning team. Perhaps 10 seconds or less with a very cryptic description of the team's work seems to be standard FLL policy. I suggest that your kids walk around and talk to the other teams and learn from each other.

    Do not be discouraged. Remember that everyone is a volunteer. Remember that FIRST LEGO League is not the same as WRO or other competitions you may have encountered. FLL is more than the robot score. Gracious Professionalism is key driver for the kids and adults in FIRST.
    Mentor: Not the Droids You Are Looking For - retired (World Festival Champion's 2018)
    Judge: Western PA, World Festival, Razorback Open, Mountain State
    Head Referee: Western PA Championships

    Comment


    • #3
      The tournaments we attend specifically call out that they won't be accepting video or photography for scoring disputes.
      Coach, FLL Team 3146 Peace By Piece 2013 - 2016; Team 29410 The Dragon Bots 2016-2018
      Judge, FTC 2014-2015; Field Technical Advisor, FTC 2016-2018; Robot Inspector, FRC 2018

      Comment


      • #4
        Occasionally there is a call for more transparency in judging and awards. A few years ago something happen and the N. Cal. tournament that resulted in a thread with over 198 posts and 62,000 views. So you aren't alone thinking that FLL does some things wrong. But if you look at the number of teams competing in FLL and compare that to the number of complaints, you are in a very small minority. Still doesn't mean you are wrong.

        I don't agree with the idea of using video review for the robot game. Having video review means that it is important to get the score right. That isn't the case. The score for the robot game is not important. This is not a robot competition. FLL is a STEM activity that uses competition go generate interest and because competition is fun. It isn't important that you win an award or advance to the World Festival. It is important that the team learns some things and associates STEM with fun. You get three runs to get your best score. You might get jobbed on a few points, it happens, but it is unlikely to happen three times in a row. My girls lost out on winning the robot game once because of a scoring error. They learned to graciously accept that errors happen and thanked the referees and judges for volunteering their time, just like they did at every other tournament. They also learned to be a lot more careful about signing the score sheet.

        I find it odd that robot game scores were not posted at your tournament. I haven't been to a lot of tournaments, but I have talked to a lot of coaches, and it is common for scores to be posted during the competition. My teams don't know their score until it is posted for all to see. The "score sheet" doesn't have any scores, only check boxes for what missions were accomplished. The team could do the math, but the anticipation of waiting with the other teams for scores to appear is intoxicating. They come running back to the pit area screaming "We got 105! We're in 2nd place!". Maybe your affiliate doesn't like running and screaming and jubilation. I can't think of any other reason for keeping robot game results secret. I agree this is strange and should change.

        There is not a big pool of FLL judges. You can have people who know about FLL and know how to program and build robots and understand the research project and core values or you can have people who aren't FLL coaches. I judge when I'm not coaching a team at a tournament. Because I mentor multiple teams this sometimes means I'm judging at a tournament where I have an affiliation with a team (spell check changed that to affliction which is funny because it is sadly appropriate). Once I judged a team I had mentored for that season. That was awkward and I think unfair to that team. We now have a field on the tournament signup page for entering numbers for teams you are affiliated with. A coach or mentor should not judge their own team, but I have no problem with coaches acting as judges otherwise.

        There is a reason why the award announcement is short. They are written by really tired judges at the end of a really busy day. My comments on the judging forms are copious, eloquent and insightful. However, by the time I've filled out that last form and we've picked the award winners I don't know if I could spell "CAT" if you spotted me the "C" and the "A". So what you get is a short little quip that only I, and possibly the team, find funny.

        Things haven't always been this way. Way back in the early days of FLL my affiliate had the winning project teams deliver their presentation at the award ceremony. What a great idea! Never mind that some FLL teams have a hard time working up the courage to present in front of two judges and a handful of parents. There should be no problem speaking in front of 600+ people at the end of an emotionally draining day. There was also the problem that the presentation is 5 minutes followed by a 10 minute interview, and a lot of the presentation awards are won during the interview. Sometimes the award presentation left teams thinking "What was so great about that?" Maybe nothing was great about that presentation and the great thing was when the presentation judge asks "If these enzymes in tea are good for your dental health, why not drink tea instead of using your dental health chewing gum?" This is followed by a thoughtful pause and the answer "Not everyone has access to clean water for making tea." The project presentstion was dropped from the awards ceremony after a few seasons and nobody misses it.

        I am a big fan of transparency. In Minnesota we have an open door policy for robot design and project judging. Due to the nature of the instant challenge we don't allow spectators for Core Values judging, This is not common, but I cannot understand why. Rookie teams can go to a tournament to watch the robot game and sit on robot design and project judging. Teams with a record of success get to do some inadvertent sharing when other teams sit in to learn how you can score 400 points in the robot game or what it takes to win the creative presentation award three years in a row.
        Last edited by Dean Hystad; 01-24-2018, 01:42 AM.

        Comment


        • #5
          Originally posted by stemdroid View Post
          [B]1. After our teams turn in the game, our team was asked to sign-off on a certain score. When our team disputed, they were threatened to be disqualified. We had recorded a video of our teams' robot game and had proof of certain missions done by our team but the organizers refused the see the video and said that the judges decision was final.Despite requesting them several times, they refused to see the video taken by us. When asked why they hadn't recorded it, the organizer replied that ''We dont even know what is a video'' !!

          Now, STEM Foundation has been posting videos of some games on their facebook page for the sake of marketing and publicity. I am also aware that there was a professional photographer and a videographer at the event. Then why is STEM foundation refusing to re look at the recorded videos and resolve the dispute raised by the team?
          I'm not aware of any region here in US/CAN that would accept video for a dispute and I don't imagine it would be any different in the rest of the world. A video doesn't matter because (1) there is a live person watching the field and your team is to sign off on the sheet or the Head Referee will make a final call if there is anything called into question and (2) our program is More Than Robots. And I'm not saying your team shouldn't have been given the benefit of the doubt, and I'm not saying that didn't happen either, but when a Head Referee makes a call, it is final, and questioning volunteer decisions--especially with adult involvement--is in violation of the Challenge Guide/Participation Rules and can lead to consideration for disqualification. I understand this can be frustrating but really--it isn't about the robot scores.

          2. During the announcement of the results, the organizers made no mention of the scores obtained by the qualifying teams for the national Level competition. We had keenly observed every teams performance and were very sure about a certain teams' performance which was absolutely not upto the mark. But on seeing them qualify for the nationals, a couple of other teams raised a dispute, but the organizers simply refused to address the issue. We asked them to show the score sheet but neither were they ready to show the score sheets nor did they even tell the score verbally. They insisted that they will share the scanned copies on email after a few days, but will not share immediately despite having the copies in their hand. Why are the scores kept a secret? When the game is being conducted openly and everyone can watch it, why aren't the scores also displayed openly OR at the very least announced in the end?
          Game scores are not secret by any means, but deliberations to determine advancing teams is and anything decided is final. How they choose to announce teams is up to them, and I understand where you're coming from with it being "fishy" but I imagine the real issue was they didn't want to argue with you about it while they were at the event. I'll also add here it really is not about the Robot Game Score anymore (and never was, really). It is about embracing all aspects of the program. More Than Robots. Another note: Sometimes event organizers need time to review disputes before speaking about them. For instance, if a team contacts me and says something unfair happened at an event, I have to email the Judge Advisor/Head Referee and the Event Host and review that event's files before I can speak to anything that happened.

          3. We also observed that Coaches of certain teams are judges for Jr FLL/ Robot design /Core Values/ Project. Isnt that a conflict of interest? Why cant STEM foundation have some neutral parties coming in for judging ?
          In my 11 years, I haven't ONCE ran an event that was fully staffed as ideally as we'd like. If we enter a scenario, for instance, where we are short on Judges, we have to do what is necessary to minimize the impact or adverse affects on teams and if that means parents are in a Judging Room, as long as they aren't judging their own team, that isn't a big deal. Judge Advisors and Partners are trained to mitigate conflicts of interest in deliberations if necessary anyways. (I can only speak to US/CAN here because the training I refer to isn't mandatory outside of US/CAN). Because of the way our Deliberations System and Process is designed, it also helps to mitigate these issues should they arise.

          4. During the announcement of results,the organizers- India STEM Foundation, just announced the winning teams. There was no effort made to even spend 2 minutes to share with the audience, what the winning project was all about OR why was a certain team awarded the best Robot design award. We need not see the entire project presentation but at the very least some mention and a brief description of the same needs to be made. After all this has to be a learning experience. If we know how the other teams performed, we would know what to strive for in our next attempt.
          Sometimes event organizers need to be out of a venue by a certain time, sometimes the event runs long and everyone is tired, sometimes they just are interested in announcing the awards and calling it a day. There is no requirement that any specific scripting be used. Sure, it would be nice, but it isn't required. This would be good feedback to share directly, that you would like to see something like this at a future event, and perhaps you can volunteer to work with them and design a scripting template for them as a suggestion.

          Comment


          • #6
            I think all of us who have responded to the thread are US/Canada. I know someone else who was at that event and asked for additional insight.

            Apparently, the normal advancement criteria we in the US are familiar with was not used. Teams were told they need 100pts on the game to move on to the National level. They didn't need to win or do anything particular in another category. The teams were not informed of their score at the end of their runs. The 8 teams that moved on were simply announced at the end of the event. It sounds like it was all about the robot. :-(

            I can understand why stemdroid was anxious to see the score. Even teams that moved on have no idea what score they got.

            The story seems consistent with the information on this page: https://www.indiastemfoundation.org/...sion-criteria/
            Mentor: Not the Droids You Are Looking For - retired (World Festival Champion's 2018)
            Judge: Western PA, World Festival, Razorback Open, Mountain State
            Head Referee: Western PA Championships

            Comment


            • #7
              With every other team advancing I can see why they are using a different criteria. FLL criteria start to break down when you advance 25% of the teams and you have to start adjusting the robot game threshold numbers, otherwise it is the only thing that decides who advances and the project, robot design and core values mean nothing.

              This is the problem with treating the robot game differently than project, design and core values. When everything was weighted evenly at 25% you didn't have to worry about the size of the tournament and the number of teams advancing. I never had a chance to see which problem this modified scoring was meant to fix.

              Comment


              • #8
                Originally posted by shelbydavis View Post
                The tournaments we attend specifically call out that they won't be accepting video or photography for scoring disputes.
                It's not a tournament thing; it's in the rules. GP5 - Information Superiority says "Pictures and video have no authority, except when talked about in #1, #2, or #3. (The numbers refer to the order listed earlier in GP5).

                From a practical standpoint, trying to watch video to review something is difficult. Often, the angle and/or resolution wouldn't show the details of whatever referee ruling is being questioned. Many teams don't have access to the decent video equipment, so accepting video would introduce inequalities into something that really doesn't need that depth of review. And there's rarely enough time to do that kind of review anyway. As many others have pointed out in this thread, it's Not About the Robot.
                Kansas City Region Head Ref 2014-present
                KC Region coaches and teams can ask FLL robot game rules questions at kcfllref@gmail.com

                Comment


                • #9
                  Originally posted by stemdroid View Post
                  1. After our teams turn in the game, our team was asked to sign-off on a certain score. When our team disputed, they were threatened to be disqualified. We had recorded a video of our teams' robot game and had proof of certain missions done by our team but the organizers refused the see the video and said that the judges decision was final.Despite requesting them several times, they refused to see the video taken by us. When asked why they hadn't recorded it, the organizer replied that ''We dont even know what is a video'' !!
                  On our regional compatition in Germany we had videocameras everywhere because the event was streamed on Youtube. But no one watched a Video to clarify any discussion (this would cost a lot of time). If the Kids had some complain with a decision of the Jury while Robot game, they discuss with them. The judges was friendly, patient (even while longer discussions) and explain the reason why the decision was made. I think it is part of learning the not each Situation can be won by the Kids. Sometimes they have to accept a decision, even if they dont like it.
                  On the other competionions (design, Team work, research) we dont get insight in the decision-process while competition is running. We get the Sheets nearly two weeks after.

                  Originally posted by stemdroid View Post
                  2. During the announcement of the results, the organizers made no mention of the scores obtained [...]
                  All results in middle-europe you find on the Website for each competition, I dont know why this isnt possible in india. But im sure there is a reson for. Maybe you should talk with them.

                  Originally posted by stemdroid View Post
                  3. We also observed that Coaches of certain teams are judges for Jr FLL/ Robot design /Core Values/ Project. Isnt that a conflict of interest? Why cant STEM foundation have some neutral parties coming in for judging ?
                  We had this too, but we feel never that they privilege theyr own team.

                  Originally posted by stemdroid View Post
                  4. During the announcement of results,the organizers- India STEM Foundation, just announced the winning teams. There was no effort made to even spend 2 minutes to share with the audience, what the winning project was [...]
                  Same here, i whished they told us more. But it is a question of time and effort to collect and process the Information in a matter other People understand this.

                  Originally posted by stemdroid View Post
                  I run a small institute named STEMdroid Maker Shed ( http://www.stemdroid.in ; http://www.facebook.com/stemdroid) where we impart STEM learning to Children aged 6-16 years of age.
                  I have a team of Kids here, just for fun. I spend all expenses and dont get money from the parents. It is my Hobby. Most of the people i met while the competition and during talkings with other teams dont make it with a professional background. Maybe there is a reson for the difference in our view. I think it is okay how they do it here and i wouldnt complain much about the way in india (even if i dont understand some situations you had). But if you get money for your time with the kids you will try to deliver a good result and want / need to explain some disappointments.

                  Maybe it is a good idea to try to contact the origaniszation and try to be a part of the process. It is easier to understand Details from a insight view.

                  Comment


                  • #10
                    I would recommend that you volunteer as a judge at a tournament. Without the emotion of your team competing, you can see how the process works and the great efforts the judges take to ensure a fair competition.

                    Comment


                    • #11
                      Originally posted by CoachEnrico View Post

                      All results in middle-europe you find on the Website for each competition, I dont know why this isnt possible in india. But im sure there is a reson for. Maybe you should talk with them.
                      Hi Enrico,
                      I am also from Germany. The "problem" with the publishing the scores in other countries is that they simply don't have scores. It seems like Central Europe is (almost?) the only region worldwide that has the scoring system with 50 points for each category. In most other countries, there are only evaluations and judging discussions, but in the end, the decisions about the award winners, including the Champions award, are subjective and the result of those discussions - and not, like here, (almost completely) the result points on the scoresheets.

                      Comment


                      • #12
                        Hi All,

                        This has been a long discussion thread, it is only reasonable that I respond back after the feedback that I have provided.

                        I am completely with all of you when you point out that FLL is an holistic learning experience and should not be looked at the myopic view of Robot Game and the points scored by virtue of completing the missions. That is what I as a coach have always believed in and have taught my pupil.

                        As feedbacks go, they are an opportunity to listen, understand, fix & course correct if possible. If we all just "let it go.", why strive for making things better than they already are.

                        @ sseshan: Thanks for sharing the criteria of qualification to next level, but, I was always guided by the progression criteria described in the link. https://www.indiastemfoundation.org/fllindia/progression-criteria/ & was agreeable to it. Yes, "The teams were not informed of their score at the end of their runs. The 8 teams that moved on were simply announced at the end of the event. It sounds like it was all about the robot. " . I wish it was more than that.

                        @Dean Hystad, @ JT Yoerger : I am completely with you if say video might not be the best way to review a robot game, but, I am sure you will agree with me that there must be some method of review. When team/coach requests for a review, they should not be at least brushed aside saying "We were busy", or even worse, threatening disqualification to participants.

                        @Dean Hystad : The request for transparency is usually raised when the participants feel a tinge of unfairness, at that moment, the organizers should be willing to communicate & be as transparent as possible to allay some misunderstandings. @sseshan, you've pointed out "ask for your own team's score and rubrics sheet and see if the kids reached their potential" . That's exactly what we asked for and have not been provided our own rubrics sheet yet.

                        @ CoachEnrico : Just so that you know, I don't get any money for the time with kids either, this is not about delivering a good result, it is about fairness and transparency. I don't think I am asking for too much.

                        Dear members, I do not intend to keep on harping about the need for transparency, it is the Foundations' choice. The Robot game is indeed the most visible aspect, I haven't participated in the Robot Design/Project/Core Value presentation sessions & hence unable to comment about them. I will indeed be participating in subsequent FLL seasons, but, will always have this need for more transparency lingering in my head.

                        Comment


                        • #13
                          [QUOTE=stemdroid;n83787]Hi All,

                          @sseshan, you've pointed out "ask for your own team's score and rubrics sheet and see if the kids reached their potential" . That's exactly what we asked for and have not been provided our own rubrics sheet yet.

                          My friend in Bangalore said that rubrics have already been provided to teams (a few days ago, in fact). If you didn't receive yours for some reason, contact the organizer. I am sure it will help your kids improve for next year.

                          @ CoachEnrico : Just so that you know, I don't get any money for the time with kids either, this is not about delivering a good result, it is about fairness and transparency. I don't think I am asking for too much.

                          You provided a link to a company/business. Isn't the FLL team through the academy/company you linked?
                          Mentor: Not the Droids You Are Looking For - retired (World Festival Champion's 2018)
                          Judge: Western PA, World Festival, Razorback Open, Mountain State
                          Head Referee: Western PA Championships

                          Comment


                          • #14
                            @sseshan,

                            1.Thanks, just checked my inbox, there doesn't seem to be any. Will followup with India Stem Foundation.
                            2. Sure, STEMdroid does charge a fee for various courses, The costs of FLL are borne at actual by parents, we even pool funds to buy samosas ;-) .STEMdroid contributes by providing a place to work when they don't have their regular class room sessions.

                            Comment


                            • #15
                              Originally posted by sseshan View Post
                              I think all of us who have responded to the thread are US/Canada. I know someone else who was at that event and asked for additional insight.

                              Apparently, the normal advancement criteria we in the US are familiar with was not used. Teams were told they need 100pts on the game to move on to the National level. They didn't need to win or do anything particular in another category. The teams were not informed of their score at the end of their runs. The 8 teams that moved on were simply announced at the end of the event. It sounds like it was all about the robot. :-(

                              I can understand why stemdroid was anxious to see the score. Even teams that moved on have no idea what score they got.

                              The story seems consistent with the information on this page: https://www.indiastemfoundation.org/...sion-criteria/
                              The FIRST LEGO League Advancement Policy's implementation is global, not based in US/CAN, as it is enforced through our Global Program Standards. I can't speak to any politics though nor how these events are structured because Global Standards only apply to "official events" -- Based on the link I'm not sure because it depends on how their advancement is structured, and even then, there may be specific local exceptions made to India -- The issue would need to be addressed with LEGO Education as they administer the program and manage Operational Partners that are based outside of US/CAN, not FIRST. Personally, I also strongly disagree with the philosophy listed here of requiring "100 points" specifically because I think it does, indeed, make it all about the robot -- which it isn't supposed to be and the Advancement Policy was intentionally revised this year to allow for almost all teams attending to be eligible for advancement irregardless of robot performance insofar as such performance wasn't below the 75% threshold.

                              Originally posted by stemdroid View Post
                              @ JT Yoerger : I am completely with you if say video might not be the best way to review a robot game, but, I am sure you will agree with me that there must be some method of review. When team/coach requests for a review, they should not be at least brushed aside saying "We were busy", or even worse, threatening disqualification to participants.
                              I agree. You'll have to speak with the Operational Partner about this.

                              Comment

                              Working...
                              X