Announcement

Collapse
No announcement yet.

Referee/Robot Game Training

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Referee/Robot Game Training

    Use this thread to discuss Referee/Robot Game Training

  • #2
    We would like to collect best practices from all FLL regions for referee training, and identify recommendations for standards for training all Referees and Head Referees.
    Steve Scherr
    Referee and Judge, Virginia-DC, Maryland, and Ohio
    FLL Global Head Referee

    Comment


    • #3
      In our region we feel we are stuck in a rut with regards to ref-training. I too am anxious to discuss/discover some concepts to make this a more effective process.

      We tend to have three types of refs: a) high-school and younger folks, FRC and/or former FLL types usually, b) interested (but busy) adults like invested parents, teachers, former FLL coaches, etc, and c) a volunteer that is pulled in often last minute to fill the spot. Each have different motivations, learning styles, and availability.

      Our region usually manages only one "practice day" type of event, often in early Nov. Informal, mostly unscheduled scrimmages, demos, project reviews, etc. We try to invite all potential refs to this event to learn the game and actually rehearse with live bots and teams. Refs at least go thru the table setup, and get acquainted with launches, interruptions, and penalties. The scoresheet is harder to get any depth on, as most teams have only a few missions operable at that point.

      Closer to Dec qualifiers, we have tried WebEx style conferences. You can put a graphic of the mat on the screen and use some basic annotations to emulate certain robot actions and discuss ref responses. The session is mostly a large open Q&A. We repeat this session for Jan Champs as well.

      But I see many events where day-of Saturday morning ref training is still where the bulk of info is learned. This can only lead to X amount of quality in my opinion, and can be a time distraction to key volunteers if there are still setup activities to complete. At events where we can schedule a "practice round" for all teams, then that has been a key training time as well. Also gives head-refs the chance to watch/review table refs in real action.

      Sorta separate, but related question of my own. As a head-ref it is fairly easy to monitor a match and ensure the table ref is handling the team/robot as needed. I find it much harder to "monitor" the filling out of scoresheets. One feels you are hovering over their shoulder to watch every mark. Sure, many teams will catch any mistakes, but many will not. I watch a pair of tables, I linger around to see if teams have any questions - but I am soon off to the next pair of tables. I know the proper decisions actions were done during the round, but I sometimes have less confidence that it got scored correctly.


      David Stolz
      Texas - South (Houston)

      Comment


      • #4
        I've watched the ref training videos here in the Referee workspace (Evergreen Rules, Prep, and EventDay). I've got to say they're not worth very much. They don't take advantage of the video format at all. It's just a badly synthesized voice reading the same text material we've been reading for a couple of years, set over static images from tournaments. I wouldn't recommend that anyone actually use these for training of new refs, as the static images aren't used to illustrate the narration, and the way the images all slowly zoom in and out gives me a bad case of vertigo.

        I had hoped for quite a lot better effort from FIRST on this. The videos that FIRST has provided are a great example of how to NOT do referee training.
        FIRST LEGO League Mentor and Referee/Head Referee since 2011.

        Comment


        • #5
          Here's the ref training method I've been using.
          Note: The region is pretty geographically spread-out, and I'm not located in the major population center. So in-person training sessions are not a practical option.

          - I write up a set of my own notes to supplement the materials that FIRST provides. The notes include the overall referee expectations, details and (when necessary) local interpretation of any rules issues, and discusses each of the missions and how to score them. These are distributed by email to all of the referees starting six to eight weeks before the local qualifying tournaments. I develop and review my notes with the help of some of the very experienced local tournament head referees.

          - Because the text in the robot game manual doesn't really cover the dynamics of how the models work, I create my own training videos that highlight specific issues for that year's game. It might be a walk-through of how to set up various models (example: last year's refrigerator), or details on how a model works (such as the brick sorter from a two years ago, or the animal exchange model from last year). I borrow a complete field setup kit and table from my local tournament partner, and use it in the video. The idea for this video is to get refs closer to recognizing what they'll see live at a tournament. I also include a walk-through of how to complete the scoresheet, and include some examples of various post-match scenarios that might lead to scoring confusion. I post this training video on a private youtube account, and provide the link to the referees.

          I had hoped that I would not have to make my own training videos this year, but (see my reply elsewhere in this thread), it looks like I will have to do so again this year. Sigh.

          This year's training video is going to include a recap of what "in base for every launch" means, as we had a significant issue with that at the last state tournament. It's an example of why you don't overlook the basics in training, regardless of how experienced the referee crew is.

          - The training sessions themselves consist of three 90-minute teleconferences (not video chats, just audio). Each teleconference is identical, and refs are expected to call-in to one of them in order to be approved to referee at a local tournament. The teleconferences are a fairly brisk walk-through of the materials I sent out in advance. with a lot of time for Q&A. The regional operating partner provides the teleconference line. We typically have 10 to 20 referees on each call.

          - After each teleconference, I send the FIRST "referee guide and quiz" document to those who were on that call, and take questions via email.

          We do not allow day-of-event training for referees. Refereeing a match is too complicated and too important to cover in a short briefing. That would not do justice to the time and energy the that teams have put into preparing for the tournament.

          For the local tournaments I keep track of the ref training status using a Google Docs form, since I only have access to VMS for the state championships event. I rely on the local head referees for qualifying tournaments to be sure that all of their refs have attended a training session.

          To referee the state championships, refs must have refereed at a local qualifying tournament for that season. I do not run an additional training session for the state championships, because the timing (across the winter holiday season) makes the logistics impossible.

          The pool of referees consists of adults only, it seems to primarily be engineers from the regional technology industry, and quite a few folks who are mentors of FTC or FRC teams.

          For staffing an event, my goal is to have two referees for each table (so four refs for each table pair), plus one head referee for each table pair. Novice referees are always paired-up with an experienced referee. The head referee watches both matches for any tricky situations, verifies that the table refs are making good decisions, audits the scoresheet marking, and is also free to spend a few moments as necessary with the teams and coaches while the next match is being set up.
          FIRST LEGO League Mentor and Referee/Head Referee since 2011.

          Comment


          • #6
            We don't have very systematic referee training across my region, but I am starting to get some ideas already from the postings here.

            We are able to bring many of our qualifier head referees together for a Tournament Directors/Key Volunteers training meeting, usually about a month after the Challenge release. We have a chance to be hands-on with the field and mission models. I have a basic slide deck (based on the Referee Guide), talking about Head Referee responsibilities, with some sample slides for a table referee training, focusing on the FLL big picture and mechanics of refereeing a match. I also put together slides for a walk-through of the missions, with setup, scoring, constraints, and any interesting details that we've noticed already. If there's a question that needs some discussion (e.g. "what is the floor of the shark tank?") I'll also use that meeting to talk with the other head referees, socialize it, and get a consensus. (I look forward to doing much of that on this forum now!)

            For in-person training, we have a couple of tournaments in my area where I get a chance to meet with all the referees and go through everything in front of a table, about a week before the tournament. That works great An evening before the tournament review is also good, because sometimes that's the first opportunity people have to see and touch the a full field.

            My ideal tournament morning is to have a morning meeting with the referee/assistant team, to go over anything new that we might have learned in previous weeks, and to have the opportunity to emphasize a couple of items. Then send the referees out to watch teams practice in the pit, so that they can mentally go over judgement calls, and also have the opportunity to mention to teams when they notice something that might not be allowable. Afterwards, a practice round helps the volunteers settle in and orients the teams, and then we're ready to go! (This doesn't happen nearly often enough--even when I get to design the schedule for the tournament...)

            At the Championship-level events, it works great to have a second Head Ref who can talk with the competition area volunteers when I am occupied with the Coaches' Meeting.


            I'm not sure what I expected from the evergreen rules video, which I watched/listened to today. If it is a first introduction to the Robot Game, then it's probably good that it focuses on the Rules text, especially since people have different learning styles. I'll always point them to the written text afterwards, of course, because that's what GP5 says to do. One of the things that I am hoping to get from everyone's participation in this forum is the next step--collecting the body of knowledge that we want to provide to new/experienced referees to help us ensure that the game is refereed consistently.

            Steve Scherr
            Virginia-DC Referee Advisor, and hanger-on in other places
            Last edited by scherrsj; 08-10-2017, 12:17 AM.
            Steve Scherr
            Referee and Judge, Virginia-DC, Maryland, and Ohio
            FLL Global Head Referee

            Comment


            • #7
              I cannot overemphasize the importance of a timed and scored practice round before the official matches. This serves many purposes, it gets the teams oriented to the competition process (what I call the "choreography") and it enables the referees to get on-the-job training before it really counts. With the table referees on the tables, the head referee uses this time to observe and correct any weaknesses in the officiating. I also find that most teams have no idea on how to score the score sheets and by scoring it in an unofficial manner, they get that practice. I also require the scorekeepers to attend the practice matches since this is the first time the "scoring system" (people, equipment and processes) is exercised. I consider such a practice match mandatory for all of my events.

              One other related topic that I would like to offer in this forum is the concept of a "supervisory referee". For tournaments that have N tables running concurrently (where a "table" is two teams, two robots and two field setup kits), there should be N supervisory referees if you can get enough volunteers. The table referees are doing the active monitoring and scoring of each team, but the supervisory referees are overseeing two table referees for assistance. There still is only one official head referee according to the rules (who can be one of the supervisory referees), but the supervisory referees make for a much more consistent experience for the teams. In summary, for each event where there are N tables, there are 2*N table referees, N supervisory referees and 1 head referee (where the head referee can be a supervisory referee).

              Finally, as part of the competition day training, having a referee "huddle" after each particular round (practice, 1, 2, 3) to discuss controversial team tactics and associated referee rulings are invaluable. This is an effective way to ensure consistency (e.g. in round 1, the referee on the Red Table saw team 1324 do XYZ and ruled that it was allowed according to the rules. Now when Team 1324 goes to the Green Table for round 2, that table's referee is ready to observe the same tactic and will rule the same way.).

              Bill Aucoin (AKA RobotBill)
              Referee & Head Referee, Judge & Judge Advisor
              DC-MD-VA (and now probably CA)
              Bill Aucoin

              AKA Robot Bill
              RobotBillMD@gmail.com

              Comment


              • #8
                Hi Bill,
                There's a lot of helpful advice in your post.
                Running a practice round is a great idea, but there isn't time in the day to fit one in with the event schedule that my operating partner uses.
                At an event with multiple table pairs, I also like to have a "local" head referee for each pair.
                And the huddle-up with the referees after each round is very effective also.
                At a tournament with just one pair of tables, one of my tasks as head ref is to scout the first-round matches so I can tip-off the table referees for issues to look for in the later rounds. My schedule ends up looking like a baseball game scorecard with all sorts of scribbles and notes in the margin.
                FIRST LEGO League Mentor and Referee/Head Referee since 2011.

                Comment


                • #9
                  We hold in-person compulsory training for referees before the event (not the event morning).
                  Last year I tried the following which drew awareness, and seemed to 'spice up' the training:
                  I selected some video clips of robot runs that teams published (made sure that these are not teams that any of these referees might judge, for examples teams from other countries).
                  I showed these clips (usually partial runs, only a short part I thought is interesting), and had the trainees discuss these.
                  Yosi K. Karl
                  Mentor for the 'Lego Panda' Fll team #261, Modiin, Israel

                  Comment


                  • #10
                    Originally posted by YosiKarl View Post
                    We hold in-person compulsory training for referees before the event (not the event morning).
                    Last year I tried the following which drew awareness, and seemed to 'spice up' the training:
                    I selected some video clips of robot runs that teams published (made sure that these are not teams that any of these referees might judge, for examples teams from other countries).
                    I showed these clips (usually partial runs, only a short part I thought is interesting), and had the trainees discuss these.
                    That's a very good idea.
                    FIRST LEGO League Mentor and Referee/Head Referee since 2011.

                    Comment


                    • #11
                      I've searched so I hope this is not a duplicate question. Are there referee training materials available for the Into Orbit Challenge specifically and being a referee in general. This is my first year as a head ref. Please provide links to the material(s).

                      Comment


                      • #12
                        Originally posted by dstrich View Post
                        I've searched so I hope this is not a duplicate question. Are there referee training materials available for the Into Orbit Challenge specifically and being a referee in general. This is my first year as a head ref. Please provide links to the material(s).
                        FIRST publishes some referee training materials via the "Partner Wiki". In the past it's been a bit sparse - not nearly as complete as the materials for Judge training. I understand there might be some improvements being made for this season.

                        Your operating partner has to authorize you to be granted access to the Wiki.

                        Here is the link, but you need an account to log in.
                        https://firstlegoleague.pbworks.com/...o_page=%2Fn%2F
                        FIRST LEGO League Mentor and Referee/Head Referee since 2011.

                        Comment

                        Working...
                        X