ICS 31/32/33 TA Guide

[To read this most easily, make your browser window narrow.]

In a research university, TAs provide some of the closest attention students receive early in their careers. TAs have a great opportunity to shape students' experiences (positively or negatively). We hope this guide will help you and we welcome suggestions for improvement. Note that details vary from course to course and from instructor to instructor, so consult the instructor of your course for definitive information. [You may find this most readable in a narrow browser window.]

1. Before the quarter starts
• If you don't have one already, obtain a UCInet ID and also an ICS Lab Account for access to networks and campus services.
• Take a look around these web services we may be using. Not all have documentation on line and we haven't made final decisions yet on which we'll use in Fall 2016:
• EEE, UCI's Electronic Educational Environment, which we'll use for recording scores, getting rosters, and administering online quizzes. EEE is gradually merging with the commercial learnng management system Canvas.
• Checkmate, ICS's system for students to submit solutions to assignments (checkmate.ics.uci.edu)
• Piazza, where students can get their course-related questions answered quickly (piazza.com)
• UCI Replay, where recordings of each class meeting (audio plus screen capture) will be posted (with about 90% reliability) (replay.uci.edu, but links to the recordings will be sent to the class mailing list and archived on EEE)
• Gradescope.com, for expediting the grading of paper exams (and possibly assignments)
• The ICS Partner App, for expediting the grading of paper exams (and possibly assignments)
• Pythontutor.com, which gives illustrated traces of Python program execution
• codingbat.com, which provides online programming exercises
• You should locate these materials on the web or on paper:
• Course syllabus/reference sheet
• Course reference sheet, including course outline, policies, and due dates. This is available on the Web through eee.uci.edu or from the instructor's home page.
• ICS academic honesty policy and any policies specific to the instructor of the course
• Course assignments page (typically linked from the syllabus on line)
• Textbooks are (or soon will be) available for TAs to check out from Lumen Hwang in the ICS Student Affairs office.
• Read these documents thoroughly; they state policies your students (and you) are bound by.
2. During the first week and first lab(s)
• Check your electronic mail frequently. We will announce meetings, schedule changes, new policies, and so on; your students will suffer if you don't have the up-do-date information. Get into this habit now, and keep it throughout the quarter.
• Mark your calendar for the afternoons and evenings following the midterm(s) and final exam. We will get together and do all the grading of all the exams and recording the scores at that single time; things go fastest and most smoothly that way. Bring a laptop; we'll grade using Gradescope (on scanned exams). TAs and the instructor will meet again the day after the final exam to set the final grades. Every TA must do his or her part for this process to run smoothly. In particular, this means that all of your labs must be graded and recorded in the EEE GradeBook the day before the final exam. Since you know this now, please arrange your own exam studying and travel schedule to accommodate these constraints.
• TAs can get a roster of enrolled students via EEE. If students add the course late, keep a record of when they added (so you don't penalize them for making up the lab assignments after the deadline).
• Your students need to create accounts to get access to the ICS lab machines, but they don't have to do it immediately. For a week or two, they can log in as ics-temp with the password Anteat3r . On their own time, they should go to the third-floor lab and sign up for an ICS account.
• Consider setting office hours. With scheduled lab hours three days a week, students can see you regularly. But sometimes students can use five or ten minutes of one-on-one explanation. The conventional way to provide this is by scheduling some office hours; a difficulty with that approach is that sometimes nobody will show up to a scheduled hour and some students will have conflicts with whatever hours you schedule. It's fine to make the gesture by scheduling an hour or two, but these days it's equally fine not to schedule hours in advance and instead to make it clear repeatedly to the class that you will be glad to consult with them one-on-one; they just need to ask you and you'll work out a mutually agreeable time. Experience shows that the latter approach consumes less of your time in total. You can devote some of the time you save to answering questions on Piazza.
• Look at the photo roster of your students (under WebRosters on EEE) so you can learn their names. You should learn names not only because students will learn better if they know you know them by name, but also because you'll be able to identify any non-students who try to come and take an exam in a real student's place.
• Check that they've completed the demographic questionnaire at EEE and nag them until they comply. (We have yet to work out a mechanism for separating your sections' students from the group.)
• Students must choose a different partner for each lab assignment; they may not repeat partners during the quarter. The ICS PartnerApp should enforce these rules, but you still need to know what the rules are.l
• Most students should pick their partners for the next week's lab at the end of Friday's lab. Any stragglers can partner up on Monday. If there's one odd student, that student can join an existing pair (but only once during the quarter) or can work alone (also only once). But if another odd student shows up, the third person or solo worker should join the latecomer to form a pair. The ICS Partner App allows a TA (not a student) to register permission for solos or trios; make sure you do that so the student isn't penalized for doing it without authorization.
• It's generally best for students of similar experience and ability to be partners; that makes the give and take more even. However, it's hard to measure ability and experience levels and the pairings will never be perfect. Here's a technique to try early in the quarter: Have all the students stand up and form a rough line, with more experienced students on one end of the room and less experienced students on the other. The ordering doesn't have to be perfect, but you can suggest that they choose a partner from the people who are nearby.
• Students will complete partner evaluations at the end of each lab, using the ICS Partner App.
3. At every lab session
• Be on time. Students depend on having you there at the scheduled time. And don't cut out early.
• Be prepared to answer students' questions. Presumably you already know the subject matter, but you must also know the assignments. The only way to do that effectively is to read them carefully in advance and, where possible, try to work them out. If a student asks whether something is important or required, you want to be certain that you're giving the right answer (and check with the instructor if there's a question that the assignment writeup doesn't answer clearly): If you give a wrong answer on something like this, the student may be graded down, and that's not a happy situation. Don't guess about assignment requirements; if you're not sure, check with the instructor (or post on Piazza) before giving the student a definitive answer.
• Make yourself available to the students: Even if the students all seem busy, don't read your own e-mail, don't IM your friends, don't make cellphone calls, don't do your own classwork, because students will hesitate to interrupt you. When you're in the lab, you should circulate around and help students, not simply wait for them to call on you. Don't get so deeply involved in conversations that you don't notice when students need help; you should cruise around the room at least once every five minutes. Whenever you're in the instructional lab, you should be prepared to answer students' questions courteously, even if you aren't officially on duty at that time. It's unfriendly to tell a student, "I'm not even supposed to be here now," even if it's technically true. If you need to do your own work on lab machines, you may be able to work with fewer interruptions in the Open Lab (ICS 364).
• Spread yourself around to everyone rather than concentrating on just a few. If many people want you at once, leave the current person with a task to do independently while you make the rounds. Make sure each student gets at least "first aid" help so they can get back to work; after that, feel free to spend more in-depth time with students who need it. We don't want students to be able to say, "We couldn't get anything done in lab because we had to wait an hour to talk to the TA." (Avoid even joking about things like only paying attention to the cutest students; some people won't take it as a joke.)
• Give students help, not answers. It's fine to tell people how to solve system-related problems, or how language features work, but you should lead them to their own solutions of programming and CS problems. They shouldn't become dependent on you; they should learn for themselves how to use the web safely for reference, how to insert extra debugging print statements, how to do execution traces. Be pleasant but ruthless about this. Students will beg you for a quick-fix answer, and it's easier for you just to give it than to point them towards finding the solution themselves. [This applies to programming problems; early in the quarter you can answer direct questions about how to use IDLE and what the syntax of a Python feature is.] Nonetheless, resist the temptation, because students will come to expect you to fix their problems for them. In the long run, that hurts both them and you. Your class may expect students to use a "design recipe," writing out a function signature (input types and return type), purpose statement (docstring comment), and examples (in the executable form of assert statements) before they write any code; if so, insist on seeing these steps.
• Encourage students to follow pair programming for assignments that require it: two people, one computer, one driver, one navigator, switch roles regularly. Make sure the partners are talking and changing roles, and that the less experienced students don't just sit quietly and watch. If they don't get into this pattern right at the start, it will be much harder to get them to do it later. If you see a pair member who seems disengaged, that's a cue that you can intervene to provide some help.
• Keep an eye out for academic dishonesty. Pair programming does not mean that it's okay to get answers from anyone or anywhere. Students may freely communicate with their partner, with the lab tutors, with their TA, and with the instructor. Students may also post questions on Piazza.com, though their posting should not include significant portions of lab assignment solution code. Communication with others (non-partner classmates and outsiders) is limited to questions about language features and system setup. This quarter, we'll be a little more relaxed about cross-communication in the first three weeks (for the first three lab assignments); we won't worry much about students getting help from each other as they get up to speed. But starting with week 4, the official guidelines are in effect.
• Help students solve problems using what they already know. Especially in Python, there may be some advanced language feature or library function that solves the problem in a snap. But the goal is to promote their abilities with the basic features covered in class and the textbook. Covering advanced features too soon places too great a cognitive burden on the students.
• Find out what the students are supposed to know. To help students use what they already know (or are supposed to have learned), you have to know what that is. Most instructors have published course notes or a schedule keyed to a textbook; it's part of your job to keep up with these so you don't inadvertently tell the students something they're not ready for. A key principle of teaching introductory programming is that you can't teach everything all at once—you have to leave some things out or cover them later. It's the instructor who gets to decide what the schedule is, and the TAs and tutors need to be aware of those decisions and support them.
• Don't feel guilty if you can't immediately identify the student's problem or bug. While most of us can see bugs quickly for the early, trivial assignments, that may lead students to expect (unrealistically) that we'll always be able to find and fix their problems at a glance. Of course nobody can do that for significant programs, and TAs need to make this clear to students, by explaining it and by encouraging them to find their own bugs by adding unit tests, test cases, extra print statements, and tracing their code—whichever methods your instructor has taught them. (You may have to show some students how to do this step by step the first time; not everybody knows how to debug instinctively.) If a student objects to the painstaking work involved in such tracing, remind them that they can minimize the need for it by spending more time in the design phase. Some novice students program "expressionistically," just putting down code that "feels right"; they need to learn more systematic design.
• Remind students to prepare for lab by reading and thinking about the assignment beforehand; they should be ready to work when they sit down with their partner.
• Encourage students to post course-related questions on Piazza if you can't answer the question in person. With luck, they might get an answer before the lab session is over.
• Maintain decorum in the lab: Don't shout across the room to someone (or let students do so); gently calm down students who get too rowdy or frustrated. Have students using sound-producing software keep the volume low or off.
• Ask students to take their food and drink out into the hall. The carpets are new; they won't be replaced for another ten years. Students say they'll be careful, but they can't guarantee that their drink won't get knocked over by the student in the next row trying to squeeze by.
• Last one out, lock the door. If your lab section is the last one in the evening, be sure to pull the door closed after the last student has left. This activates the alarm.
• This hasn't happened for years, but if problems arise with a student, try to deal with it calmly yourself, and if that doesn't work, contact the lab manager or the instructor. In extreme cases, call the campus police (this has never been necessary in the past, though). If you suspect cheating or other major infractions of University rules, contact the instructor directly with as much hard evidence as you can; you should not attempt to handle academic dishonesty issues yourself.
4. Periodically during the quarter
• TAs will attend weekly meetings where we can compare notes on how things are going.
• Check your electronic mail at least once daily. Good communication is essential, with the instructor and with your students. Respond to every message in a timely way, within a day under normal circumstances. The response might be, "I'll send you a full answer later," or it might be, "This would be better to discuss in person than over e-mail." But at least send some response promptly. Official course announcements come by Email, too; they are archived at EEE.
• Check Piazza.com periodically. Feel free to answer questions on Piazza.com as an instructor. But keep these points in mind:
  • The whole class will read your response, including students in other TAs' sections. Other TAs may have slightly different grading criteria than you do, or may have resolved certain minor requirements differently than you did. So if you suspect there may be variation, say in your posting "This is for students in my sections; check with your own TA for your section's requirements." Piazza makes it easy for other TAs to add to your response and say "This applies to my section, too" or "I want students in my section to do it this way." [Remember that the class is graded separately by TA, so your students aren't compared directly with other TAs' students.]
  • Your response will be read by students who are less far along than the questioner (and those who are farther along). Be careful that your specific answer doesn't create more confusion among other students; qualify your answer so that it applies to the specific question.
  • Stop short of posting actual solution code on Piazza. It's okay to correct a single line or two and it's okay to give examples of helpful techniques (in a context that's related to but not the same as the assignment). But students shouldn't be able to copy and paste whole solutions to problems out of Piazza.
  • Unless you're pointing to something that already exists in writing (on the syllabus, in Email to the class, on assignment pages), never make any comments about the likely contents of an exam or about how scores or grades will be assigned. For the lab assignments, each TA has some leeway on grading decisions, but for exams and the course as a whole, leave those questions to the instructor; it's important that the instructional staff speak with a single voice on these issues. Don't assume that everything will be the same this quarter as it was when you TA'd this class before.
• Keep in touch with the instructor in person or electronically. Let us know how things are going, and especially if anything's going wrong. It's much easier (and more effective) to correct problems early, before they get out of hand. TAs especially are on the "front lines" dealing with students' difficulties; we need to know what those difficulties are.
• Keep track of your students, watching for those who aren't coming to section, who wait too long to start on their assignments, who aren't active partners in the lab, or who otherwise aren't doing well; if you encourage them into good work and study habits early, you may save them from a sorry fate later on. With sections as large as ours are, there are limits on how much of this you can do, but try to keep an eye out for students having trouble.
• Students with no previous experience often feel intimidated by the more experienced students, especially in the early weeks when they still need to learn the system and they see the experienced students furiously typing away. Be sure to encourage the novices to start early and ask questions of their partner; reassure them that their difficulties are normal and help foster their feeling that they can complete the work and do well.
• Very experienced students have other problems: Often they think they can wait until the last minute to complete assignments; this may be true for the first few assignments, but significant programs take substantial time, even for people who understand the concepts. Experienced students also may not be used to following rigid specs imposed by someone else. Recreational programmers can just change the requirements, but our students have to solve the hard problems rather than working around them.
• Handle various administrative and pedagogical tasks as they're assigned, usually during weekly staff meetings. These may include preparing test data, examples, or review questions.
• For more information on being a TA, talk to David Kay or the Center for Engaged Instruction office on campus (cei.uci.edu, phone extension 46188).
5. Grading lab assignments
• TAs are responsible for grading lab assignments. There's an assignment due every Friday; under normal circumstances you should grade it over the weekend and have the scores and comments recorded by Monday's lab. The feedback students receive from you is essential to their learning; if they're most of the way through the next lab before they see how they did on the previous lab, they've lost much of the benefit of the assignment. Conscientious new TAs and graders often want to give students lengthy, detailed comments. That's good, but you have to balance that with your other obligations and the need for rapid feedback. This does mean that you may not be able to give as detailed feedback as you'd like. It is far better to give less feedback sooner than to give more feedback later. If you'd like advice on how to give feedback effectively in the available time, talk with the instructor and with experienced TAs.
• Students will submit their work electronically using Checkmate; you can download students' submissions from Checkmate after the due date. Just one partner from each team submits the work; they're supposed to but both partners' names in the submission itself (but the ICS Partner Appp is also a check on this). Score them on the holistic, "coarse-grained" check-plus/check/check-minus scale described below; this should save you time and encourage students to focus on mastering concepts rather than earning points.
• plus: This is for the rare assignment that is better than perfect, that does more than what was required. To get this score students must do something extra, beyond just submitting a perfect solution. It's best not even to mention the "plus" and the "minus" score when you're describing the grading to the students; just stick with check-plus/check/check-minus.
• check-plus: This assignment is complete and correct, perhaps not perfect but with only very minor flaws.
• check: This assignment was mostly complete and mostly correct; a few parts were missing or wrong, but mainly it was okay. Code that doesn't compile doesn't count towards a check; if none of their code compiles, check-minus is the highest possible score. The assignment guidelines tell them to run their code before submitting.
• check-minus: This assignment was missing significant parts or had significant mistakes.
• minus: This is for the rare assignment that is a mere sketchy attempt; few problems are done or correct.
• [Subject to change, Fall 2016] Record the scores for your section(s) in the EEE gradebook. Each graded item has a place for a score (which must be a number) and for comments. Use the numerical equivalents below for the score; in the comments field, start with the real score ("check-plus:", "check-minus:", whatever) followed by enough feedback so the students know what they should have done to receive a higher score. The numerical equivalents are: plus=67, check-plus=56, check=45, check-minus=34, minus=23. These numbers are not scores—they're just encodings of "check," "check-plus," and so on. If a student submits nothing, leave the score empty; do not fill in a zero. Each member of a pair receives the same score and comments.
• Use comments to tell students (a) what they lost points for and (b) what to do differently next time. Feedback doesn't have to be lengthy (and can be supplemented by a message to the whole section mentioning commonly occurring problems), but it should inform the students about why they lost credit (which can often be expressed constructively as advice for how to handle similar issues in the future).
• Keep your grades up to date in the EEE gradebook. We never use the letter-grade features in the EEE gradebook, but the EEE gradebook is a way to record and distribute scores and comments reliably. Lab assignments are due on Fridays; make it a point to have them graded and recorded by Sunday night. If you let grading pile up, you will be miserable (and a much less effective grader) and your students will also be miserable (and will learn much less because of the delayed feedback).
• The assignment pages should be explicit about what's required, but if you make requirements, clarifications, or grading criteria specific to your sections, in addition to announcing them frequently in section you should send them to your sections' mailing list. Students are most frustrated when they don't know how they'll be graded, or when they think they were graded on criteria they didn't know about. Giving specific written criteria is the best approach. We will be happy to provide further guidance on this.
• Maintain high standards; don't give everybody check-pluses. If most everyone gets the highest score, they'll all think they're getting As in the course and there will be a lot of disappointment. If the assignment is explicit about what's required, it's okay to reduce scores for students who don't do those things. If a given issue wasn't covered explicitly one week, it's okay to tell the class that next time you'll deduct points for it. (An exception to this may be the first assignment or two, which are typically simple and straightforward enough that most people do complete them perfectly.)
• If there were any particularly good or particularly bad assignments, or anything else unusual that you think the instructor should see and consider, don't hesitate to point them out.
• Keep good (backed-up) records. One reason for using EEE is that it's maintained and backed up professionally. Don't let scores sit around on your machine; upload them to EEE right away.
• If a student has a question or complaint about a score, don't treat it as an adversary situation:
• Don't be forced into making a snap decision on the spot. If you can't tell immediately that you made a mistake, ask to take the paper back and review it, when nobody's breathing down your neck.
• Don't get into protracted arguments. Once you've given your reasons, if the student keeps arguing with you, say calmly but clearly something like, "I'm sorry, but that's how I graded everyone's paper. You know how to answer this kind of question in the future," and then turn to someone else.
• If you find you made a mistake or overlooked something, change the score swiftly and gracefully. There's no shame in making the mistake, only in refusing to correct it.
  
6. Exam and general grading guidelines
• Even if you have an answer key, work through all the problems yourself before starting to grade. This is not much fun, but it's the only way you can get a feel for all the points being tested and the difficulties involved in doing the problem.
• Don't be rigid when grading—it's often possible to have a solution that's correct (or nearly correct) even if it looks crazy at first glance. Try to figure out what the student meant, and where he or she went wrong; don't just mark something wrong because it doesn't match the answer key precisely. But don't supply answers that aren't there—the student must communicate to you; you can't be clairvoyant.
• Grade consistently—People who show the same level of mastery should receive the same number of points. Students always compare their graded papers with their classmates', and nothing makes them legitimately unhappier than losing three points when someone else lost two for the same thing. Take special pains to grade consistently, including the following:
• Grade vertically: A single person must grade every student's work on a particular question or assignment. (This becomes an issue on exams more than lab assignments, where there's nobody but you to grade your section's work.) Except for multiple choice questions there's no way to keep grades consistent by splitting the same work (e.g., when one person grades work from students with names in A-M and the other does N-Z). Grading vertically also means it's best for a single grader to grade everyone's Problem One, then everyone's Problem Two, and so on; it's easier to remember your criteria that way.
• Keep notes of the partial-credit criteria you use. This helps not only during the grading, but later on when students come back with questions. It's especially important for later projects and the final exam, when it's not you but the instructor who will have to field the students' grading questions.
• Recheck the papers you graded first, because your criteria probably will shift during the course of the grading.
• Shuffle the stack after each pass, so you don't grade the same student first (or last) on every question.
• Once some papers are graded, recheck the criteria by showing some of the graded work to the instructor or another TA; this is another way to avoid discrepancies between the requirements and the grading.
• Gradescope helps automate some of the above.
• Remember that there are people behind the papers you're grading. Make all your comments helpful and constructive. Explain where the student did well, or went wrong. (You may want to make a list of generally applicable comments for distribution to the class rather than laboriously copying the same comment into many individual EEE gradebook entries.) It's best not to grade in red pen; a paper bathed in red is pretty jarring. Try to avoid giving zero points, except for a completely blank answer (or a problem worth just a small number of points). One point out of ten, even for a completely wrong answer, isn't as discouraging as a zero (and having 10% instead of 0% won't significantly affect a final grade). You can't be so generous with problems that have fewer than ten points, of course.
• Having assignments and exams returned promptly is absolutely imperative—people can't study for a test without their homeworks, and they can't do projects without seeing their previous mistakes corrected. Education experts say that the value of feedback diminishes greatly with time. Your goal should be to grade, record, and return every item before the next section meeting after it was due. Recording exam scores promptly is essential because we can't compute classwide statistics until all the scores are recorded. All the due dates are on the course outline, so you can plan your schedule accordingly.
• Be professional. Nobody on the instructional staff should second-guess or criticize other staff members to the students. Students' confidence in the course and the grading process breaks down when they see that the instructional staff can't agree among themselves. If a student expresses to you a problem or concern with the course or anyone involved with it, you should pass that concern along to the instructor, keeping the student's name to yourself if you think that's necessary or appropriate.
• Be conscious of security while grading exams. Circle answers, or make lines through blank spaces, so nobody will be tempted to add correct answers after the things are passed back. (Do this as an automatic, unfailing habit.) Also, don't leave answer keys or assignments lying around unattended—this especially means you must stand by the printer as they come out, so nobody picks it up by mistake.
• Be conscious of privacy. Do not let students see their classmates' scores, Email addresses, or student ID numbers, all of which are private information. If you post or distribute a list of grades outside of EEE, use the last four digits of the students' IDs, sorted numerically; that's random. Don't keep students' scores on a computer that someone else has access to unless you have password protection.
7. At the end of the quarter
• Return your checked-out textbooks.
• We'd appreciate it if you'd send us a written commentary about your experiences as a TA, with suggestions for improving any aspect of the course. We do change the course based on these comments.
8. Electronic Resources
• Encourage your students to post course-related questions on Piazza.com. They will also have your Email addresses and the instructor's, which they can use for personal issues like having to miss class. But if they send general questions directly to you, gently encourage them to use Piazza instead, so everyone can benefit from seeing the question and answer. As you answer questions on Piazza, stay attuned to the possibility that other TAs may have given slightly different clarifications or interpretations of a problem; it's best to qualify your responses (e.g., "in my section ...") in such a case. On the other hand, for some assignments we may designate a specific person to handle all questions about the specs; this will promote consistency across sections.
• To send messages to your students, use the mailing list(s) for your lab section(s) maintained by EEE. Use this list rather than one you build yourself; the registrar automatically updates it as students add and drop your section, and EEE maintains a web-based archive of all the messages.
• To get a current roster for your section(s), including the option of a photo roster, use the roster tool on EEE.

We would be pleased to hear about other information you think would be helpful to include here.

David G. Kay, kay@uci.edu