Chapter 8
Evaluating Students

INTRODUCTION

Dr. Carol E. Baker, Director of the Office of Measurement and Evaluation of Teaching

Throughout your years as a student, you have experienced a variety of educational assessment procedures that teachers used to gather information and make decisions about the degree to which you had achieved their learning objectives. They also used this assessment data to make decisions about their teaching and to assist them in finding approaches that would help students succeed. As you now assume teaching responsibilities, you will need to think about the methods you will use to provide feedback to students and to yourself. The large number of assessment techniques available to you range from informal observations to more formally gathered data from tests, homework assignments, lab work, papers, projects, and oral presentations. In planning your course, it is important to consider the kinds of assessment you will include. You must also decide how much weight you will assign to each of the assessments that will contribute to the final grade. This section will introduce several topics related to the evaluation of student performance, an important aspect of teaching. Even if you do not have primary responsibility for teaching an entire course right now, you will probably be asked to assist with tasks related to the assessment and grading of students.

When you have primary responsibility for teaching, it is essential to communicate to your students at the beginning of the course your course objectives and the manner in which you will assign grades. It is best to do this both on the written syllabus and in your oral introduction at the first class meeting. Students are concerned about how their grades will be determined, and you can do much to relieve their anxieties about the course by making your standards clear.

Assessment results can be used both formatively (to diagnose learning needs and help plan instruction) and summatively (to evaluate student learning after one or more units of a course). Although you will undoubtedly use many different sources of information, teacher-made quizzes and tests play a large role in the evaluation of student performance. For this reason, and because test construction is probably not an area that you probably have encountered in your program of studies, some suggestions for preparing good tests will be introduced here. There are many excellent sources for you to further your study of this aspect of teaching, and several will be mentioned below.

TEST CONSTRUCTION AND SCORING

Dr. Carol E. Baker, Director of the Office of Measurement and Evaluation of Teaching

As Nitko (1996) and others emphasize, [1] the following three principles are important to follow when constructing classroom assessments:

When you construct an assessment, you should ask yourself whether each item measures something that you really think is important for students to know. Even if a test is difficult, students will consider it fair if it covers the learning objectives and content you have stressed. If students have mastered the learning objectives, they should be able to perform well on your tests. Likewise, if students are having difficulty with the learning objectives, these deficiencies should be reflected in their test scores. If items are ambiguously or poorly worded, if directions are not clear, if examples use material that is not familiar to all students, then some students who have attained your learning objectives may still answer items incorrectly. If you are aware of and make an effort to follow the three principles, you will develop tests that have the desired technical properties referred to as reliability and validity.

PLANNING YOUR TEST

The first step in test construction is to develop a test plan. This is sometimes referred to as a "table of specifications" or "blueprint" for your test. It serves to help ensure that the test will be a valid representation of your learning objectives and that the various content areas covered by the test reflect the emphasis you have placed on them in your lectures, readings and class discussions. By listing the content areas/topics and the percentage of emphasis each will have, you develop a structure for your test. You may also want to consider the level of understanding you want the students to have about each topic. For example, do you want students to be able simply to recall information, or do you want them to comprehend or apply the information to a novel situation. You may also have written some teaching and learning objectives for your course that require students to analyze, synthesize or evaluate. (For more information on instructional objectives and learning skills, see Chapter 10.) You can create a matrix by listing your content areas/topics as rows, and levels of the thinking process (such as recall, comprehend, apply, analyze, synthesize or evaluate) as columns. When you start to write or select items from other sources, you should follow your plan and the resulting sample of items will be representative of your course content and emphasis.

This planning before the actual writing of test items is very important, but it can be simplified so that it does not become a burden for you. For example, you could use "recall," "comprehension," and "higher levels" as the categories of learning. Each time you teach the course, you can make adjustments and refinements to your test plan.

Developing a test plan also helps you determine the types of items you want to write for your test. Your choice between essay items and objective items depends on the types of objectives you have developed, the level of thinking you wish to test and, practically speaking, on your class size. You may want to combine several types of items in your test. Essay items are so named because of the need for a written response. The term "objective," however, refers to the scoring procedures used rather than to the type of response given. If a student receives the same score regardless of who scores the test, the test is said to be objective.

Objective tests allow you to sample adequately from a large amount of course material, thereby increasing content validity. They are easy to score, but it does take time and skill to write effective items. Item formats include true/false, multiple-choice and matching. One common criticism is that these items often test only basic recognition, but test developers have demonstrated that effectively constructed items can also be used to test higher levels of thinking.

Essay tests include items requiring short answers and/or more extended responses. It takes less time to develop essay test items, and they give students an opportunity to demonstrate skills of synthesis and evaluation as well as organization and expression. However, the scoring of essay items is time-consuming and must follow certain guidelines in order for the test to be reliable (i.e. others reading the essay would assign the same score as you). Remember that it is important to consider your test plan in deciding on the type of test items to use.

Depending on your discipline, you may use problems as test items. They may be categorized as either objective or essay items, depending on the scoring system that is used. If the problem is simple and you are scoring only for a correct or incorrect response, it would be considered an objective test item. However, many problems involve complex solutions requiring several steps and perhaps allowing alternative approaches to their solutions. These problems are more difficult to score and they present the same reliability challenges as the essay item.

WRITING TEST ITEMS

It has been said by measurement specialists that writing good test items is a combination of science and art. There are a number of guidelines that should be followed to produce the qualities desired in your items. Even when these suggestions are followed, however, some item writers are better than others at creating clear, concise items that measure their specified learning objectives. Item writing will become easier as your teaching experience increases and you learn from the feedback provided by statistical data generated from your tests and from student feedback. There are entire chapters in measurement textbooks covering the guidelines for writing various types of objective and essay test items. A handout covering these guidelines is available from the Office of Measurement and Evaluation of Teaching (OMET). A brief summary of these guidelines is given below.

If you are planning an objective test, you may have several sources of previously written items available to you. Some textbooks supply items that have been developed for each chapter. Other instructors who have taught the course may be willing to share their items with you. In either case you will have to review carefully the items and adapt them to your specific test plan. Published items are not always well written and even those that are may not cover concepts that you consider important. Remember the three principles mentioned above when you adapt or write your items.

The most widely used type of objective item format is multiple-choice, consisting of a stem that poses the question/problem and a series of options, each representing possible answers to the stem. Suggestions for writing multiple-choice items include, but are not limited to the following:

Make sure that you give yourself enough time to write your test items. It takes time to write good test items, and it is helpful for you to review the entire test after several days have elapsed. It is a good idea to proofread and develop the answer key for your objective test before printing copies. This will allow you to catch errors that might otherwise require writing corrections on the board the day of the test or even having to drop items from your test (if, for example, you discover you have no clearly correct answer or more than one correct answer). Check the numbering of the items as well to make sure it is accurate.

When writing essay items, try to:

With essay items, it is also a good idea for you to specify the ideas that you will be looking for when scoring if you are going to use an analytic scoring method (see explanation under Scoring). This will assist you in determining the total number of possible points for each question and will enable you to make adjustments if the weighting is not what you had intended. Even if you are going to use a holistic approach to scoring, you can describe the quality of the papers that you plan to put in each scoring category.

If you are developing problem-based test items consider the following:

In writing test items, you must consider the question of test length. Consider both the length of the class and the reading level of your students. You do not want students to feel rushed and frustrated because they were not able to demonstrate their knowledge of the material in the allotted time. Some general guidelines regarding time requirements for high school student test takers (as reported by Nitko in1996) are as follows:
 
Task Approximate Time Per Item
True/False items 20-30 seconds
Multiple-choice (factual) 40-60 seconds
Multiple-choice (complex) 70-90 seconds
Matching (5 stems/6 choices) 2-4 minutes
Short-answer 2-4 minutes
Multiple-choice (with calculations) 2-5 minutes
Word problems (simple math) 5-10 minutes
Short Essays 15-20 minutes
Data analysis/graphing 15-25 minutes
Extended essays 35-50 minutes

If you are combining multiple-choice and essay items, these estimates may help you decide how many to include. One mistake often made by first-time teachers is having too many questions for the time allowed.

Once your questions are developed, make sure that you include clear directions to the students. For the objective items, specify that they should select one answer for each item and indicate the point value of each question, especially if you are weighting sections of the test differently. Students must use pencils if they are responding using the NCS optical-scanning sheets provided by OMET for machine scoring. For essay items, indicate the point value and suggested time to be spent on the item. For problems, indicate whether partial credit will be given or only the correct solution will be awarded credit.

If you are teaching a large class with close seating arrangements and are giving an objective test, you may want to consider administering several versions of your test to decrease the opportunities for cheating. Multiple versions have become quite easy to develop with the availability of word-processing software. You simply create versions of your test with different arrangements of the items. It is not even necessary to change the position of the correct response for each item. You can also then print these versions on different colors of paper so that it is obvious to students that they have a different test than the person sitting beside them. (Some instructors simply use different colors of paper without changing the order of test items, but it is best to do both.)

TEST ADMINISTRATION

On the day of a test, it is a good idea to take extra pencils with you if you are using the NCS answer sheets for your objective items. Many departments have pencils available to you for this purpose. If your class is large, try to seat students in every other seat and have every row the same so students are seated directly behind one another. You may also want to talk to faculty in your department who have taught large classes to see how they have handled the potential problem of students taking tests for others. Many faculty members announce that students must present their IDs at the completion of the test when they submit the tests and answer sheets. Announcing this in class ahead of time and even including it on your syllabus will deter students from asking others to take the test for them. Even if your class is small and/or you are giving an essay test, taking precautions against cheating and making clear to students that this will not be tolerated will create a fairer situation for all students. Cheating on an exam is unethical and a violation of the University’s policy on Academic Integrity (see Chapter 6).

Some instructors may wish to post the scores for tests outside their offices or outside of the classroom. You need to be aware that posting scores by name or by social security number is not allowed under federal law unless the student has given you permission. If you have objective tests and use the NCS answer sheets, there is a method that OMET developed to facilitate your posting of scores. It requires students to give permission for scores to be posted by social security number by filling in a specific circle on the answer sheet. A list is then generated only for those students. OMET can give you more information about this. Alternatively, some faculty use a unique number that they assign to each student.

TEST SCORING

If you have given an objective test, you may have your test electronically scored by the Office of Measurement and Evaluation of Teaching (OMET). A description of the service is available at OMET. Answer sheets are provided at no cost to you and can be obtained from OMET any time before your test. (Many departments maintain a supply of OMET answer sheets as well.) After your test, you submit to OMET the student answer sheets and a key that you have recorded on an answer sheet. Multiple versions of the test are submitted separately with a key for each version, but you can complete a form indicating the item order if you would like OMET to combine all versions in the printout described below. You will be asked to complete a cover sheet with identifying information about your course and the length of your test. Tests submitted by 5 p.m. are available for you to pick up after 10 a.m. the next working day.

In addition to the scores printed on each answer sheet, you will receive an item analysis printout that summarizes the test scores for the class, lists individual scores for your record book, and also provides you with valuable information about each of your test items. By examining the statistics related to item difficulty and item discrimination, you can identify problems students are having with items, items that need to be revised before you use them again, and items that worked in terms of discriminating between those students who knew the material well and those who didn’t. It also can include an optional list of the items that each student incorrectly answered, which will help you answer questions that students may have about their performance. A handout explaining the statistics provided on the item analysis printout is available at the office, and the OMET staff can answer any questions you may have.

OMET stores a computer record of the actual responses that students made to each item on your test and keeps that record for one year. If you are handing the answer sheets back to students, you may want to mention this to discourage students from trying to cheat by changing answers or filling in items they had left blank while taking the test. Attempts to cheat in this way have never been successful because of the computer record.

As you can see, scoring of objective tests is quite simple. Scoring of essay tests and problems takes considerably more time and you have to worry about being consistent across student responses. This is why essay items should be limited to those learning objectives that require the essay format. Once you have used an essay item, it is your professional responsibility to score the responses properly and to provide useful feedback to students.

There are two methods of scoring essay items, the analytic approach and the holistic approach. To use the analytic method, you develop a list of the major elements you believe students should include in the ideal answer. It was suggested above that this be done before the test is printed so that adjustments can be made if necessary. If you are assisting a faculty member in a course and have been given responsibility for scoring essay items either you or the faculty member has developed, you will need to meet with him/her to discuss the points to be assigned to the responses.

You may find that the analytic approach is easier to employ when the essay item requires a restricted or short answer. You can decide to give partial credit for each element in your list or simply to give full or no points for each item. This may be difficult to do if you are teaching this material for the first time since you will not be aware of the types of answers students will give. You may want to read a sample of the papers to check on your scoring scheme and make necessary adjustments. If you keep track of how students perform on each element in your list, you can identify the points that are weak and need to be reinforced in your teaching. If you provide the students with the elements of the ideal answer, they can gain information about their strengths and weaknesses.

The holistic approach to scoring essay items involves your reading an entire response and assigning it to a category identified by a score or grade. You can develop a description of the type of response that would illustrate each category before you start, and then try out this draft version using several actual papers. After reading and categorizing all of the papers, it is a good idea to re-examine the papers within a category to see if they are similar enough in quality to receive the same points or grade. It may be faster to read essays holistically and provide only an overall score or grade, but students do not receive much feedback about their strengths and weaknesses. Some instructors who use holistic scoring also write brief comments on each paper to point out one or two strengths and/or weaknesses so students will have a better idea of why their response received the score it did. Again, if you are assisting a faculty member who is responsible for the course, you might check with him/her after both of you have read several papers to ensure that your scoring is consistent.

Regardless of whether you decide to use an analytic or holistic method of scoring, there are several guidelines to consider with respect to the scoring of essay tests:

It is obvious that you will need to spend a great deal of time and effort to do a thorough job in scoring essay items. You may want to start out by using short-answer essay items and as you feel more comfortable with the process, refining your items to require a more extended answer. Use essay items judiciously; consider whether you can test the learning objective you have set with complex multiple choice items instead. If you decide that essay items are the best way to test student understanding of your learning objectives, give yourself enough time for the scoring of the test so that your information about students will be as reliable and valid as possible.

MATH AND SCIENCE ASSIGNMENTS

Math and science classes pose special challenges for teaching assistants. Although the general principles outlined in Dr. Carol Baker’s essay on grading apply to almost any assignment, grading problem-solving exercises (such as math or logic problems, computer language assignments, etc.) and lab reports often require that the instructor assign partial credit for process. In both cases, the student may understand how to solve the problem but may make simple mistake that lead to an incorrect answer. There are no strict rules for assigning partial credit except that graders should have clearly established criteria that has been shared with students and is applied equitably. Teaching assistants in math and science courses should provide students with a written statement of their grading criteria before the first assignment or lab is due.

In the following sections, we provide some general advice for grading lab reports, problem-solving exercises or numerical assignments, and we offer some advice on preventing cheating. Teaching assistants are encouraged to speak with their supervising instructor and their peers about the specific standards required in their discipline.

GRADING LAB REPORTS

At first glance, lab reports seem easy to grade. Since lab reports follow specific guidelines, it is not difficult to judge whether the basic requirements have been completed. If the student has finished the experiment and has submitted an accurate lab report, the student has met the fundamental requirements of a lab. However, not every experiment works out and not every lab report equally distills the larger themes from the experiment. When grading lab reports, teaching assistants should ask the following: The real key to grading a lab report is to establish standards in advance (preferably in a written lab syllabus) and to let students know how their work will be graded. Students need to know how much weight has been assigned to performing the actual experiment, to getting a correct result, to exploring alternative approaches and explanations, and to recognizing the theoretical implications of the experiment (regardless of the success of the lab)? Although lab instructors apply their own standards, keep in mind that the overall objective is for students to experience the scientific method first-hand and to apply the theoretical models that they have learned in the lecture class.

While traditionally lab grades are based on two criteria, completion of the experiment and a written lab report, it is possible to grade students on other observed aspects of the lab. Teaching assistants may consider student’s preparation, their ability to perform the lab techniques, their understanding of the procedures, and their observance of safety standards. [2] When using observation to access grades, it is especially important that you establish clear guidelines and keep accurate notes on students’ performance.

GRADING NUMERICAL PROBLEM-SOLVING

Numerical problem solving, everything from an engineering exam to a math quiz, poses special considerations for teaching assistants. Partial credit is often given to students who follow the correct procedures even if they ultimately get an incorrect answer. Once again, establishing clear guidelines and making your grading criteria available to your students avoids justified frustration on the part of your students and can make grading easier. The following advice may be helpful: Take the Exam. Taking the exam has two advantages. First, if you can not solve a problem, it may indicate that the problem is poorly worded or is not appropriate for the class. Keep in mind that questions posed on an exam should reflect the teaching objectives of the course. (For more on objectives, see Chapter 10.) Second, taking the exam will help you to establish grading criteria. Are the formulas employed simple? If so, how much credit should you assign to a student who identifies the right formula or procedure, but does not know how to use it? Is there only one answer? Does the math (or other task) require multiple steps creating natural divisions to assign grades to?

One Standard. Even before the exam is distributed, the instructor and the teaching assistant (or teaching assistants) should meet and discuss the criteria that will be used for grading. All graders should use the same guidelines so that students are graded fairly.

Patterns. As a teaching assistant you may not have written the exam and the grading criteria may not be your own. However, watch out for patterns that may indicate that an exam problem was poorly written. If all of the students in the class are making the same mistakes, bring the situation to the attention of your supervising faculty member. If a disproportionate number of students have come up with the same wrong answer, it is often a sign that the question was poorly constructed or misleading.

Alternative Solutions. Your grading criteria will probably be based on how you yourself solve the problem. Often the procedures that you, other teaching assistants and the professor work out are the most efficient and even most appropriate means of answering the question. However, there may other acceptable answers. In advance, determine how alternative correct answers will be handled and be alert to detect creative solutions so that credit can be assigned fairly.

Take Breaks. Grading 200 problems in a night, or even a week, can be difficult. If you are grading numerical answers, grading fatigue can lead to serious mistakes. Take breaks. Only grade papers when you are alert.

WRITTEN ASSIGNMENTS

DESIGNING WRITTEN ASSIGNMENTS

There are no hard and fast rules for designing a written assignment. A writing assignment can be as tightly structured as a lab report, where specific rules guide the students in formatting and presenting information, or as loosely structured as a creative writing assignment where a poem or graphic essay might be considered acceptable. Many written assignments, however, share the same objectives. Papers demonstrate students’ knowledge of the subject matter, their capacity to evaluate information, and their ability to craft an argument creatively and logically. In other words, papers often test the depth of a student’s understanding of the subject matter.

Be creative when writing essay or paper questions, but be realistic. Consider your teaching objectives and what your students can reasonably be expected to know or to figure out. Keep the language clear, precise and simple, but include enough information that the student is given a basic outline of your expectations. Avoid using phrases that could be misinterpreted and avoid topics that cannot be answered within the page length and time you have set. For more information on developing paper topics, see Dr. Carol Baker’s discussion of essay questions in the previous section.

GRADING PAPERS

No matter how well a question is worded and how many guidelines are offered, no two papers are ever alike. Two very good writers may marshal the same facts and come to the same conclusions, but how they present their argument—the weight given to the evidence and the stress placed upon the conclusions—are bound to be different. Grading written assignments, then, is an inherently difficult chore. Two very different papers may both be A papers. Jim, a University of Pittsburgh teaching assistant, described his frustration: I read all the papers before I actually begin to grade them and sort them into piles. All the As here, the Bs there, the Cs . . . Usually after reading thirty or forty out of sixty papers, I have a pretty good idea what is a good paper and what is a bad paper. But there is always one quirky paper that spoils my system. It is too short and needs to be edited, but it makes a point, answers the question and does so in a way that no one else in the class has thought of. Now which pile do I put that paper in? Most of us feel some discomfort with our own writing so perhaps it is not surprising that we are reluctant to critique and criticize others. [4] Yet when taken seriously, grading writing assignments is an important part of teaching. Since few of us are born great writers, the feedback we receive on our written work teaches us how to organize, present and communicate our ideas. And good writing is not "just an academic concern;" it is a practical skill that will help your students achieve success in almost any field that they explore.

Each discipline has its own style of writing and, as a result, its own standards by which papers are graded. If this is your first time grading, you should sit down with your teaching mentor or a fellow teaching assistant and discuss writing in your discipline and what is typically expected of undergraduates in your department. Bring a few papers along so that you can both read them and then discuss specific issues that the papers raise.

In the following section, we provide a guideline for grading papers. This checklist walks you through the process of grading papers. In addition, we have provided two online handouts. The first, "Grading Standards," offers criteria for assigning letter grades to papers. The second handout, "Paper Evaluation," is an evaluation sheet that you might use to insure that you are giving your students thorough feedback. Although your grading standards and assessment tools should be fitted to the course, your teaching objectives and the department you are teaching in, these may serve as starting places for thinking about what is an appropriate grade.

GUIDELINES FOR GRADING WRITTEN ASSIGNMENTS

Establish Clear Criteria for Your Students. If you do not provide your students with a clear guide to your grading policies, few of them are going to meet your expectations. Your syllabus should contain a detailed written description of the assignments or you should provide your students with a handout stating your policies. Even if you are not the one who designed the syllabus and the assignments, if you are the one who is grading the papers you should let your students know what your expectations and guidelines are. For more information on grading policies, see Dr. Carol Baker’s comments on grading essay exams in the previous section.

Graders Unite. If more than one person is grading papers, you need to establish a consistent standard for your grading. One simple way to do this is for each of you to grade five papers and then to trade papers. Would you have given the same grades? If not, why? As you discuss the differences in your grades, outline a rough standard for the rest of your grading. When you are done, you should each have a set of matching criteria for assigning grades. If as you are grading, you come across a paper that does not seem to fit the criteria you have established, assign a grade and then pass it to your fellow grader for a quick review.

Some instructors believe that when two teaching assistants are grading papers in a class, they should grade each other’s recitation. This would seem to establish a greater level of objectivity since you probably do not know the students in your co-teaching assistant’s class. But the approach also has some significant disadvantages. It is seldom that two teaching assistants present exactly the same material. If the material you presented in your recitation is different from the material your coworker presented, you may find it difficult to grade the other teaching assistant’s recitations. In addition, your students have adapted their writing (as good writers should) to their audience. The verbal feedback you give in class helps to build students’ expectations of how their papers will be graded. Switching graders can put your students at an unfair disadvantage.

Read All the Papers First. Before you start to grade, read all (or at least most) of the papers through once. Reading the papers will help you to establish the range of responses and to distinguish the best papers from the worst. Many graders, like Jim, even sort the papers into rough representative piles as they read through them. Then they grab the B paper pile and assign grades with respect to the other B papers.

Be Objective. Needless to say, you should not allow your personal feelings about a student to influence your grading. What is often more difficult, but less obvious, is that you should not let your personal feelings about the position taken in an essay or paper influence your grading. If you hate capital punishment but your student just wrote a clear, well-argued essay advocating the electric chair, you should give it a good grade. When grading, you should focus on the quality of the argument not the specific values the student advocates. Thus, you are looking at how the student addresses the reader, explains his or her position, establishes the facts and draws conclusions.

Be Consistent. If one student lost three points for not mentioning new research into the optic nerve, every student who missed that argument should lose three points. Sometimes, after forty or so papers, keeping track of these little deductions can be difficult. Many graders keep a "diary" in which they make notes about grading decisions so that they can apply fair and consistent standards across a large number of essays especially when those essays are graded in three or four different sittings.

Provide Clear and Concise Written Comments. There is nothing more frustrating for a student than to get a paper back with two or three scribbled and indecipherable comments in the margins. The grade then seems arbitrary and the conscientious student is left with very little advice on how to improve his or her writing for the next assignment. Clear, concise comments that offer advice on how to remedy mistakes are the most useful to students.

Place short notes on grammar or style in the margins near the text you are referring to, but save longer notes on the presentation, organization and argumentation for the end of the paper. Until you finish reading the paper, you are not going to be able to make comprehensive remarks. Many educators believe that "holistic" comments on errors that suggest illogical or incomplete thinking, rather than picky corrections of minor mistakes, are more useful for developing students’ writing skills. If you want to comment on a specific section of the text, place a symbol in the margin and then reference it in your closing comments.

Provide Corrective Comments. It is not enough to tell your students that they made a mistake; you also need to explain to them how to make their writing more effective in the future. Perhaps one of your students has demonstrated a clear knowledge of the subject you are teaching, but in his papers he consistently makes arguments without drawing conclusions. You might do the following:
 

Explain the problem to the student. "Joe, you seem to know the material, but you have not drawn any conclusions from the evidence."

Select an example of the problem and explain why it is a problem. "This was a problem throughout the paper, but let us look at one example. On page three you provide a number of quotes that demonstrate that Freud had a low opinion of women. Good, but what is the effect of Freud’s bias? Does it mean that we should take Freud’s theories less seriously or can we trust that modern psychology has routed out these biases?"

Offer a solution. "Joe, the paper is weak unless you draw some conclusions from your analysis. You might, for example, explain how Freud’s bias taints the foundations of his theory. I might conclude the section you wrote by saying: ‘Freud’s bias can not be simply dismissed as a product of early 20th century chauvinism since it fundamentally influences the psychology that is practiced today. Freud’s entire theory of infant development, a cornerstone of this larger work, is . . .’" Avoid rewriting the entire passage; instead, offer students some direction on how they might rewrite it themselves.
 

Obviously, time will probably not permit your making such extensive comments about every problem that a student faces. Select the most pervasive problem, the one that you feel the student needs to work on the most. Learning to write is long-term project best accomplished by tackling one problem at a time.

Consider the Four Factors of a Good Essay. First, the essay needs to make clear its purpose. Addressing its audience, it should frame the argument that is being made. Second, the essay should marshal evidence and present that evidence with arguments in a reasoned and logical manner. Third, the arguments should be organized in a persuasive way that builds toward the conclusions. Finally, it must be readable. Let your students know that the use of good grammar, correct spelling and a precise vocabulary make it easier for the reader to understand the argument that they are making.

How these four factors (purpose, reasoning and content, organization and presentation) determine the grade is up to you to decide. Not every instructor places the same weight on spelling as on reasoning. However, in most cases the grade you assign will be determined by looking at these four categories (see the "Grading Standards" handout). Communicate this clearly to your students in your comments. If they have organized the paper well but have failed to draw logical conclusions from the evidence, compliment their organization and then show them how they might have used their evidence more effectively.

Provide Positive Feedback. Your goal in grading papers is to encourage students to perform better on the next assignment. Offering only negative comments merely discourages students. Be sure to comment when the student has made an impressive argument or has demonstrated a clear understanding of the problem being addressed.

Allow Yourself Plenty of Time. Grading papers takes time. After your preliminary reading, allow at least twenty minutes for each three to four page paper.

Use a Grading Sheet or Handout. Often when grading a stack of papers, you will find yourself writing the same comments over and over. Telling each and every student to indent quotations is painstaking work with little reward; in fact, such comments often distract you from the longer and more significant comments that you may want to write. One solution is to make a list of common problems that you found when grading a set of papers and to prepare a handout to distribute when you return the papers. Another possibility is to create a grading checklist. A grading checklist is a form attached to each paper that you grade with typical comments listed and mistakes checked off. (A grading checklist is not, however, a replacement for in-depth comments. You should use it only to mark consistent grammatical and organizational errors, and you should review it in class so that your students are clear on what the pre-fashioned comments mean.) We have provided a typical grading checklist you might use or revise.

ACADEMIC INTEGRITY

Each discipline also has unique concerns when it comes to students’ cheating. In a literature class, plagiarism may be the most common form of academic integrity violation. In math and science, where answer sheets are easily altered, cheating can occur after the exam or assignment has been graded. Teaching assistants in math, engineering and science classes have developed their own unique safeguards against academic dishonesty. (For a fuller discussion of preventing cheating during an exam, see Dr. Carol Baker’s essay in the previous section.)

You can reduce the occurrence of academic integrity violations and catch offenders by doing the following:

In Chapter 6, we include a detailed discussion of the University of Pittsburgh’s Academic Integrity Policy. Cheating is an ethical violation and teaching assistants have an obligation to report instances of cheating to their supervising instructor or, if they are the primary instructor, to confront the student. If you believe a student has cheated, carefully document the violation. (For example, if the cheating takes during an exam while another teaching assistant or the instructor is present, ask your colleague to verify the academic violation.) Once you have documented the violation, speak with the student or students involved. If an instructor and a student can reach a mutually agreed solution, it should be put in writing and submitted to the appropriate dean’s office. If, however, an agreement on what has happened and how it should be remedied can not be reached, then the violation should be reported to the school’s academic integrity hearing officer. For more information on the adjudication of violations of academic integrity at the University of Pittsburgh, see Chapter 6 or the section on plagiarism below.

PLAGIARISM

POLICY

The University of Pittsburgh considers plagiarism a violation of its code of academic integrity. In "Student Obligations and Adjudication" (University Policy 02-03-03), the University states that "students have an obligation to exhibit honesty, and to respect the ethical standard of his [sic] chosen profession in carrying out his [sic] academic assignments." In the case of plagiarism, the policy goes on to say that a student has "violated this obligation" if the student: In addition, the policy states that a student has violated his or her obligation to the University when he or she has given or received assistance on an assignment where the instructor has expressly prohibited that assistance.

The policy establishes strict guidelines for how an alleged violation should be adjudicated. It states that: "No student should be subject to an adverse finding that he [sic] committed an offense related to academic integrity, and no sanction should be imposed relating thereto, except in accordance with procedures appropriate for disposition of the particular matter involved." These procedures are for both the student and faculty member’s protection. They are described below.

Advise the student that there is reason to believe that the he or she has committed an offense related to academic integrity. Once the student has been informed of the charge, the student must be afforded at least an informal opportunity to respond. If the teaching assistant and the student can come to a specific and mutual resolution, they should put it in writing and submit it to the dean’s office. The matter is then considered closed.

If, however, a resolution can not be agreed upon, the instructor (faculty or teaching assistant) must file a written statement of the charges with the designated academic integrity hearing officer (and, if it is the end of the term, issue a G grade for the student). Once the statement is filed, a hearing will be convened to evaluate the case.

It should be stressed that this procedure must be followed to protect both the student and the teaching assistant (or faculty member). University policy holds that failure to follow these procedures is grounds for dismissal. In addition, some universities have concluded that in a public institution of higher learning, an individual faculty member does not have the legal right to punish a student. [6] Although you may give a student a failing grade for not meeting the academic standards of the class, attempting to punish a student without following the University’s guidelines may deny that student his or her 14th Amendment Constitutional rights to due process and expose the instructor to a legal liability.

Despite these strict guidelines, plagiarism is fairly common in first-year students’ papers. In many cases, it is not that the student wants to cheat, but rather that the student does not know that what he or she is doing is cheating. Sometimes students just forget that they need to cite material drawn from a published work; other times, given an assignment that is supposed to be based on one or two texts, students paraphrase the original source because they cannot think of a better way of presenting the material. These are serious errors that need to be corrected, but instructors should use some discretion in determining the right action to take with first-time offenders whose errors seem to stem from ignorance rather than intent to defraud. In accordance with University policy, you should meet with the student, outline the problem and work out a solution with the student. If the student agrees, having the student rewrite the paper can help the student to learn the rules that govern academic writing at a university.

Other cases are not so ambiguous. Papers that have been lifted wholesale from a text, have been copied from a fellow student, or have been produced by a paper mill are clear violations of the University’s policy. Handle such cases in person and as quickly as possible as outlined in the policy.

PREVENTING PLAGIARISM

Ignorance and panic are, more often than not, the two major causes of plagiarism. As an instructor, you can play a role in preventing both these causes.

In your syllabus, include a clear statement explaining what plagiarism is and that it is a violation of University of Pittsburgh policy. Review this statement when you give your overview of the syllabus and the assignments during the first class and stress the seriousness of this offense. In classes where many of the students are writing university papers for the first time, you may want to offer more extensive guidelines. These guidelines should specifically explain what plagiarism is and should tell students how to properly cite others’ works. In this handbook, we have provided two handouts ("Avoiding Plagiarism" and "Ending Plagiarism") that you may use as models for your own anti-plagiarism handout.

Plagiarism often occurs when a student has waited until the last minute to start the assignment and, facing a deadline, looks for a quick way to finish the paper. You can reduce the likelihood of this happening by offering your students as much assistance with the writing process as possible. Set deadlines for each stage in the paper writing process, schedule special "writing workshop" office hours, and offer to look at outlines and read drafts.

Finally, reconsider your assignments. Assigning the same paper year after year or asking students to write "research" papers using only one or two sources does nothing to discourage plagiarism. New, creative assignments not only make it more difficult for students to plagiarize, but also inspire interest in the work so that students are less tempted to plagiarize.
 

PREVENTING ONLINE PLAGIARISM
Student Plagiarism in an Online World (Prism Online article by Julie Ryan)
http://www.asee.org/prism/december/html/student_plagiarism_in_an_onlin.htm

"Plagiarism.org" for educators concerned with the growing problem of Internet plagiarism
http://www.plagiarism.org/

THE LOGIC OF GRADES

Dr. Carol E. Baker, Director of the Office of Measurement and Evaluation of Teaching

As you plan your course and develop your syllabus, you should also consider your grading criteria. You will be using various types of assessment during the term to help determine the students’ achievement of your learning objectives. At the end of the term, however, you will be required to use a system of letter grades (see a description of the University of Pittsburgh system in the next section). There are several important decisions that you need to make regarding grading so that you can inform students of your grading policies on the syllabus and in class.

Grading is a difficult aspect of teaching for many instructors. Making judgments about others is often uncomfortable, especially since, as teachers, we want to be helpful and motivate students to do their best. There are different philosophies about grading, and these result in different methods of evaluating students. You may want to discuss grading practices with faculty in your department as you develop your own point of view. Students will ask you about this and you need to have a coherent and logically consistent opinion.

There are two basic frameworks to consider. One uses a relative scale and the other an absolute scale. Grading that uses a relative scale is sometimes referred to as norm-referenced: grades are assigned on the basis of how a student’s performance compares with others’ in the class. Those preferring this approach base their arguments on the value of knowing a student’s standing compared with others and the reality of competition in life.

Grading that uses an absolute scale is sometimes referred to as task-referenced grading: grades are assigned by comparing each student’s performance to a defined set of objectives to be learned or tasks to be done. With an absolute scale, all students in your class could receive grades of A if they all meet the absolute standards you set. The argument for using this approach is that it is of value to know what each student has learned, and each student’s grade should be independent of that of other students.

Regardless of which framework you select, the next step in determining your grading system is to decide what will count towards the student’s grade. Several types of assessment have been discussed above; you must also decide whether other aspects of the course, like attendance and class participation, will contribute to the course grade. Some may be used only formatively to provide feedback to students; others may be used summatively and count towards the final grade in the course. Deciding this as you plan the course is essential so this information can be included on the syllabus. Changing your grading system during the term or not indicating your grading system is not fair to students, and they will be vocal with their frustration.

After you have determined what is to count towards the grade, you must decide how much weight you will assign to each component. Think about the emphasis of each component in your teaching and learning objectives. Try not to develop too complicated a weighting scheme; what sounds reasonable at the beginning of the course can become a real problem as you are faced with grade deadlines at the end of final exam period.

At the end of the term, after you have recorded the points for the various assessment tasks you have decided to include in your final grades, the procedures you will follow to determine grades will depend on whether you have chosen to use a relative or absolute grading scale.

ASSIGNING RELATIVE-SCALE GRADES

With a relative grading scale, you first develop a composite score for each student and rank the students with respect to this composite score. However, the problem with simply multiplying weights by scores and adding them is that the components with the largest weights should contribute the most to the final ranking. The complication is that the rank of a composite score is influenced by the standard deviation of the components. Those that have more variability associated with them will contribute more to the ranking. So if you have given an assignment that produces similar scores for all students, the contribution of that component to the overall ranking will be smaller than one in which the scores have a large range.

The solution to this problem is to form "standard scores" for each component before they are weighted and added together. This involves a formula that can be programmed into a spreadsheet or grade book program. It generates composite scores that you can then rank and know that the weights that you have specified will have the desired influence. If you decide to use this method, staff at OMET can provide you with the formula.

Once the rank ordering is done, you must then decide on the percentages of each letter grade that you wish to give. Contrary to what many believe, there are no rules for selecting these percentages! Traditionally, "grading on the curve" refers to dividing the range of the normal curve into five equal-length intervals, producing 3.6% A’s and F’s, 23.8% B’s and D’s, and 45.2% C’s. However, it is very hard to justify that the achievement of the students in your class is normally distributed, and therefore true "grading on the curve" is not recommended. Instead, you should select the percentage of each grade. You may want to ask others who have used relative-scale grades what they have done to determine the percentages.

There is a second relative-scale grading method that uses multiples of the standard deviation of the composite scores to determine limits for each grade. For instance, you might decide that the mean + ½ standard deviation will determine the lower limit for a B. If you have not taught the class before, it is difficult to know how students may do, which makes determining the intervals difficult. Remember that they need to be determined before the class starts so students can be informed.

ASSIGNING ABSOLUTE-SCALE GRADES

Two methods can be used to assign grades that are task-referenced. One uses fixed percentages and the other uses the total points that you have assigned. To apply the fixed percentage method, the correct percentage is calculated for each assessment entering into the final grade. Then these percentages are multiplied by the weights you have determined for each component, these products are added together, and this sum is divided by the sum of the weights. This gives you a weighted percentage score for each student. Deciding the relationship between these percentage scores and letter grades is arbitrary and varies from instructor to instructor and class to class. What others in your department use, your own experience with the kinds of students you are teaching and the difficulty of the material will help you in determining the percentage intervals for each letter grade. Some use a 90% - 100%=A, 80 - 89%=B etc. definition, but there are a number of different grading schemes employed across the University.

The total points method requires that you decide in advance the points that you want to assign to each assessment task that is going to contribute to the final grade. The number of points assigned reflects the weight that you give to each component; if you want your midterm and final exam to be equally weighted, you would assign the same number of points to each. By summing the maximum points for the components, you will have the total possible. If you decide you would like to have a total of 150 points, then your components would be weighted so that they will sum to 150. If you decide that homework is to contribute 10% to the grade, you would assign 15 points to homework. This method also requires you to set the boundaries for assigning letter grades, but here you are using the total points instead of percentages. Perhaps an A would be assigned to the range from 135 to 150 points.

The difficulty of the total points method is that you must decide this ahead of time and then fit your assessments to the points. If you decided to assign 50 points to the midterm, but find you need only 40 items to adequately cover your learning objectives, each item must then be worth 1.25 points. This can create some awkward scoring and is difficult to explain to students.

As you can see, assigning grades is a complex task that is based on different philosophies of grading and different methods within each framework. Some instructors make their task even more complicated by letting students drop the lowest score on the tests they have taken, or accumulate extra credit by answering additional questions or doing optional assignments. At the beginning of your teaching career, keep your grading method as simple as possible and make sure that it is clearly explained to students on the syllabus. It will never be simple, but as you gain experience, you will become more comfortable with this important part of teaching.

Although you are required to make a number of decisions regarding grading, you can receive some assistance with the actual calculations through computer spreadsheet programs or one of the grade book computer programs that are available. These grade book programs provide you with a spreadsheet already designed for recording and calculating grades. Some allow you to select from a variety of grading methods. Not all grade book programs are of high quality, however, and you need to make sure you understand how they handle missing data and the rounding of numbers in calculations. If you do use a spreadsheet or develop a program yourself, you should be aware that you can copy the objective test results produced by OMET using the University of Pittsburgh’s mainframe computer. This saves you from having to type in student names and scores, which can be timesaving if your class is large.

THE UNIVERSITY GRADING SYSTEM

GRADING OPTIONS

Depending on the policy of the individual academic units, faculty may decide to offer the following grade options to students who take their courses:
 
GRADING OPTION DESCRIPTION
LG Unless otherwise specified by the instructor or the department, the letter grade option (A, B, C, D, F) is the default option.
S/N In addition to the standard letter-grade system, the College of Arts and Sciences (and some other schools) offers a Satisfactory/Audit (S/N) option. This system was designed to encourage students to explore new and potentially difficult subjects without the risk of failure. Under this option, a student who does satisfactory work (C or better) in a course receives the grade of S; if the student’s work is unsatisfactory (C- or lower), he or she receives a grade of N.
LG and S/N If this option is available, students may choose to the take the course either for a letter grade or a satisfactory/audit grade.

Students must choose a grade option by the fourth week of the term or the second week of a summer session. It is their responsibility to (1) obtain Grade Option/Audit forms (available in the appropriate dean’s office), (2) fill them out accurately and completely, and (3) submit them to the dean’s office before the deadline. Late, incomplete, or inaccurate forms will not be processed. Once the S/N option has been requested, the decision cannot be changed. Since this form represents a contract between the student and the University, the grade issued must conform to the agreed-upon option.

Evaluation of a student’s ability and achievement in a course is not eliminated by the Satisfactory/Audit system. Recitations, tests, and papers may be required and assessed by the instructor, who should convey to the student his or her judgment of the worth of the student’s work.

Students may also choose to take any course on an audit basis and receive an N on the transcript rather than a letter grade. Students who wish to audit a course must register for the class as usual and must then process a Grade Option form in their dean’s office by the fourth week of the term or the second week of the summer session.

LETTER GRADES

The letter grade (LG) system at the University of Pittsburgh is described below. Note that there is no official system for translating percentage grades into letter grades. Thus, a 90 on an exam is not necessarily an A. For a more detailed explanation of how to calculate grades, see Dr. Carol Baker’s explanation of how to assign grades in the preceding section.
 
LETTER GRADE QUALITY POINTS UNDERGRADUATE GRADUATE PROFESSIONAL
A+ 4.00      
A 4.00 Superior Superior Exceptional
A- 3.75      
         
B+ 3.25      
B 3.00 Meritorious Adequate  
B- 2.75      
         
C+ 2.25      
C 2.00 Adequate Minimal Adequate
C- 1.75      
         
D+ 1.25      
D 1.00 Minimal    
D- 0.75      
         
F 0.00 Failure Failure Failure

ADDITIONAL GRADES

In addition to the standard letter grades, the University of Pittsburgh also offers a number of other "letter grades" designating incomplete work, withdrawal or that the student has chosen the Satisfactory/Audit grading option.
 
GRADE DESCRIPTION
G The G grade is to be given only when students who have been attending a course and have been making regular progress are prevented by circumstances beyond their control from completing the course after it is too late to withdraw. There should be some specific, clearly defined work (e.g., a final examination or term paper) to be done to complete the course. The instructor is under no compulsion to grant a G grade for substantially incomplete work. If a significant portion of the course work is incomplete and the student has not officially withdrawn, the appropriate grade is F.

The G grade must be replaced by a final grade during the student’s next Fall or Spring Term in residence (or, in the case of students who are not in attendance the following term, within one year); otherwise the privilege of completing it is withdrawn and the credits are lost. Only the professor who gave the grade can change it; if he or she becomes unavailable by leaving the University, the credits may be lost.

Use discretion in granting G grades. In approximately 75% of the cases where a G grade is granted, students do make up the work. We recommend that you do not assign a G grade without speaking with the student and having the student sign a contract specifying what work needs to be completed and when the work must be submitted. Ultimately, however, it is the student’s responsibility to make arrangements with you for finishing the class.

I The I grade is a research grade. It indicates that the work for the course for which it is awarded has not been completed. It is to be awarded only to students who have been doing the regular work of the course but who need more time than the term allows to complete research on a project. In other words, the extenuating circumstances ought to arise from the nature of the course work rather than from the student’s personal difficulties (in which case a G grade is the appropriate one—see the preceding section). The student and instructor should agree on a definite program of work that the student must complete; when the work is done, the instructor will submit a final grade. The work should normally be completed not later than the following term, but in any case, it must be complete within one calendar year after the I is given.
N The N grade is awarded to students who have selected the Satisfactory/Audit option and have not achieved a C average or better. It is also assigned to students who choose to audit a class.
R When students resign from all courses through the Office of Student Accounts, they receive the grade of R in each of those courses.
S The S grade is awarded when a student has satisfactorily completed the course requirements (C or better) under the Satisfactory/Audit option.
W W grades are given when a student officially withdraws from a course. After the end of the add/drop period, and until the end of the ninth week of the term (or third week of Summer Session I or II), a student may withdraw from any course by filling out a "Monitored Withdrawal" form, available from the dean of the appropriate school. The form requires the instructor’s signature but not the instructor’s approval. After the established deadline for the term or session, a student may appeal for late withdrawal from individual courses by processing an "Appeal for Late Withdrawal" form in the Dean’s Office. Late withdrawal is granted only in exceptional and extenuating circumstances with the permission of the Dean.
Z Z grades appear on grade reports and transcripts when the instructor for a course has given an inappropriate grade: for example, an S grade when a student has not signed up for the Satisfactory/Audit Grading Option, or a W grade when a student has not officially withdrawn from a course. Students who receive Z grades should contact the instructor of the course. Students do not receive credit for courses in which Z grades are given unless the Z is replaced by a letter or satisfactory grade.
** No grade reported.

LATE GRADES

It is the instructor’s responsibility to turn grades into the Office of the University Registrar or, in some cases, a departmental authority prior to the expiration of all established deadlines.

GRADE CHANGES

An instructor wishing to change a grade must complete a grade change request form. While each school may determine a time limit for grade changes, in virtually every case, changes should be put though no later than one year after the initial grade was assessed. With the exception of I grades, all grade changes must be approved by the dean of the school in which the course is offered before they will be honored by the Office of the Registrar. Often if the grade change takes place after one semester has passed, the dean’s office requires a letter of explanation before it will approve the change.

COURSE REPEATS

Under most circumstances, students may repeat a course. The last grade received is the one that will be used in calculating credits and in computing the QPA although both classes will appear on the student’s transcript. A "Course Repeat" form must be filed in the appropriate dean’s office after the course is repeated. Students are encouraged—and sometimes required—to repeat any course in their major in which they have earned less than a C.

Course repetitions are subject to limitations. For more information, check with the register’s office or with your academic center's dean's office.

MIDTERM REPORTS

In the Fall of 1999, the College of Arts and Sciences will ask select classes to participate in a pilot program where midterm progress reports are prepared for first-year students. Midway through the semester (near the end of October) instructors and teaching assistants who are participating in the program will be asked to submit grades for first-year students. It is important that instructors who are participating in this program provide sufficient opportunities for students to be graded and receive feedback before the midterm report is issued.

The intention of the midterm progress report program is to increase retention rates for new students by providing prompt assessment and assistance to students who are struggling to make the transition from high school to college work. If this program is successful, it may be extended to additional courses in the future.

GRADE VERIFICATION REPORTS

Grade verification reports are mailed to instructors approximately one month after the term or session has ended. Instructors are required to compare this report to the grades that they submitted and to make changes if an incorrect grade has been assigned.
 
FOR MORE INFORMATION ON THE UNIVERSITY OF PITTSBURGH'S GRADING SYSTEM
 
Office of the University Registrar
http://www.pitt.edu/~srfsweb/fgtoc.htm
 
Course Lists and Exam Schedule

http://www.pitt.edu/~srfsweb/crseinfo.htm

 

PRIVACY OF STUDENT RECORDS

The University of Pittsburgh, in accordance with the Family Educational Rights and Privacy Act (FERPA) of 1974, also known as the Buckley Amendment, sets forth requirements designed to protect the privacy of students. The statute governs access to records maintained by educational institutions.

The Buckley Amendment provides for the confidentiality of student records, including grades. Each instructor must take care that student records are not revealed to anyone other than the student. For instance, if you plan to post grades or exam scores, you may not use the student’s name, social security number or University of Pittsburgh ID number as an identifier without the student’s written consent. Instead, a unique ID number should be issued confidentially to each student. Only you and the student to whom it belongs should know this number.

The practice of leaving tests, quizzes, papers or homework in a specified location for students to pick up on their own is also taboo. Graded work must be returned individually to protect the privacy of your students. If a student is unable to pick up the paper in person, arrange for the student to leave you a self-addressed envelope and mail it.

The release of information about a student is a very delicate matter—especially if it is the student’s parents who are requesting the information. If the request is made by phone, no information can be released under any circumstances; in such cases it is impossible to establish whether the person on the phone is who he or she claims to be. Furthermore, information can only be released to parents without the student’s consent if the student is under the age of 18 or if the parents can prove the student is a financial dependent. Financial dependence can be demonstrated with a certified copy of their most recent Federal Tax Return; either way, the parents should offer picture ID confirming their identity. Release of information to a third party can only be authorized by a signed, written request from either the student or the student’s parents when the parents have met the above standards. For more information on the Buckley Amendment, contact the Office of Judicial Affairs at the University of Pittsburgh.

STUDENTS IN TROUBLE

If one of your students is not attending classes or not turning in assignments and has not withdrawn, you should attempt to contact the student. If for any reason you are unable to contact the student or if you feel the student needs academic advising that you are unable to provide, you should contact the student’s advisor. In the College of Arts and Sciences, this can be accomplished by calling the Advising Hotline. In other schools, contact the appropriate dean’s office. (Telephone numbers are available in the resource guide in this handbook.)
 .
 
 
Chapter 7 . Chapter 9
..