_{1}

^{*}

Elements of correspondence (“coincidences”) between a student’s solutions to an assigned set of quantitative problems and the solutions manual for the course textbook may suggest that the stu-dent copied the work from an illicit source. Plagiarism of this kind, which occurs primarily in fields such as the natural sciences, engineering, and mathematics, is often difficult to establish. This paper derives an expression for the probability that alleged coincidences in a student’s paper could be attributable to pure chance. The analysis employs the Principle of Maximum Entropy (PME), which, mathematically, is a variational procedure requiring maximization of the Shannon-Jaynes entropy function augmented by the completeness relation for probabilities and known information in the form of expectation values. The virtue of the PME as a general method of inferential reasoning is that it generates the most objective (
*i.e*. least biased) probability distribution consistent with the given information. Numerical examination of test cases for a range of plausible conditions can yield outcomes that tend to exonerate a student who otherwise might be wrongfully judged guilty of cheating by adjudicators unfamiliar with the surprising properties of random processes.

Numerous reports over the past decade by the news media [

In brief, portable devices linked to the Internet make it easy for students to rapidly search, download, and insert into exams, homework exercises, and written papers the exact words or calculations of other people without attribution. In the matter of dishonesty in university-level courses in the natural sciences and engineering, improper attribution of credit is usually not the only concern: the very act of submitting someone else’s solution to a problem is not just plagiarism, but a more serious form of cheating. Whereas plagiarism is a kind of intellectual property theft over which academics may argue in regard to definition and importance [

Nevertheless, to many students, the vastness of the Internet, coupled with the perceived inability of an instructor to verify the originality of all submitted work, provides a sense of security that plagiarism or cheating will go undetected.

This paper is concerned with assessment of the occurrence of plagiarism in individual cases of a kind that is more likely to be prevalent in the physical sciences, engineering, and mathematics in which assignments call for numerical answers or mathematical analyses with relatively brief written discussion, in contrast to assignments in the humanities and social sciences that often take the form of long papers.

It is relevant to mention that in my own classes, which include technical courses for physics majors as well as general interest courses for students with little science background, I give both kinds of assignments. In the case where a paper is turned in that contains many lines of text plagiarized from an internet-accessible source, there is little if any need for statistical analysis. A student presented with the URL (universal resource locator) and online content that was copied into his or her paper will ultimately admit to the source, since it would be ludicrous to argue that the hundreds of corresponding words in sequence came about by pure chance. (A specious argument that is made by students is that plagiarism did not occur because the copied text was referenced in some way, although never explicitly shown as a direct quotation.)

More problematical, however, is the case where a student’s solution to an assignment suggests elements of correspondence with some other source such as the instructor’s solutions manual to a textbook. The term “elements of correspondence” is unavoidably ambiguous; it may refer to (1) brief verbal phrases occurring in both the student and textbook solutions; (2) the precise format of certain mathematical expressions such as use of a

radical sign

common in multiple places to both student and textbook solutions; (3) the occurrence in the student solution of the same misprints found in the textbook solution; (4) the occurrence of identical diagrams or plots in both the student and textbook solutions, or a variety of other possibilities. This is a matter of perception by the instructor, for which, in view of the intrinsic uncertainty, a statistical analysis and probabilistic inference may be called for.

It has been my experience over many years that few instructors, even among scientists and mathematicians, solve the correct statistical problem when faced with the perception of plagiarism of this kind. The correct problem to be solved is, in fact, a subtle one, with results that can be surprising to the instructor. The problem of assessing whether plagiarism occurs under circumstances where the evidence is suggestive but not conclusive and where the student is adamant about his or her innocence is a vital one because the consequence of a false judgment to a student can be disastrous.

To be clear on the matter of consequence, here is what may transpire at a college or university in the US when a student is accused of plagiarism. An honor panel or committee, comprising an administrator (e.g. a dean of students), faculty, and perhaps also several students, convenes to adjudicate the charge. The accusing instructor presents the case against the student; the student rebuts the accusation; the panel then decides the matter perhaps by simple majority vote.

If the student is found culpable, the penalty can take the form of (1) a permanent or time-limited notice of censure on the student’s academic record; (2) suspension from the institution for a specified period of time; or (3) permanent expulsion from the institution. The lightest penalty (1) is often accorded for first offenses, but its consequences can be far from light. A censured student is no longer in good standing and may be denied academic honors at graduation. If the censure remains on the student’s record at the time the student applies for admission to graduate programs or for financial assistance through graduate fellowships, the applications will likely be unsuccessful since, after all, a graduate school or granting agency would hardly want to foster the education of someone lacking in personal integrity. Moreover, if the censured student is a foreigner, failure to be accepted into a graduate program can result in discontinuation of a student visa, whereupon the student must leave the host country and return home in disgrace. In light of serious potential consequences to a student, the reader can appreciate how critical it is that an honor panel reaches the correct decision.

Different institutions have different protocols, but the foregoing summary is representative. Also characteristic of the procedure is that participants in the honor panels are sought broadly from all faculties of the university and need have no special training or even rudimentary familiarity with probability, statistics, and methods of inferential reasoning. Although administrators, faculty, and students of a university would probably look upon this “diversity” as a laudatory feature of the democratic process, it is also the feature most likely to lead to what in statistical language is termed Type I and Type II errors of judgment [

The illustrative case to be analyzed is drawn from physics because the fundamental nature of that subject is the most compatible with the kind of plagiarism referred to in Section 1.2. The laws and principles of physics are reproducible statements concerning the physical world accepted without contest (at least provisionally, until demonstrated otherwise by experiment) by the vast majority of physicists. These laws and principles are regarded by physicists as describing the physical world and not the mental state of the physicists who use them; they are expressed through the language of mathematics, and their applications are preferentially mathematical rather than verbal.

In applying these mathematical statements to solve a problem, a physicist or physics student may need to demonstrate that certain conditions prevail under which a particular law is applicable1, but in no case do the actual laws of physics depend on matters like politics, economics, religion, philosophy, or any other aspect of human culture involving potentially divisive interpretations and personal opinions. In other words, physics is a subject for which the corpus of knowledge is reasonably well defined, broadly accepted by practitioners, and very largely independent of cultural biases and opinions.

The point to the preceding two paragraphs is this: In contrast to questions posed in disciplines that call for lengthy discussion, there are usually only a few appropriate ways to solve a well-defined academic physics problem and only one correct answer. There exists, therefore, a possibility for a significant degree of overlap between the sparse verbal and mathematical expressions in the solutions manual and in the exam or homework paper of a physics student, especially if the student is bright and answers questions correctly and in the most efficient way. The difficult issue to be addressed by statistics is whether or not such coincidences signify plagiarism.

Consider the following representative case, which is an amalgam of situations I have observed:

A student S turns in a homework paper with solutions to

P then reasons (either subconsciously to himself or explicitly before the honor panel) in a manner like the following:

“I could accept the possibility that S’s paper contained one problem with wording and equations similar to the solutions manual. But five are too many to be attributable to pure chance. Suppose the probability of a coincidence between a student’s solution and the textbook solution of the same problem is

To members of an honor panel unfamiliar with probability and statistics, the preceding mode of thought may seem rational and convincing. But it is wrong on several accounts.

The first error, known widely in forensic statistics as the “prosecutor’s fallacy” [

where H is the hypothesis (referred to as the null hypothesis) that the student is guilty of plagiarism, and outcome O is the alleged k coincidences out of m problems. Equation (1) can be re-expressed in the form

in which a bar over a symbol signifies negation. Thus

The prosecutor’s fallacy and an analogous specious argument known as the defense attorney’s fallacy have drawn attention of news media, especially in the UK and US, because of its association with sensationalist legal cases such as the Sally Clark cot death case [

Irrespective of whether P’s calculation was done correctly or not, P was pondering the answer to the wrong question. Implicit to this representative case study is that S had not admitted to any guilt, nor had any evidence been presented to prove guilt, apart from inferences drawn by P from a single homework assignment. Thus, in a society in which the legal system puts responsibility on the part of the accuser to show cause, rather than upon the accused to prove innocence, it ought to be presumed at the outset that S did not commit plagiarism. Under that presumption, the pertinent null hypothesis, therefore, is that the coincidences observed by P occurred by chance. The correct statistical question, therefore, is not what is the probability of plagiarism given coincidences, but this:

What is the probability

In other words, in scrutinizing the alleged evidence for plagiarism, the accusing instructor and adjudicating honor panel must focus not on guilt, but on the likelihood that the observed coincidences could have occurred randomly. In the context of a criminal trial, members of a jury are ordinarily unequipped by experience and bias to think this way [

Bayes’ theorem (2), in which the null hypothesis H is now (and for the remainder of this paper) taken to be that S did not plagiarize, can in principle be used to estimate the probability of S’s innocence given the evidence submitted by P. However, this is not a satisfactory way to proceed because it entails making highly subjective decisions regarding the prior probabilities

Alternatively, one might argue that the state of ignorance about S’s guilt or innocence is best represented by setting

The primary purpose and accomplishment of this paper is to arrive at an objective probability function with which to infer whether a student may have committed plagiarism or not. As explained in the following sections, this is achieved through use of the principle of maximum entropy (PME), which furnishes a probability distribution, based only on known information, that the observed elements of correspondence between a student’s paper and the instructor’s answer book could have occurred by chance. Subjectivity enters the process only at the endpoint where the honor panel must decide how to use this probability.

The problem raised in the case study of this paper calls for a statistical evaluation of the significance of coincidences. The term “coincidence” as defined here is consistent with, although more narrowly focused than, the use of the term by Diaconis and Mosteller [

What is noteworthy, however, is Fisher’s use as early as 1924 of

Also interesting from a historical perspective is that the method of analysis employed here to assess the probability of coincidences likewise draws its inspiration from a distribution initially introduced by Fisher [

In the analysis of this section, the probability p is taken to be specified a priori. For example, in the (specious) argument of Section 2.2, the professor P instinctively adopted

It then follows that the probability that the student’s paper does not contain at least k coincidences is

Equation (4) therefore gives the probability that the student’s paper contains a number of coincidences ranging from 0 to

Since it is assumed that the students in the class act independently (an assumption that does not hold when students are permitted to collaborate on homework, which is ordinarily not the case in university science courses), the probability that no student in the class of n students has turned in a paper with at least k coincidences is given by

Therefore, the probability

Equation (6) is the sought-for probability function by which to test the significance of alleged coincidences signifying plagiarism under the circumstances outlined in previous sections.

A graphical examination of the statistical content of Equation (6) will be undertaken in a following section, but first it is necessary to consider how to assign a reasonable value to the probability p. There are several ways this might be done of which the final method based on the PME is the most objective:

Method I: p is simply specified by the course instructor based on his/her personal feelings as to what is reasonable. In the hypothetical illustration of Section 2.2 (which reflects an actual occurrence), the accusing professor was comfortable with

Method II: Calculate a value

A difficulty with Method II is that, like Method I, it is also highly subjective. The value of p is obtained from an arbitrary statistical threshold, which could have been set differently. Therefore

Method III: Use the principle of maximum entropy (PME) to determine the distribution of p. The PME yields the least biased probability distribution consistent with known information. Moreover, since the procedure yields a distribution function and not merely an arbitrary value of p, one can determine the uncertainty (e.g. variance) and other statistical moments. An explanation of the method is given in the following section.

The term “entropy” in physics (derived from the Greek root for “change”), together with “energy” (derived from the Greek root for “work”), is a seminal concept in thermodynamics and statistical mechanics. It is beyond the scope of this article to explain in detail the various meanings and applications of entropy. (See, however, Ref. [

Besides the association, known since the 19^{th} Century, of entropy with physical processes involving exchange of work and heat, a connection between entropy and the concept of information was recognized by C.E. Shannon in the late 1940’s [

Equation (7) bears a resemblance to the quantity (called “Fisher information”) adopted by Fisher to score coincidences. The two expressions for information have very different properties; the more significant quantity by far is Shannon’s H.

It is demonstrable in equilibrium statistical mechanics (ESM) that the thermodynamic entropy (ordinarily symbolized by S) is equal to the Shannon entropy H up to a universal scale factor, i.e. ^{th} Century ESM was the recognition by physicist E.T. Jaynes [

Jaynes, however, developed the method beyond ESM to show that it represented a general mathematical principle of inferential reasoning, not tied to physics, by which to derive the least biased probability distribution consistent with known information [

to ensure that the entropy (i.e. information) is invariant under a transformation of parameters. In other words, if the probability density function of a distribution is first characterized by parameters

relation for probabilities

As a matter of terminology, it is important not to confuse the principle of maximum entropy (PME) with the different, but statistically more familiar, method of maximum likelihood (MML) introduced early in the 20^{th} Century by Fisher [

The PME distribution is least biased because it depends only on the given information and not on any supplementary assumptions either explicit or implicit. For example, in the case of a sought-for probability distribution contingent on the expectation values of several independent system properties, the PME distribution will lead to zero cross-correlation of these properties. Distributions arising from variation of other functionals than the Shannon-Jaynes entropy (8) ordinarily give rise to non-vanishing cross-correlations of system properties. Clearly, if the given information did not include correlations among the system properties, then an unbiased probability distribution must not generate any.

The PME solution to a statistical problem employs only (1) the mathematical properties of probability and (2) whatever information is specified―i.e. has been observed―about the system at issue. Regarding information (1), it is assumed that the probability

If no other information than completeness is available, the solution to the PME variational problem leads to

In dealing with alleged plagiarism by a student, however, the accusing instructor P and adjudicating panel can acquire an important piece of information: the mean number

Here, then, is the formal problem to be solved:

Statement of the problem: Find the probability

(1) Completeness Relation:

(2) Observed Mean:

The functional

is the Jaynes entropy (8) (with Lebesgue metric

As previously stated, the Lebesgue metric for a given system is proportional to the prior distribution in absence of all information other than the completeness relation. In the present case, therefore, one can set

which is a binomial distribution because there are m possible outcomes (i.e. homework solutions) with only two outcome categories (coincidence or no coincidence) for each solution, and the principle of insufficient reason assigns equal probability to each outcome in the absence of information to the contrary.

Solving the set of equations

to obtain probabilities

where the Lagrange multiplier

Despite the appearance of the exponential factor, Equation (14) is not the probability function of an exponential distribution

from which it follows that

Upon substitution of Equation (16) into Equation (10) and transformation of variable

Use of Equation (17) in Equation (16) leads to the final form of the coincidence probability distribution

which upon substitution into Equation (6) provides the least biased probability function

for judging whether the occurrence of k or more coincident solutions out of m homework problems is acceptable or implausible within a class of n students for which

It is to be noted that the PME solution (18) is actually a binomial distribution

parameter

The implications of Equation (19) will next be considered.

The simplest way to get a sense of the information content of Equation (19) is to examine a number of test situations graphically. Recall that Equation (19) is the probability that at least one student in a class of n students will turn in a homework paper of m solutions with k or more random coincidences with the instructor’s answer book. The mean number of such chance occurrences for this assignment in a population comprising all the students in the class is

at the 3^{rd} or 4^{th }year level at a US liberal arts college might well have about 10 students. An outcome of 5 coincidences or more exceeds the 5% threshold in a class size of 30 or more students. Thirty students is not unusual for an introductory general physics course at 1^{st} or 2^{nd} year level at a US liberal arts college. The corresponding class size at a US university could be well beyond 30, and easily exceed 100.

If a set of assigned problems is particularly easy, as may be the case if the instructor puts little time into the task and uses the same questions from year to year, then the mean number of coincidences may be higher than 1. From

An alternative way to examine the implications of Equation (19) is illustrated in

To return to the hypothetically constructed (but accurately representative) scenario of Section 2.2, an instructor, faced with a student paper containing 5 solutions out of 10 problems allegedly coincident with the answer book, judged the probability of the event to be implausibly low:

The point of examples like these is to illustrate how counterintuitive the chance of occurrence of random coincidences may seem, especially to people who have had little exposure to probability theory and statistical analysis. Indeed, I have found [

Widespread occurrence of plagiarism and related forms of academic dishonesty facilitated by access to the Internet is an increasingly serious matter at universities and colleges. While it is understandable that the faculty and administrators of institutions of higher learning will want to take strong and decisive action against perpetrators,

it is also necessary to be mindful that false judgments can damage or destroy the careers of nonculpable students charged with plagiarism. This balance of judgments is particularly critical in cases such as may occur in the physical sciences, engineering, mathematics, and other quantitative disciplines in which outright plagiarism is harder to recognize than in the humanities and social sciences because of the greater possibility for overlap (i.e. coincidence) between the written work submitted by a student and the instructor’s solutions to the same problems.

The analysis of this paper leads to a probability function by which to determine the statistical significance of such coincidences. It is to be emphasized that use of the term “coincidence” does not mean merely that a student’s numerical answer to a problem coincides with the instructor’s answer. For an advanced course in a quantitatively rigorous subject like physics or engineering, an instructor might generally expect most students in the class to answer problems correctly. Rather, the term “coincidence” refers to unusually close elements of correspondence between the expression of the student’s solutions and instructor’s solutions that might suggest illicit copying.

The methodology of this paper addresses two seminal questions:

(1) WHICH PROBABILITY?

ANSWER: In the interest of fairness to an accused student who has denied culpability, the adjudicating panel should ascertain the probability that the evidence offered by the accuser could be plausibly attributable to pure chance. This emphasis is diametrically opposite to that of many trial juries in the US and UK [

(2) HOW TO CALCULATE?

ANSWER: The Principle of Maximum Entropy (PME) provides the most objective method of determining the probability distribution of coincidences consistent with known or readily ascertainable statistical information.

Implementation of the PME entails maximization of the Shannon-Jaynes entropy

Note that a statistical threshold, whether 5% or some other adopted value, serves only for assisting the adjudicating panel to decide how to act upon the objectively calculated PME probability (19)―that is, for judging whether the resulting probability is sufficiently high to sustain the null hypothesis of random coincidences between the solutions of student S and the answers of professor P. Arbitrary thresholds of significance play no role in the calculation of the probability of coincidence.

The PME calculation of probability provides an unbiased method of inferential reasoning in the absence of complete information. The interpretation of the statistical significance of this probability is the only point at which a subjective judgment enters. Nevertheless, there is always a non-zero chance that inferential reasoning, however, unbiased and carefully executed, may still lead to a false judgment. The results of this paper permit one to calculate what that chance is. To reduce that chance to zero, however, is not a matter of statistics, but requires fundamental changes in the way courses are taught. I offer the following suggestions drawn from my own teaching experiences at all levels of undergraduate and graduate instruction in physics.

Since opportunities for plagiarism will, if anything, only increase as more information, including solutions manuals to standard textbooks, are legitimately or illegitimately posted on the Internet, the most effective course of action is to adopt a teaching strategy whereby plagiarism confers no advantage.

In my own courses, I inform students at the start of each academic period (of duration one semester where I work) that assigned homework problems are exclusively for their own benefit to help them determine to what extent they understand the concepts and examples worked out in class. Consequently, students are not penalized for errors in their homework, nor are they rewarded for solving problems correctly. Students are permitted to work together on problems and to use various resources, including the Internet, for assistance if necessary―although they are urged to try to do assignments by themselves first and not simply to copy someone else’s work.

A due date is given for each problem set, at which time solutions to the problems are posted, and the next set of problems is assigned. The students’ homework papers are not graded, but may be collected to give me an evolving sense of how individual students and the class as a whole are dealing with the subject matter of the course.

Students who turn in papers with solutions copied from the Internet or some other source reap no benefit in the form of points toward their course grade. The advantage, however, to students who have worked the problems for themselves is that they presumably have acquired a deeper understanding of the subject matter than the copiers and will perform better on tests that actually contribute to their grades. Under the conditions of a test, devices of all kinds (computers, mobile phones, etc.) that connect to the Internet are not permitted in class; the only auxiliary device students may use is a hand calculator.

It is possible, of course, that a student who copied solutions from the Internet may nevertheless have understood the material well enough to get a good grade on a test. To an ethical “purist” who believes that no bad act should go unpunished, the thought of allowing a student who plagiarized an assignment, even one that is not graded, to escape judgment may be unacceptable. To this objection I can only reply that there is no practical way to prevent students from searching the Internet if they are determined to do so. An instructor can only take steps, such as outlined above, to ensure that copying from the Internet or any other source brings no reward.

One final point: Besides tests, an important part of a student’s grade in nearly every science course that I teach is a written paper and associated in-class, computer-projected slide presentation of a topic pertinent to the course that each student researches during the academic period. Early in the semester, students are informed about, and warned against, committing plagiarism in their research projects. For assignments of this kind, which entail extensive research, writing, and speaking (in contrast to relatively brief mathematical solutions of quantitative problems), the occurrence of plagiarism is usually readily discernable and easily provable even without formal statistical analysis.