I have been studying the fair use defense in copyright law for 25 years, writing a treatise about it, and a number of articles, most recently with Judge Richard Posner (92 Cal. L. Rev. 1639 (Dec. 2004)). Regrettably, I wouldn't say I understand the doctrine any better now than when I started. This lack of intellectual growth is a reflection, principally, of my own shortcomings. I say "principally" rather than "solely" because the doctrine has undergone some seismic changes, first with the Supreme Court's 1984 Sony decision (464 U.S. 417), and then with the Court's virtual abandonment of Sony 10 years later in Campbell (51o U.S. 569). The contours of fair use in cyberspace and to what civil law countries would call personal use, are other sources of tension. These wrinkles aside, others seem to have a more trouble-free attitude toward how the doctrine works than I.
The fourth fair use factor "the effect of the use upon the potential market for or value of the copyrighted work" is one area where there is considerable debate. There are a number of problems. I mention only two: (1) can one include lost licensing fees as harm? (2) is the fourth factor the "single most important" of the factors?
The argument that the fourth factor is "undoubtedly the single most important factor" is traced to Harper & Row's statement to that exact effect, 471 U.S. 539, 566 (1985). The statement is erroneous and it relied entirely the Nimmer treatise for the proposition. I have, in my treatise on fair use, shown how none of the cases Nimmer cited support the proposition. Accuracy of case law citation aside, treating the fourth factor as the most important would make fair use a one factor doctrine, and to the exclusion of important questions such as the transformative nature (or not) of the use. Fortunately, in Campbell, the Supreme Court quietly jettisoned Harper & Row on this point, holding that the fourth factor "no less than the other three, may be addressed only through a sensitive balancing of interests." 510 U.S. at 591. The Second Circuit quickly grasped Campbell's abandonment of the most important factor argument in the Texaco case, 60 F.3d 913, 926 (2d Cir. 1995).
The Texaco case also confronted whether licensing fees may be taken into account in considering potential harm to the market. Fair use is a privileged, uncompensated use: if you establish it, there is no payment; if you don't, you have to pay up (usually). Yet, one does not know if the defense has been successful until all the fair use factors have been analyzed, including the fourth. Hence a perceived dilemma: if one includes the potential loss of licensing fees in evaluating the fourth factor (and the defense as a whole), are you prematurely making a decision that the use isn't fair, thereby prejudicing the outcome? On the other hand, if you exclude potential lost licensing fees on that ground, aren't you also prejudicing the outcome the opposite direction?
In literary theory, philosophy, and religious text interpretation, this problem is called the hermeneutic circle: one can't understand a part unless one understands the whole of which the parts are of course components; but one can only understand the whole by understanding its parts. Rather than being a Road to Nowhere the problem may merely be a way of expressing how understanding occurs: as more information is acquired, one's interpretation of both the whole and the parts change until one comes to an ultimate conclusion. This is not dissimilar from Charles Peirce's theory of abductive reasoning.
As applied to fair use, Texaco takes the approach that the existence or nonexistence of licensing for the use in question is a piece of information one may take into account. One might also consider custom in the industry and whether the market is one reasonably within the copyright owner's monopoly; for example, a copyright owner may have been able to extract license fees merely because it is cheaper to pay them rather than litigate: no court is likely to perceive a license fee for quotes in a book review to be harm cognizable under the fourth factor. The Sixth Circuit followed Texaco in the Princeton University Press case, 99 F.3d 1381, 1387 (1996)(en banc), but much thinking needs to be done still on the issue.