Integrating Orbitofrontal Cortex into Prefrontal Theory: Common Processing Themes across Species and Subdivisions

Neurons in the Orbitofrontal Cortex Encode the Acquired Incentive Value of Cues

Although neurons in the orbitofrontal cortex do appear to represent associations between cues and reinforcers, a model that simply implicates the orbitofrontal cortex in representing such associations would be incomplete. After all, much behavior that depends on these associations is left unimpaired following a lesion to the orbitofrontal cortex. Conversely, some of the impairments caused by such lesions do not appear to result from deficits in associative representations. For example, patients with orbitofrontal lesions show impulsive behavior and poor decisionmaking. (Harlow 1868; Damasio 1994). This function of the orbitofrontal cortex has been formally implicated in performance of a gambling task, in which subjects must choose from decks of cards associated with rewards and penalties of different sizes (Bechara et al. 1997). Optimal performance requires that subjects learn the rewards and penalties associated with the different decks and then bias their choices based on this information to avoid large penalties. Patients with orbitofrontal damage are impaired on this task, while showing normal performance on other measures of prefrontal function (Bechara et al. 1998). In addition, regions within the orbitofrontal cortex exhibit marked hemodynamic changes correlated with the rewarding and punishing outcomes in versions of this task (Rogers et al. 1999; O'Doherty et al. 2001), and cells recorded from depth electrodes in human orbitofrontal cortex also appear to be activated in this task (Adolphs et al. 2000). Nevertheless, in many cases patients with orbitofrontal damage are able to verbalize the outcomes associated with the different decks of cards, demonstrating that they have acquired some information about the associations between the cards and their outcomes. Similarly, humans, nonhuman primates, and rats with orbitofrontal lesions can show intact performance on discriminations that require responding to neutral cues that predict reinforcement, while exhibiting marked impairment on reversals that presumably depend on similar information (Rolls et al. 1994; Meunier et al. 1997; Saddoris et al. 2001). Orbitofrontal lesions also affect performance in so-called devaluation paradigms. In these tasks, animals (rats or primates) with orbitofrontal damage respond normally to associations between cues and food reinforcers, but fail to modify their behavior to the cues when these representations are probed by reducing the incentive value of the food (Butter et al. 1963; Gallagher et al. 1999; Baxter et al. 2000). Similar to patients with orbitofrontal damage, these animals can acquire associative information regarding the cues but are not able to modify their responding to reflect that knowledge (Box ).

These results indicate that processing of information in the orbitofrontal cortex is not simply related to encoding associations between stimuli and reinforcers. Rather, it may be that firing in the orbitofrontal cortex in response to cues represents, in part, information regarding the current incentive value of the associated reinforcer. This suggestion reflects the connections of the orbitofrontal cortex with the basolateral amygdala (Krettek and Price 1977; Price et al. 1987; Kita and Kitai 1990; McDonald 1991; McDonald et al. 1996), and the observation that, on identical tasks, lesions of the amygdala frequently result in impairments similar to those caused by orbitofrontal lesions (Jones and Mishkin 1972; Gaffan and Murray 1990; Hatfield et al. 1996; DeCoteau et al. 1997; Malkova et al. 1997;Bechara et al. 1999). Although changes in the incentive value of reinforcers alter firing in the orbitofrontal cortex to sensory properties of primary reinforcers in rats (D. Katz, pers. comm.) and primates (Critchley and Rolls 1996b), recording studies have only recently examined the relationship between differential firing in the orbitofrontal cortex to cues that predict reward (such as that illustrated in Fig. 1) and the value of the associated reinforcer (Rolls et al. 1996; Schoenbaum et al. 1999; Tremblay and Schultz 1999).

In these studies, one approach has been to assess neural firing to cues as an animal's preference for the associated reinforcer changes. Schultz and colleagues have done this by recording as monkeys are presented with discriminations involving differently-preferred reinforcer pairs. They found that cue-selective firing activity in the orbitofrontal cortex changed across blocks of trials in which different pairs of cues and reinforcers were presented (Tremblay and Schultz 1999), and that the changes in selectivity correlated with reinforcer preference, which was defined prior to recording in a choice version of the task. For example, a neuron that fired selectively to a cue paired with the preferred reinforcer in one block would decrease firing to this cue and switch selectivity if it became the nonpreferred cue in a subsequent trial block (Fig. 2). These data demonstrate that firing to discriminative cues in the orbitofrontal areas does not represent reinforcer identity, but instead correlates with the current incentive value of the reinforcer.

Another approach has been to reverse previously learned reinforcement contingencies in discrimination tasks. Rapid acquisition of reversals is dependent on the orbitofrontal cortex in both rats and primates (Rolls et al. 1994; Meunier et al. 1997; Saddoris et al. 2001), and cue-selective firing in both rat and primate orbitofrontal cortex is severely affected when the cues associated with appetitive and aversive reinforcers are switched (Thorpe et al. 1983; Rolls et al. 1996;Schoenbaum et al. 1999). Indeed, the majority of cue-selective neurons in both species respond to reversal by becoming nonselective. In rats, activity in these cells no longer discriminates between the cues after reversal (Fig. 3A), not even when rats reach criterion performance on the reversed discrimination (Fig. 3B). These data are consistent with the proposal that this activity encodes incentive value and also demonstrate that encoding of incentive value is linked to specific cues. In other words, the firing represents the specific conjunction between a particular cue (or range of cues) and the current incentive value of an associated reinforcer. Similar conjunctive encoding is evident during reversal in primates (Thorpe et al. 1983). These conjunctions are disrupted by reversal of the reinforcement contingencies, causing the cells encoding them to become nonselective; a largely separate population of cells represents the new conjunctions after reversal (Schoenbaum et al. 1999). As we have noted, the orbitofrontal cortex appears to be crucial for using information regarding the acquired incentive value of cues to guide responses (Box ).

Neurons in the Orbitofrontal Cortex Encode Incentive Value When Relevant to Performance

Unlike dorsolateral prefrontal subdivisions, the orbitofrontal areas are implicated in emotion, social behavior, and decisionmaking (Damasio 1994; Bechara et al. 1997). These functions appear to be unique to the orbitofrontal regions (Bechara et al. 1998) and may depend on strong reciprocal connections with the amygdala (Krettek and Price 1977; Price et al. 1987; Kita and Kitai 1990; McDonald 1991; McDonald et al. 1996). Studies linking neural activity in the orbitofrontal cortex to the acquired incentive value of cues confirm the unique role of this area of the prefrontal cortex in processing affective or motivational information attached to cues. However, if the orbitofrontal cortex is important for translating incentive value into goal-oriented behavior, one key question is the relevance of encoding that has been observed in this region to such behavior. Several recent experiments indicate that, in fact, encoding of incentive value in the orbitofrontal cortex is strongly correlated with behavior and task performance.

Initial recordings in rat orbitofrontal cortex showed that task variables in an eight-odor discrimination task were represented by the firing of the population according to the requirements of the task (Schoenbaum and Eichenbaum 1995a,b). As discussed earlier, neurons in the orbitofrontal cortex of well-trained rats fired selectively to the odor cues in this task (Fig. 1). Ensemble analyses (Fig.4A) confirmed that activity in these neurons represented odor identity and valence (Fig. 4B), features that were useful in the performance of the task. In addition to these neural correlates, however, activity during odor sampling also reflected aspects of the odor sequence, both consequential and irrelevant. For example, some neurons fired differently to odors that predicted the odor of the following trial. This prospective encoding was reflected in the rat's behavioral performance; rats initiated trials more rapidly or more slowly when such odors indicated that a positive or negative odor could be predicted on the upcoming trial. Other neurons fired differently to a given odor depending on the valence of the previous trial. Such retrospective information had no bearing on the rat's performance in this task. Subsequent ensemble analyses of encoding of these features by these same neural populations revealed that the cells preferentially encoded the information that was behaviorally relevant. Irrelevant features of the odor sequence, such as the valence or identity of the odor on the previous trial, were poorly represented (Fig. 4C), but the predictive relationships between specific odors were well represented (Fig. 4D). Consistent with the preferential encoding of behaviorally relevant information, this information was well encoded by the ensembles prospectively (Fig. 4D, left panel) but not retrospectively (Fig. 4D, right panel). In other words, information conveyed by the odors was represented in the orbitofrontal cortex according to its usefulness in the task.

If neural activity reflects the use of incentive value information in guiding behavior, then changes in discriminative behavior should be reflected in the development of encoding in the orbitofrontal cortex. To test this hypothesis, we recorded in rat orbitofrontal cortex during learning of new odor discrimination problems (Schoenbaum et al. 1999). Discriminations involved either two or four novel odors paired with sucrose or quinine. We examined activity in a population of neurons similar to those from earlier work (Fig. 1). Activity in these neurons was examined across several phases in each session related to learning and choice performance (Fig. 3C). We found that the vast majority (91%) of orbitofrontal cortex neurons with selective firing during stable performance on the task only developed that selectivity during highly accurate performance (post-criterion, Fig. 3A). In other words, these cells did not fire selectively at the start of the session when the odors were novel (early phase, Fig. 3A), nor did they fire selectively during a later phase of training (late phase, Fig. 3A) characterized by chance performance on the discrimination but development of long-latency responses to negative cues (Fig. 3C). Similar changes in response latency have been shown to be related to the incentive value of neutral cues paired with reinforcers (Holland and Straub 1979). Yet in our paradigm changes in response latency are not reflected in population firing activity in the late phase when choice performance is unchanged. As the example illustrates (Fig. 3B), selectivity only emerged during postcriterion performance when choice performance changed to reflect the information provided by the odor cues. These data indicate that the firing changes in the orbitofrontal cortex may be separated from other behaviors dependent on incentive value. Furthermore, as mentioned earlier, selective firing in the majority of these neurons was disrupted by reversal of the reinforcement contingencies of the odor cues (Fig. 3A,B). This population was, in effect, replaced by a separate group of neurons (∼25% of the remaining cells) that became selective after reversal. Thus, the selective activity did not simply reflect the discriminative response or motor set but, rather, reflected the linkage of the acquired incentive value of the specific odor cues with the use of that information to guide choice performance.

If firing to odor cues in orbitofrontal cortex represents information that is relevant to task performance, then one would predict (Schoenbaum and Eichenbaum 1995b; Schoenbaum 1998) that correlations to odors would be different if the way in which odors were used to guide performance were different. This prediction is borne out in a recent study by Ramus and Eichenbaum (2000). In this study, rats sampled an odor on each trial and were rewarded for responding if the odor did not match the odor sampled on the immediately preceding trial (continuous delayed-nonmatch to sample, or cDNM, task). Although odor identity was important in determining whether a match or nonmatch was made, the identity of the odor was not, by itself, correlated with outcome. Thus, the rule guiding the odor-based performance differed from that in the discrimination task. Notably, orbitofrontal cortex lesions can affect the performance of rats in both tasks (Eichenbaum et al. 1983; Otto and Eichenbaum 1992).

In this context, in which a given odor could be rewarded on one trial but not on the next, selective firing to any of the eight different odors was observed in <15% of the orbitofrontal cortex neurons (43 neurons) compared to 77% of neurons in the eight-odor discrimination task. In addition, many odor-selective cells (30 of 43 neurons) actually fired differentially to a particular odor based on whether it was a match or a nonmatch presentation, rather than on the basis of the odor identity (Fig. 5A). In fact, nearly 64% of the orbitofrontal cortex neurons recorded in this task (175 of 276 cells) fired differently to the same group of odors when match and nonmatch presentations were compared, and there was a strong correlation between performance in different sessions and the proportion of cells with match/nonmatch correlates and no correlation with representation of odor (Fig. 5B). In contrast, such retrospective information about odor sequence was represented at the level of chance during discrimination performance (Fig. 4C; right panel, Fig. 4D), and odor identity was well represented (Fig. 4B).

In summary, when odor-guided performance was based on matching comparisons, orbitofrontal cells responded differently according to whether an odor was a match or nonmatch presentation. Moreover, encoding developed with accurate choice performance, as was observed in the orbitofrontal cortex during go/no-go odor discrimination training (Schoenbaum et al. 1999). Although this first report of activity in orbitofrontal cortex in the cDNM task did not separate learning from changes in choice performance, we predict that more subtle changes in behavior (such as response latency), based on the associations between match or nonmatch odor presentations and outcomes, were present in earlier sessions in which choice performance was poor and match/nonmatch correlates were rare. Nevertheless, the findings are consistent with our earlier report indicating that information regarding the acquired incentive value of cues or cue configurations is encoded when relevant to task performance.

Considered together, these two studies in rat orbitofrontal cortex are similar to recent reports of encoding of visual cues in primate dorsolateral regions (Rainer et al. 1998; Asaad et al. 2000). Although earlier reports of activity in these regions stressed segregation of encoding properties according to anatomical location (Wilson et al. 1993), these more recent studies found that encoding properties were also related to task demands. Cells responsive to object identity were prevalent when the discrimination performance of the monkey was based on associations between specific objects and outcomes, whereas cells responsive to object location or place were prevalent when the monkey's discrimination performance was based on associations between these cue features and outcomes. These findings and those in the orbitofrontal cortex indicate that across species and across subdivisions, processing in the prefrontal cortex may be characterized by encoding of specific attributes of cues according to the use of these attributes to guide task performance. In the dorsolateral regions, the cue attributes are best defined by physical features such as object identity, color, or location. In the orbitofrontal regions, cue attributes are related to acquired motivational or incentive value.

Task- or performance-relevant processing bears a striking similarity to the concept that the prefrontal cortex represents rules that can be applied to guide behavior. Proponents of rule representation (Dias et al. 1996; Wise et al. 1996; Wise and Murray 2000) hold that the prefrontal cortex is specialized to identify or acquire rules that allow appropriate behavior in different circumstances or contexts. Prefrontal mechanisms are thought to be particularly important when conflicting priorities or contexts must be resolved as in reversal tasks or social interactions, but one would expect that they would be operative in all situations (Cohen et al. 1998). If this were true, one might expect firing activity in prefrontal subdivisions to reflect the linkage of specific information, such as incentive value or features of visual cues, to appropriate behavior. This, of course, is precisely what we and others have observed in the orbitofrontal cortex.

One implication of the idea that prefrontal areas, including the orbitofrontal cortex, display rule-based encoding is that the constituent elements that make up the rule are accessed or represented independently (Schoenbaum 1998). For example, if an animal chooses not to respond to a cue to obtain a reward after selective satiation of that reward, then the rule guiding behavior has changed but the association between the cue and the reward has not. The orbitofrontal cortex must access that information from elsewhere in the brain when evaluating the context (satiation) and making the decision not to respond. Accessing and manipulating that information to construct the neural representation appropriate for a given context requires a working memory function. Such a working memory function is, of course, readily apparent for visual attributes of cues in studies of primate dorsolateral cortex (Wilson et al. 1993). Next we present data that argue for a similar function in the orbitofrontal regions of the prefrontal cortex.

Neurons in the Orbitofrontal Cortex Encode Incentive Value During Delays

To some extent, the wide acceptance of the idea that the dorsolateral prefrontal cortex is involved in working memory comes from the demonstration that cells recorded in this region in visually cued delayed-response tasks exhibit selective activity during the delay phase (for examples, see Fuster et al. 1982; Funahashi et al. 1989;Wilson et al. 1993; Rainer et al. 1998). Animals with dorsolateral prefrontal cortical lesions are impaired on these tasks, and neurons in this region fire selectively to features of the visual cues that the animal must remember across the delay, such as the color, identity, or location of the particular item that was shown. Careful examination has shown that a large proportion of these neurons do, indeed, represent these attributes of the cue rather than encoding the motor response. Delay-selective neural activity has also been reported in the orbitofrontal cortex, but unlike neural activity in dorsolateral regions, activity in the orbitofrontal areas represents the incentive value or significance of the cues during delays.

Orbitofrontal neurons are active in both rats and primates during short delays interposed between responses and reinforcer delivery (Schoenbaum et al. 1998; Tremblay and Schultz 1999, 2000a) or during delays between cue presentation and responses to instructions preceding reinforcer delivery (Hikosaka and Watanabe 2000). For example, in rats learning new odor-discrimination problems (Schoenbaum et al. 1998), orbitofrontal neurons exhibit activity during a delay prior to reinforcement (Fig. 6). This activity appears to encode expectancy of the impending outcome, independent of the response made to gain access to the outcome. Activity differs when the same response is made but different reinforcers are anticipated. Similar findings have been made in primate orbitofrontal cortex (Tremblay and Schultz 1999). In that report, differential firing was not observed during delays when a response was made but no reward was expected (Tremblay and Schultz 2000a). Similar to activity during presentation of predictive cues, delay-related activity in the orbitofrontal cortex emerges with learning (Schoenbaum et al. 1998,1999; Tremblay and Schultz 2000b) and is linked to the incentive value of the impending reward rather than to the specific identity of the reward (Tremblay and Schultz 1999; Hikosaka and Watanabe 2000). Similar encoding during delay periods or in anticipation of events has subsequently been reported in other tasks in which predictions can be made regarding impending events or outcomes (Lipton et al. 1999). Thus, the orbitofrontal cortex may prospectively encode a variety of meaningful events in tasks.

The encoding of anticipated events by selective subsets of orbitofrontal neurons is particularly intriguing when compared to the orbitofrontal activation reported in imaging studies when expectations are not met (Nobre et al. 1999) or when guesses regarding likely outcomes must be made (Elliott et al. 2000). Although the hemodynamic changes that produce these signals likely reflect activity in many neurons (Scannell and Young 1999), the anticipatory encoding we and others report in specific subsets of cells may form the basis for the more massive activation seen with imaging techniques when outcomes deviate from expectations. Notably, the selective firing during delays is observed in rat orbitofrontal cortex early in learning and before changes in response selection are evident, suggesting that expectations are formed on the basis of even incomplete information. The resultant signal demonstrated in these studies where outcomes do not match expectations could represent an attentional function as inactive cells are engaged to encode new information. We observe just such a phenomenon following during reversal training (Schoenbaum et al. 1999).

Finally, although anticipatory encoding of reinforcers during delays is also observed in other prefrontal regions, the firing of delay cells in the orbitofrontal areas is related to the incentive value of the outcome rather than the specific identity of the reinforcer. Schultz and colleagues demonstrated this property in their recent work described above (Tremblay and Schultz 1999), in which monkeys were presented with a visual discrimination task involving several different visual cues paired with distinct and differently-preferred reinforcers. Selective firing during a delay interposed between the response and reward delivery changed based on the monkey's relative preference between the reinforcers available in a given block of trials. In other words, a neuron would fire differently during a delay in anticipation of a reinforcer depending on whether it was the preferred or nonpreferred of the two in the trial block. Similar findings have since been reported during delays that precede responding as well (Hikosaka and Watanabe 2000). Those authors, in fact, compared delay-selective activity in the orbitofrontal and lateral prefrontal areas and found that selective firing during the delay was far more likely to represent reinforcer identity in the lateral prefrontal cortex, whereas similar activity in orbitofrontal regions typically did not discriminate between reinforcers except when food preferences changed during testing (Fig. 7). The dependence of these correlates on the incentive value of the reward therefore appears to distinguish anticipatory delay activity in orbitofrontal cortex from similar activity elsewhere in prefrontal cortex (Watanabe 1996; Hikosaka and Watanabe 2000). Viewed from this perspective, it is worth noting that several studies have found delay-dependent effects of orbitofrontal cortex lesions, a hallmark of working memory in primates (Box ).

Conclusions

Here we have reviewed behavioral and electrophysiological evidence from studies in rat and primate orbitofrontal cortex to argue that orbitofrontal function can be integrated into the framework of prefrontal theory. This evidence indicates that several features of encoding (which appear to be conserved across species) in primate dorsolateral prefrontal cortex may be generalized to explain encoding in the orbitofrontal region. The first of these is an integrative function operating across multiple phases of a task that results in the preferential representation of afferent information according to the information's relevance to task performance. These representations could in effect correspond to the rules prominent in several models of prefrontal function, linking information regarding context and incentive value with actions. This proposal predicts that behaviors based on information critically contained in these representations, such as the incentive value of cues in a given context, will be disrupted by manipulations that alter these representations in intact animals.

The second of these features, implied by this model, is a working memory function, which allows information about cues to be accessed and manipulated even when those cues are not present in the environment. Networks in the orbitofrontal cortex would interact with other structures in this function in the same way that dorsolateral regions are proposed to function in concert with posterior cortical areas (Wilson et al. 1993; Goldman-Rakic 1996). Our discussion has highlighted encoding of incentive value and connections with the amygdala (Schoenbaum et al. 1998, 1999; Gallagher et al. 1999), but connections with other structures are also evident in the encoding properties of orbitofrontal neurons (Schoenbaum and Eichenbaum 1995a;Schoenbaum 1998; Lipton et al. 1999; Ramus and Eichenbaum 2000) and in activations reported in recent imaging studies (Frey and Petrides 2000). We predict that encoding in the orbitofrontal cortex should be sensitive to damage to related structures, and preliminary results indicate that this is indeed the case (Schoenbaum et al. 2001). In addition, performance across delays based on information critically represented in the orbitofrontal cortex, such as that regarding the incentive value of anticipated outcomes, should be adversely affected by orbitofrontal lesions.

In conclusion, effective rule learning seems to require representational memory capabilities, and representational memory divorced from the ability to acquire rules fails to describe the complex deficits attributable to damage to the prefrontal cortex. Accordingly, it appears that both rule learning and representational memory may provide the functional homogeneity proposed by Rosenkilde (1979) for this otherwise heterogeneous brain region.