letters to nature

27. Henson, S. A. & Warner, R. R. Male and female alternative reproductive behaviors in fishes: a new to four items, and that performance is not significantly influenced
approach using intersexual dynamics. Annu. Rev. Ecol. Syst. 28, 571–592 (1997).
28. Giraldeau, L. A. & Caraco, T. Social Foraging Theory (Princeton Univ. Press, Princeton, 2000).
by perceptual or verbal processes1,3.
29. Sober, E. & Wilson, D. S. Unto Others (Harvard Univ. Press, Cambridge, Massachusetts, 1998). In the first experiment, we recorded ERPs to the onset of a four-
30. Boyd, R. & Richerson, P. J. Group selection among alternative evolutionarily stable strategies. J. Theor. item memory array so that we could observe the sustained electro-
Biol. 145, 331–342 (1990).
physiological response during the memory retention interval. A few
previous ERP studies have observed a sustained response during
Supplementary Information accompanies the paper on [Link]/nature.
working memory tasks for foveally presented stimuli, but did not
Acknowledgements We thank A. Dornhaus, M. Enquist, E. Fehr and L.-A. Giraldeau for examine lateralized effects13,14. In contrast, we took advantage of the
comments on a previous version of this Letter. primarily contralateral organization of the visual system by pre-
senting lateralized stimuli in each hemifield so that we could
Authors’ contributions J.M.M. formulated the main ideas as a result of conversations with A.I.H.;
J.M.M. also formulated the model, and was responsible for the material in Box 1; Z.B. carried out
measure the spatially specific hemispheric responses to memory
the computations, and prepared the figures; A.I.H. surveyed the literature, and had the main arrays that were either contralateral or ipsilateral with respect to
responsibility for writing the Letter. electrode position15,16. Approximately 200 ms after the onset of the
memory array, we found a large negative-going voltage over the
Competing interests statement The authors declare that they have no competing financial hemisphere that was contralateral to the memorized hemifield, and
interests.
this response persisted throughout the duration of the memory
Correspondence and requests for materials should be addressed to J.M.M. retention interval (Fig. 1b). This response was focused primarily
([Link]@[Link]). over the posterior parietal and lateral occipital electrode sites and
strongly resembled delay activity recorded from individual neurons
in monkey visual cortex12,17.
Numerous processes contribute to visual memory performance,
and we sought to determine which aspects of processing are
reflected by the contralateral delay activity. Although this effect
.............................................................. seems to reflect the maintenance of object representations from the
Neural activity predicts individual memory array, it is necessary to rule out the possibility that it
reflects executive processes18 involved in performing the task, or
differences in visual working even more general processes such as increased effort or arousal19–21.
In the second experiment, we tested this by varying the number of
memory capacity items in the memory array to establish whether the amplitude is
sensitive to the number of representations that are being held in
Edward K. Vogel & Maro G. Machizawa visual memory. Memory arrays in this experiment varied from one
to four items in each hemifield (average capacity in this task is
Department of Psychology, University of Oregon, Eugene, Oregon 97403-1227, normally around three items3,7). To compare directly the magnitude
USA of activity across array sizes, we constructed ‘difference waves’ in
.............................................................................................................................................................................
which the ipsilateral activity was subtracted from the contralateral
Contrary to our rich phenomenological visual experience, our activity for each array size, which removes the contribution of any
visual short-term memory system can maintain representations nonspecific, bilateral ERP activity.
of only three to four objects at any given moment1,2. For over a As shown in Fig. 2a, the amplitude was highly sensitive to the
century, the capacity of visual memory has been shown to vary number of items in the memory array. Indeed, increasing an array
substantially across individuals, ranging from 1.5 to about 5
objects3–7. Although numerous studies have recently begun to
characterize the neural substrates of visual memory processes8–12,
a neurophysiological index of storage capacity limitations has not
yet been established. Here, we provide electrophysiological evi-
dence for lateralized activity in humans that reflects the encoding
and maintenance of items in visual memory. The amplitude of
this activity is strongly modulated by the number of objects
being held in the memory at the time, but approaches a limit
asymptotically for arrays that meet or exceed storage capacity.
Indeed, the precise limit is determined by each individual’s
memory capacity, such that the activity from low-capacity indi-
viduals reaches this plateau much sooner than that from high-
capacity individuals. Consequently, this measure provides a
strong neurophysiological predictor of an individual’s capacity,
allowing the demonstration of a direct relationship between
neural activity and memory capacity.
To measure the neural correlates of visual memory capacity, we
recorded event-related potentials (ERPs) from normal young adults
while they performed a visual memory task. On each trial they were
presented with a brief bilateral array of coloured squares and were
asked to remember the items in only one hemifield, which was
indicated with an arrow (Fig. 1a). Memory was tested one second Figure 1 Stimuli and results from experiment one. a, Example of a visual memory trial for
later with the presentation of a test array that was either identical to the left hemifield. SOA, stimulus onset asynchrony. b, Grand averaged ERP waveforms
the memory array or differed by one colour. Subjects pressed one of time-locked to the memory array averaged across the lateral occipital and posterior
two buttons to indicate whether the two arrays were identical or parietal electrode sites in experiment one. The two grey rectangles reflect the time periods
different. We have used variations of this paradigm previously and for the memory and test arrays, respectively. Note that, by convention, negative voltage is
have found that observers are accurate for array sizes of up to three plotted upwards.
748 ©2004 Nature Publishing Group NATURE | VOL 428 | 15 APRIL 2004 | [Link]/nature
 letters to nature
from one to two squares or from two to three squares resulted in a capacity arrays. However, if it reflects executive processes or the
substantial increase in amplitude. Moreover, because memory amount of general effort, we would expect that amplitude should
performance for near-capacity arrays can fluctuate over time, continue to increase for supra-capacity arrays.
leading to occasional incorrect responses, we compared the ampli- The results of the third experiment show that although there was
tude of the delay activity for correct and incorrect trials. The a significant increase in amplitude from arrays of two items per side
amplitude for incorrect trials was considerably smaller than that to arrays of four items per side, there was no increase from four
for correct trials (P , 0.01), further suggesting that the delay items to six items (Fig. 2b). That is, the amplitude reached a limit
activity specifically reflects the maintenance of successful represen- with arrays of approximately four items per side. We tested this
tations in visual memory. Nevertheless, it is possible that the extent further in the fourth experiment by following the same experimen-
of executive processes also increases with additional memory items. tal design but with larger array sizes (Fig. 2c). Again we found a
Moreover, there are small but reliable differences in accuracy across significant amplitude increase from two to four items per side, but
array sizes, which leaves open the possibility that increases in no increase from four items to either eight or ten items per side.
arousal or effort for larger arrays may have produced the increase These results strongly support the hypothesis that the delay activity
in amplitude. reflects the specific maintenance of representations in visual mem-
The amplitude of the contralateral delay activity may have ory because its amplitude is sensitive to the number of successful
increased as the result of increasing the number of representations, representations that are active in memory at the time. In addition,
more executive processing, or higher difficulty; however, these the absence of continued amplitude increase beyond capacity also
alternatives make different predictions for array sizes that exceed minimizes the possibility that the sub-capacity amplitude effects in
visual memory capacity. For example, when comparing a trial
containing four memory items to a trial containing eight, the
number of active memory representations should be approximately
identical, because both trials exceed a typical individual’s memory
capacity. That is, the subject can maintain only three to four items
whether the attended side of the array contains four or eight items.
In contrast, the difficulty and extent of executive processing
increases substantially for eight-item arrays compared with four-
item arrays22. Indeed, this has been a significant limitation of
previous neurophysiological studies that have reported memory
load effects, because the amount of activity continues to increase for
loads that exceed capacity, indicating that these measures are not
directly measuring memory capacity10,21,23. Therefore, in the third
and fourth experiments, we compared the delay activity for supra-
capacity arrays with memory arrays at or near capacity. If it reflects
the active representations held in visual memory, we would expect
no difference in amplitude between supra-capacity arrays and

Figure 2 ERP difference waves at lateral occipital and posterior parietal electrode sites for
experiments two, three and four, respectively. a, Pairwise comparisons yielded significant
differences in amplitude between array sizes of one, two and three (P , 0.001), but no Figure 3 Mean amplitude and visual memory capacity. a, Mean amplitude and visual
difference between three and four items (P . 0.20) in experiment two. b, c, No memory capacity across experiments two, three and four. Error bars reflect 95%
significant differences in amplitude were observed between arrays of four, six, eight or ten confidence intervals. b, The correlation between an individual subject’s memory capacity
items (P . 0.25 in all cases) in experiments three and four. and the increase in amplitude of delay activity between two- and four-item arrays.
NATURE | VOL 428 | 15 APRIL 2004 | [Link]/nature
©2004 Nature Publishing Group 749
letters to nature
the second experiment were because of increases in the size of the high-level cognitive functions has been shown to be directly
‘attentional spotlight’24, because supra-capacity arrays require a influenced by his or her memory capacity5,27–29. These results
larger spotlight than at- or below-capacity arrays, but show no provide the first link between this important cognitive limitation
increases in amplitude. and neural activity. A
The supra-capacity array sizes in these experiments provided
substantial increases in both the extent of executive processes and
Methods
the difficulty in performing the task. For example, there was a 32%
Twelve neurologically normal college students participated in each experiment (age range
reduction in accuracy between arrays of four and ten items, yet there of 21–33) and gave informed consent according to procedures approved by the University
was no increase in the amplitude of the contralateral delay activity. of Oregon. Each of these observers performed 240 trials per condition in each experiment.
Furthermore, we also observed a more centrally distributed bilateral All stimulus arrays were presented within two 48 £ 7.38 rectangular regions that were
wave during the task that was modulated by the number of items in centred 38 to the left and right of a central fixation cross on a grey background
(8.2 cd m22). Each memory array consisted of 1–10 coloured squares (0.658 £ 0.658) in
the memory array. However, in sharp contrast to the contralateral each hemifield. Each square was selected at random from a set of seven highly
activity, the amplitude of this bilateral wave continued to increase discriminable colours (red, blue, violet, green, yellow, black and white), and a given colour
significantly for arrays that exceeded memory capacity, suggesting could appear no more than twice within an array. Stimulus positions were randomized on
that it is sensitive to the amount of general effort involved in each trial, with the constraint that the distance between squares within a hemifield was at
least 28 (centre to centre). The colour of one square in the test array was different from
performing the task21. the corresponding item in the memory array in 50% of trials; the colours of the two
These results suggest that the contralateral delay activity indexes arrays were identical on the remaining trials. At the beginning of each trial, a central
the currently active representations maintained in visual memory; arrow cue instructed the subjects to remember the items in either the left or the right
that is, increasing in magnitude as the number of items increases, hemifield.
We computed visual memory capacity using a formula developed by Pashler23 and
but reaching a limit once visual memory capacity is exhausted. To refined by Cowan7. Essentially, this approach assumes that if an observer can hold K items
demonstrate this effect further, we quantified the mean amplitudes in memory from an array of S items, then the item that changed should be one of the items
for each array size for experiments two to four. As shown in Fig. 3a, being held in memory on K/S trials, leading to correct performance on K/S of the trials on
amplitude increased monotonically from one to three items, but which an item changed. To correct for guessing, this procedure also takes into account the
false alarm rate. The formula is K ¼ S £ (H 2 F), where K is the memory capacity, S is the
this increase levelled off at three items. We also computed visual size of the array, H is the observed hit rate and F is the false alarm rate.
memory capacity estimates for each subject, using a standard ERPs were recorded in each experiment using our standard recording and analysis
formula7,25. The mean capacity of the group was 2.8 items, which procedures30, including rejection of trials contaminated by blinks or large (.18) eye
is approximately when the memory delay activity reaches a limit. movements. We recorded from 22 standard electrode sites (international 10/20 system)
spanning the scalp. We computed contralateral waveforms by averaging the activity
This further supports the proposal that the specific limitation in recorded at right hemisphere electrode sites when subjects were cued to remember the left
visual memory capacity determines when this delay activity reaches side of the memory array with the activity recorded from the left hemisphere electrode
a limit. sites when they were cued to remember the right side. Contralateral delay activity was
To gauge this relationship more finely, we examined the varia- measured at posterior parietal, lateral occipital and posterior temporal electrode sites as
the difference in mean amplitude between the ipsilateral and contralateral waveforms,
bility across individuals for each measure. That is, we assessed with a measurement window of 300–900 ms after the onset of the memory array.
whether a given individual’s memory capacity specifically dictates
Received 23 December 2003; accepted 26 February 2004; doi:10.1038/nature02447.
when his or her delay activity reaches a limit. If so, one would expect
low-capacity subjects to reach the limit for smaller array sizes than 1. Luck, S. J. & Vogel, E. K. The capacity of visual working memory for features and conjunctions. Nature
390, 279–281 (1997).
would high-capacity subjects. Unfortunately, it is difficult to deter- 2. Sperling, G. The information available in brief visual presentations. Psychol. Monogr. 74, Whole No.
mine precisely the limit with a categorical data set such as array size 498 (1960).
(for example, there is no array size of 2.6). Instead, we computed the 3. Vogel, E. K., Woodman, G. F. & Luck, S. J. Storage of features, conjunctions, and objects in visual
amplitude increase between two items and four items per side for working memory. J. Exp. Psychol. Hum. Percept. Perform. 27, 92–114 (2001).
4. Jevons, W. S. The power of numerical discrimination. Nature 3, 281–282 (1871).
each subject across all experiments, the logic being that the amount 5. Engle, R. W., Kane, M. J. & Tuholski, S. W. in Models of Working Memory: Mechanisms of Active
of amplitude increase between these two array sizes should be Maintenance and Executive Control (eds Miyake, A. & Shah, P.) 102–134 (Cambridge Univ. Press, New
specifically determined by memory capacity. For example, when a York, 1999).
6. Miller, G. A. The magical number seven, plus or minus two: Some limits on our capacity for
subject with a low capacity of 1.8 items is shown a two-item array, processing information. Psychol. Rev. 63, 81–97 (1956).
capacity should be completely consumed with the amplitude reach- 7. Cowan, N. The magical number 4 in short-term memory: A reconsideration of mental storage
ing a limit, resulting in little or no amplitude increase from two to capacity. Behav. Brain Sci. 24, 87–185 (2001).
four items. In contrast, a subject with a high capacity of 4.5 items 8. Jonides, J. et al. Spatial working memory in humans as revealed by PET. Nature 363, 623–625 (1993).
9. Courtney, S. M., Ungerleider, L. G., Keil, K. & Haxby, J. V. Transient and sustained activity in a
would be expected to be well below the limit for a two-item array, distributed neural system for human working memory. Nature 386, 608–611 (1997).
and should therefore show a large increase in amplitude for a four- 10. Cohen, J. D. et al. Temporal dynamics of brain activation during a working memory task. Nature 386,
item array. 604–608 (1997).
The magnitude of the amplitude increase between two and four 11. Miller, E. K., Erickson, C. A. & Desimone, R. Neural mechanisms of visual working memory in
prefrontal cortex of the macaque. J. Neurosci. 16, 5154–5167 (1996).
items was plotted as a function of each subject’s memory capacity in 12. Fuster, J. M. & Jervey, J. P. Neuronal firing in the inferotemporal cortex of the monkey in a visual
Fig. 3b. These two measures were very strongly correlated (r ¼ 0.78; memory task. J. Neurosci. 2, 361–375 (1982).
P , 0.0001), with low-capacity subjects producing very little ampli- 13. Ruchkin, D., Johnson, R., Grafman, J., Canoune, H. & Ritter, W. Multiple visuospatial working
memory buffers: Evidence from spatiotemporal patterns of brain activity. Neuropsychologia 35,
tude increase and high-capacity subjects showing larger amplitude 195–209 (1997).
increases. Importantly, an individual’s memory capacity was not 14. Ruchkin, D., Johnson, R., Canoune, H. & Ritter, W. Short-term memory storage and retention: An
significantly correlated with either the amplitude increase between event-related brain potential study. Electroencephalogr. Clin. Neurophysiol. 76, 419–439 (1990).
arrays of four and six items or the absolute amplitude of activity for 15. Gratton, G. The contralateral organization of visual processing: A theoretical concept and a research
tool. Psychophysiology 35, 638–647 (1998).
a given array size. 16. Woodman, G. F. & Luck, S. J. Electrophysiological measurement of rapid shifts of attention during
These results show that the observed memory delay activity visual search. Nature 400, 867–869 (1999).
indexes the maintenance of active representations in visual memory. 17. Miller, E. K., Li, L. & Desimone, R. Activity of neurons in anterior inferior temporal cortex during a
short-term memory task. J. Neurosci. 13, 1460–1478 (1993).
Moreover, they demonstrate a strong neurophysiological predictor
18. Baddeley, A. Exploring the central executive. Q. J. Exp. Psychol. A 49, 5–28 (1996).
of visual memory capacity. That is, simply by measuring the 19. Hillyard, S. A., Vogel, E. K. & Luck, S. J. Sensory gain control (amplification) as a mechanism of
amplitude increase across memory array sizes, we could accurately selective attention: Electrophysiological and neuroimaging evidence. Phil. Trans. R. Soc. Lond. B 353,
predict an individual’s memory capacity. Visual working memory is 1257–1270 (1998).
20. Rypma, B. & D’Esposito, M. D. The roles of prefrontal brain regions in components of working
thought to have a central role within cognition because it maintains memory. Proc. Natl Acad. Sci. USA 96, 6558–6563 (1999).
representations from the environment so that they may be acted on 21. Ruchkin, D., Canoune, H., Johnson, R. & Ritter, W. Working memory and preparation elicit different
or manipulated3,26. Indeed, an individual’s ability to perform many patterns of slow wave event-related brain potentials. Psychophysiology 32, 399–410 (1995).

750 ©2004 Nature Publishing Group NATURE | VOL 428 | 15 APRIL 2004 | [Link]/nature
 letters to nature
22. Rypma, B. & D’Esposito, M. D. A subsequent-memory effect in dorsolateral prefrontal cortex. Cogn. memory/articulatory suppression task was administered concur-
Brain Res. 16, 162–166 (2003).
23. Rypma, B., Prabhakaran, V., Desmond, J., Glover, G. H. & Gabrieli, J. D. Load-dependent roles of
rently with the VSTM task: throughout the trial, subjects rehearsed
frontal brain regions in the maintenance of working memory. Neuroimage 9, 216–226 (1999). two digits presented at trial onset and reported them at trial offset.
24. Eriksen, C. W. & St James, J. D. Visual attention within and around the field of focal attention: A zoom Performance in this task was high and independent of VSTM set size
lens model. Percept. Psychophys. 40, 225–240 (1986).
(92–94% accuracy across set sizes; F 5,80 ¼ 0.64, P ¼ 0.67), attesting
25. Pashler, H. Familiarity and visual change detection. Percept. Psychophys. 44, 369–378 (1988).
26. Logie, R. H. Visuo-Spatial Working Memory (Erlbaum, Hove, UK, 1995). to the absence of a trade-off between the verbal and visual tasks, as
27. Kane, M. J. & Engle, R. W. Working memory capacity and the control of attention: The contributions predicted from the independence of these two working-memory
of goal neglect, response competition, and task set to Stroop interference. J. Exp. Psychol. Gen. 132, systems17,18.
47–70 (2003).
28. Daneman, M. & Merikle, P. M. Working memory and language comprehension: A meta-analysis.
Accuracy in the VSTM task declined with increased set size (set
Psychon. Bull. Rev. 3, 422–433 (1996). size 1, 97.7%; set size 2, 94.2%; set size 3, 90.0%; set size 4, 86.2%; set
29. Kyllonen, P. C. & Christal, R. E. Reasoning ability is (little more than) working memory capacity. size 6, 73.3%; set size 8, 68.5%). The number of objects encoded at
Intelligence 14, 398–433 (1990).
each set size, estimated with Cowan’s K formula11, increased up to
30. Vogel, E. K., Luck, S. J. & Shapiro, K. L. Electrophysiological evidence for a postperceptual locus of
suppression during the attentional blink. J. Exp. Psychol. Hum. Percept. Perform. 24, 1656–1674 set size 3 or 4, and levelled off thereafter (Fig. 2; t-test between set
(1998). sizes 4 and 8, P . 0.05). This behavioural function is fitted
significantly better by a quadratic function than by a linear function
Acknowledgements The research reported here was supported by a grant from the US National (P ¼ 0.01)19. Thus, VSTM storage capacity is about three or four
Institute of Mental Health.
items, which is consistent with previous studies11,13. Importantly,
Competing interests statement The authors declare that they have no competing financial
this capacity limit is not due to insufficient time to encode items in
interests. VSTM4. Tripling the sample presentation time from 150 to 450 ms
in a separate experiment did not affect the K function (n ¼ 16,
Correspondence and requests for materials should be addressed to E.K.V. P ¼ 0.28), an observation consistent with previous findings12,13. The
(vogel@[Link]).
VSTM task therefore expresses the capacity limit of VSTM storage
as opposed to a limitation in spatially attending to the display or
encoding items in VSTM.
The brain substrates mediating VSTM’s storage capacity limit
should demonstrate a response profile paralleling the behavioural K
.............................................................. function: activation should increase until set size 3 or 4 and level off
thereafter. To isolate such regions, a voxel-based multiple regression
Capacity limit of visual analysis with K-weighted set size coefficients was performed. The
resulting statistical parametric maps revealed a single bilaterally
short-term memory in symmetric area in the intraparietal and intraoccipital sulci (IPS/
IOS; P , 0.05 corrected). Time-course analysis (Fig. 3a) confirmed
human posterior parietal cortex a strong correlation between the IPS/IOS peak response amplitude
and the number of objects encoded (r ¼ 0.54, P , 0.001; Fig. 2).
J. Jay Todd & René Marois
The peak blood oxygenation level-dependent (BOLD) response
Vanderbilt Vision Research Center, Department of Psychology, Vanderbilt function reached a plateau by set size 4 (t-test between 4 and 8,
University, 530 Wilson Hall, Nashville, Tennessee 37203, USA P , 0.05) and was better described by a quadratic function than by
............................................................................................................................................................................. a linear function (P , 0.01). This parietal activation is not simply
At any instant, our visual system allows us to perceive a rich and related to task difficulty: accuracy decreased and reaction
detailed visual world. Yet our internal, explicit representation of
this visual world is extremely sparse: we can only hold in mind a
minute fraction of the visual scene1,2. These mental represen-
tations are stored in visual short-term memory (VSTM). Even
though VSTM is essential for the execution of a wide array of
perceptual and cognitive functions3–5, and is supported by an
extensive network of brain regions6–9, its storage capacity is
severely limited10–13. With the use of functional magnetic reso-
nance imaging, we show here that this capacity limit is neurally
reflected in one node of this network: activity in the posterior
parietal cortex is tightly correlated with the limited amount of
scene information that can be stored in VSTM. These results
suggest that the posterior parietal cortex is a key neural locus of
our impoverished mental representation of the visual world.
To investigate the neural basis of VSTM’s storage capacity limit,
17 subjects were scanned while performing a parametric load
manipulation 14 of a delayed visual matching-to-sample task
(Fig. 1). On each trial, subjects were briefly presented with a sample
display containing one to eight coloured discs and, after a 1,200-ms
retention interval, decided whether a single probe disc matched
one of the sample discs in location and colour. A 1,200-ms delay Figure 1 Trial design. Each trial began with the auditory presentation of two digits to be
maximizes VSTM’s capacity: with delays shorter than 1 s, VSTM rehearsed throughout the trial. A sample display containing a variable number of
capacity is inflated by sensory (iconic) representations of the dis- coloured discs was then presented for 150 ms, followed by a 1,200-ms retention period,
play15, whereas long delays not only underestimate VSTM capacity and then by a single coloured probe disc. Subjects judged whether the colour of the probe
owing to memory degradation15, but also favour the recruitment of matched the colour of the disc shown at the same position in the sample display.
rehearsal mechanisms and verbal/abstract recoding of the visual Afterwards, two digits appeared and subjects indicated whether these were the same as
material16. To minimize verbal strategies further, a verbal working- those presented at trial onset.
NATURE | VOL 428 | 15 APRIL 2004 | [Link]/nature
©2004 Nature Publishing Group 751