recording site. Then, we computed a peri-ripple
spectrogram during the verbal recall periods
(similar to the analysis in Fig. 5, B and C). We
separated the activations that occurred when
patients recalled the electrode’s preferred im-
ages from those that occurred when the patients
recalled the nonpreferred ones. We then aver-
aged the HFB power over a [–300, 300] ms time
window centered on SWR onset and quantified
the difference between preferred and nonpre-
ferred images using the bias-corrected Hedges’
geffect size measure ( 52 ), individually in each
site. Recording sites with fewer than five peri-
ripple responses in either the preferred or non-
preferred group were excluded. Content-selective
reactivation of visual information during SWR
events occurred most prominently in the fusi-
form gyrus as well as in downstream cortical
regions including the entorhinal and perirhinal
cortices. We used the anatomical atlases ( 53 – 55 )
included in FreeSurfer to subdivide the cortical
surface into six partly overlapping regions along
the visual hierarchy ( 56 ). Electrodes falling with-
in these regions were grouped together. The peri-
ripple reactivation was strongest in the fusiform
gyrus and entorhinal cortex (i.e., the higher levels
of the ventral visual hierarchy) (Fig. 5, E and F).
Memory reinstatement during SWRs
To further investigate the reinstatement of visual
information during recall in relation to SWRs,
we pooled together visually responsive cortical
sites that showed significant content selectivity
during picture viewing (see methods; see fig. S9
for electrodes’location) and constructed a multi-
site HFB activation pattern per each item during
picture viewing and recall. We used principal
components analysis (PCA) to reduce the di-
mensionality of the activation patterns during
viewing, retaining the first 11 principal compo-
nents (PCs) that accounted for 83.8% of the var-
iation across viewed items (see methods; see
Fig. 6, A and B, for PC visualization). We then
applied the same linear transformation to the
patterns that emerged during recall, bringing
all patterns to the same 11-dimensional linear
space. Next, we quantified the similarity between
viewing and recall patterns. We computed the
Pearson correlation using a 50-ms sliding win-
dow to examine how the similarity between
patterns changed in relation to SWR timing
during recall, and in relation to the onset/offset
of the picture during viewing (Fig. 6, C and D).
This analysis revealed a significant enhance-
ment in pattern correlation during the SWR
event (P< 0.05, nonparametric cluster-based
permutation test; see methods). Intriguingly,
there was a trend toward a second peak shortly
after the disappearance of the picture; however,
this peak did not survive the cluster-based per-
mutation test.
We then asked whether we could decode the
identity of recalled items on the basis of the
SWR-triggered cortical HFB patterns. To perform
this analysis, we trained ak-nearest neighbors
(k-NN) classifier on single-trial HFB patterns
during viewing and tested its classification per-
formance on the free-recall patterns (i.e., cross-
classification; see methods). Here again, the
patterns’dimensionality was reduced using an
out-of-sample extension of the same PCA trans-
formation described above (Fig. 6, A and B).
Testing the classifier performance on the view-
ing data showed 100% accuracy in decoding
the image category (usingk= 9 NN), and 47.3%
accuracy (chance level is 3.5%) in decoding
exemplar identity (usingk=1NN).Forthecross-
classification analysis, we used a 50-ms sliding
window to examine the temporal profile of visual
reinstatement during recall relative to the SWR
onset (Fig. 6, E and F). We obtained significant
decoding performance during recall for both cat-
egory (82.1% accuracy, 23/28 items) and exem-
plar identity (21.4% accuracy, 6/28 items) (P<
0.01; a nonparametric cluster-based permuta-
tion test, shuffling item labels 2000 times; see
methods). Decoding performance peaked together
with the SWR event, suggesting a temporally
precise coupling between hippocampal SWRs
and cortical activity during reinstatement of
visual information.Discussion
We used a rare clinical opportunity to measure
hippocampal SWRs and the associated SWR-
triggered cortical activity in human patients as
they memorized and freely recalled vivid photo-
graphs of famous faces and places. Our results
highlight three major new aspects of SWRs’
function and their relation to human episodic
memory. First, a transient increase in hippo-
campal SWR rate preceded the onset of verbally
reported recollections by 1 to 2 s. This increase
wascontent-selective and reexpressed the same
picture preferences observed during the encod-
ing phase. Second, the SWR rate during picture
viewing predicted subsequent memory perform-
ance of individual patients. Finally, during the
verbal report of recall, high-order cortical visual
sites showed a SWR-triggered increase in HFB
activity. Again, this broadband activation was
content-specific and occurred only when the pa-
tients recalled the pictures that preferentially
activated the sites during viewing.
The anticipatory increase in SWR rate during
recall (Fig. 2C) strongly suggests that SWRs
play an important role in the initiation of self-
generated recall events. Work in rodents explor-
ing the link between awake SWRs and putative
memory retrieval behaviors has demonstrated
that sequences of hippocampal place cell assem-
blies, representing spatial and contextual in-
formation related to past experiences, are
briefly replayed in the time window of the SWR
( 32 , 34 , 35 , 41 , 57 ). However, it was not possible
to determine in these studies the actual mo-
ment of cognitive recall, and hence its temporal
relationship to SWR events. Conducting the ex-
periment in awake human patients enabled us
to extend these previous studies by obtaining an
estimation of when each recalled item surfaced
into the patients’conscious awareness. This al-
lowed us to establish the anticipatory nature of
the SWR event.It may be argued that this anticipatory in-
crease could result from inaccuracies in the
timing of verbal reports. However, because the
SWR rate increase was transient and clearly de-
clined at the time of the verbal report proper,
such an onset“blurring”effect is unlikely. These
results are intriguingly similar to the anticipa-
tory increases in hippocampal and medial tem-
poral neurons’firing rate observed in single-unit
recordings in patients during a similar free-recall
paradigm ( 42 ). These anticipatory hippocampal
signals are compatible with a two-stage recol-
lection process mediated by the hippocampus: a
fast subconscious stage, involving reactivation
of hippocampal-neocortical memory traces, and
a slower conscious one, involving cortical pro-
cesses that operate on the retrieved content and
reinstate the mentally experienced episode ( 58 ).
However, we cannot at this point rule out the
possibility that patients thought about the re-
called items prior to their verbal responses, hence
contributing to the anticipatory activation.
The increase in SWR rate prior to recall onset
showed visual content selectivity: Specific images
that generated a higher SWR rate during the
picture-viewing stage also elicited a higher SWR
rate during recollection. In other words, the
SWR rate during recall reexpressed the con-
tent specificity found during viewing. Thus, the
phenomenon of memory reactivation is evident
not only in the spike content of the SWR (e.g.,
content-specific sequences of hippocampal place
cells) ( 32 – 35 , 57 , 59 )butalsointherateofSWRs
elicited during recall, which is linked to the rate
of SWRs elicited during the original experience.
However, it should be noted that experiences
that are encoded for the first time are likely to
engage a different set of memory processes (e.g.,
novelty detection, engram formation, etc.) that
do not repeat during recall. In line with this, the
link between SWR rates during the original ex-
perience and subsequent recall was found only
for the repeated item presentations (fig. S4).
Our results show that SWRs play an impor-
tant role in the encoding process as well. We
found that the ability of patients to successfully
recall a visual item was significantly linked to
SWR activity during picture viewing (memory-
encoding stage). This effect was observed only
during the first presentation of each image and
was maximal during the postpresentation pe-
riod, thereby corroborating previous studies
pointing to the importance of the poststimulus
periods in memory encoding ( 60 ). Specifically,
we found that the strength of the differential
signal during encoding (i.e., the difference in
SWR rate after presentation of recalled and
forgotten items) predicted the success of pa-
tients in subsequently recalling these items. A
plausible interpretation of this effect is that such
differential signal during viewing may capture
the process of memory trace formation, so that
the larger the differential activity, the stronger the
engagement of the hippocampus in the en-
coding process, which enhances the ability of the
patient to freely recall these memories later
on. Regardless of the precise mechanism, thisNormanet al.,Science 365 , eaax1030 (2019) 16 August 2019 8of14
RESEARCH | RESEARCH ARTICLE