The idea I had today is that some scientific disciplines could benefit from more replication. And what better way to do it than to have Big Brother audit your science and see if they can replicate in their lab what you did in yours. The idea stemmed from my own feelings about my field. I’ve had serious thoughts lately about trying to replicate a couple findings that have had a lot of influence in the field. They’re important findings. They reveal key functions of new neurons that could be relevant for human health. For this reason the whole field is aware of them, cites them, uses them as justification for additional research. Sooo, then why haven’t they been replicated?  

Pretty much since day one (of my science career) it’s been clear that fascinating studies appear and steer the entire field, usually in a good way. However, sometimes they’re such a perfect story that they go unquestioned or untested. And so, in thinking about contemporary examples, I’ve realized that I would really like to replicate them myself. Just to see, to know. Not because I don’t believe them, not to build on them (necessarily), but because I think it’s important to know whether it’s really the case, whether confirming or failing to confirm. I think science is ripe for this, given journals like PLoS ONE, whose aim is not to provide “interesting” science, but valid and useful science. And even if not in a traditional publication, there are new tools like Figshare that allow for datasets of all sizes to be archived, shared and cited with persistent identifiers.

But can you make a living by replicating studies? I thought about how lucky I am that I could pursue a project (mini-project?) whose aim is simply to replicate others’ work. This might not be as easy when I get my own lab. Do such grants exist?

That’s when NIRDY came to mind, the National Institute of Replicating Discoveries Y’all. This is a non-existent governmental research institution (NEGRI) that replicates a sampling of important scientific studies. Instead of only funding research, shouldn’t the government go back and doublecheck? Wouldn’t it be funny if the government couldn’t replicate any of the research they fund? Wouldn’t it be great if they could?

Factors to be considered when creating NIRDY

• How would they pick studies to replicate? Well, they’d be smart, first off. Maybe they could poll scientific communities, see which studies are getting cited the most, discussed the most on social networks, are immediately important for human health. Or go after everything published in Nature or Science.
• What effect would this have on future publications? Would scientists be more careful when publishing exciting results, to ensure validity?
• If NIRDY is ever born, can I get a job? Seriously, some of us get a kick out of systematically manipulating dozens of variables to figure out the exact conditions that are required to observe a scientific phenomenon. This relates to the question of how much replication must occur. One answer might be: until the original finding is replicated. Replication could happen on the first try. If not, maybe additional experiments are worthwhile. This was a problem with the role of neurogenesis in fear conditioning – very confusing until it was recently totally solved (though maybe I should replicate the entire study just to make sure).
• It can be very hard to replicate findings even with relatively standard techniques. Then what do we do about experiments where the technology is so advanced that very few labs are capable of replicating them? Sounds like a job for NIRDY.
• This could also all be good for reagent-sharing. I can think of a number of classic papers that used fancy mice and in the years, nay decades, since I can’t recall ever seeing those mice used by another group or in another paper.

And what happens if a study simply cannot be replicated?

Jail, obviously.

—————————————-

Update: Alex Wiltschko had an interesting point, that replication could be a useful method for training scientists (or driving them batty):

In terms of career logistics, this makes a lot of sense for techs and postdocs, who get to thoroughly learn several state-of-the-art techniques during an experiment replication.

It’s often thought that things change as you move into parts of the brain that represent more complex and abstract concepts. For example, in the hippocampus, many neurons receive the same inputs so it’s generally assumed that different neurons are equally capable of representing a given piece of information. While wiring between neurons must play a role in determining which neurons are activated, the diffuseness of the wiring means that related information need not be stored in spatially neighboring neurons as in the sensory regions of neocortex. Indeed, if you look at hippocampal neurons activated by a given experience they don’t appear to have any particular spatial arrangement but are randomly distributed, anatomically. Alternatively, it could be that certain hippocampal neurons are hard-wired to respond to specific stimuli, it’s just that we don’t understand the wiring.

I’ve mentioned before (here and here) how anatomical patterns of activity in the hippocampus are not always so random – in the dentate gyrus the same neurons are often repeatedly activated and by very different experiences. Furthermore, half of the dentate gyrus (the infrapyramidal blade) never seems to be noticeably active, period. But anatomical biases have been reported outside of the dentate gyrus too. Hampson and Deadwyler showed that spatial and nonspatial information is segregated in distinct septotemporal regions of CA1/CA3. Also, Nakamura et al. have suggested that CA1 neurons that represent a given spatial environment are more likely to be spatially clustered together.

While these studies suggest there may be a hard-wired anatomical pattern by which information is represented in regions such as the hippocampus, we really have have no idea how that pattern might be established. I was therefore intrigued to see a couple papers shed new light on this issue. One is a recent paper by Yassin et al. who used a Fos-GFP mouse to identify and record from neurons recently activated by behavioral experience. Fos is an immediate-early gene that is upregulated in neurons that are involved in learning and so, in this mouse, those neurons fluoresced green and could be examined electrophysiologically. They found that the Fos-GFP neurons fired at higher rates than neighboring neurons that were not expressing GFP and that they tended to be more connected to one another (and thus they were dubbed Facebook neurons), suggesting that there may be a subset of neurons that is preselected to be involved in representing experiences (perhaps not unlike the population of highly-active dentate gyrus neurons). There is a bit of a chicken and egg problem here, because we don’t know if the GFP+ neurons always fire at higher rates (and are hard-wired to be more involved in representing experience) or if they only fire at higher rates because they were recently activated (i.e. behavior-induced plasticity changed them). Intriguing nonetheless and a good approach for future studies I think.

The other study is pretty revolutionary I think and also has to do with predetermined, hard-wired patterns of neuronal activity. One of the exciting developments of the last 15 years has been the finding that patterns of neuronal activity are replayed during sleep. It is thought that this “replay” is the physiological correlate of memory consolidation, i.e. the rehearsal of recent experience and integration of that new information into the brain’s circuitry. Now, Dragoi and Tonegawa have found that the patterns of neuronal activity, seen as a mouse explores a novel environment, can also be seen during rest/sleep episodes before the mouse has ever been in that environment. Essentially, they discovered that the brain has created a representation (or at least a fraction) of an experience that has not even happened yet. They call the phenomenon “preplay”.

The preplay phenomenon does fit with previous data. The Mosers, in their News and Views piece on this study, note that “…place cells continue to fire in regular sequences when an animal’s position is fixed, for example, when a rat is running in a wheel. Moreover, rat pups exploring an open space for the first time show adult-like place cell sequences, which indicates that path sequences are hard-wired in the synaptic connection matrix by either genetic programs or early experience.” Also relevant is the finding from John Guzowski’s lab showing that very brief experiences (perhaps too brief to be even remembered) are capable of inducing transcription of the plasticity-related gene, Arc, in a full complement of CA3 neurons. In contrast, CA1 neurons were only fully activated after multiple experiences over multiple days, suggesting less of a role for hard-wiring and more of a role for plasticity and learning in shaping neural representations in this region.

Why preplay? One interesting hypothesis is that the hippocampus is needed to imagine the future (a reasonable role for a structure responsible for remembering the past). Could preplay be an attempt to predict future experience? Or might a shared pattern of activity simply be a way to bind together two events and create a coherent history? Don’t worry – I’m sure that, as we speak, there are rodents with implanted electrode arrays running around, working hard, to give us the answer.

Yassin L, Benedetti BL, Jouhanneau JS, Wen JA, Poulet JF, & Barth AL (2010). An embedded subnetwork of highly active neurons in the neocortex. Neuron, 68 (6), 1043-50 PMID: 21172607
Dragoi G, & Tonegawa S (2011). Preplay of future place cell sequences by hippocampal cellular assemblies. Nature, 469 (7330), 397-401 PMID: 21179088

A fundamental property of the hippocampus is its ability to rapidly encode memories while simultaneously keeping them distinct. Recording from hippocampal neurons one can clearly see that different populations of neurons are active as a rat explores two environments. This is thought to be one mechanism by which information is kept distinct in the brain.

For the last 15-20 years it has been thought that the dentate gyrus (DG), a major subfield of the hippocampus, serves to take small changes in incoming sensory information and orthogonalize them (i.e. make them more different). This idea was built in part on the fact that there are many more DG neurons than upstream cortical neurons. Thus, the DG could use completely different populations of neurons to represent different sets of incoming information and then pass on these representations to CA3, which may bind them into coherent events/memories (the interconnectedness of CA3 neurons, via “recurrent collatorals”, is thought to be a mechanism by which the different components of a memory are bound together).

However, a “problem” arose when Leutgeb et al. found that it is always the same population of dentate granule neurons (~1% of the total population) that are active as an animal explores different environments, even very different ones. This was a bit of a surprise. Still consistent with the proposed role of the DG in orthogonalizing information, however, was the fact that the DG neurons fired (i.e. generated action potentials, which transmit information from neuron to neuron) at different rates/frequencies in the different environments. Thus, changes in sensory information were represented by changes in patterns of activity within the same population of cells, not by recruiting different populations of cells. This is but one study – the question of how the DG encodes and extracts information is far from settled (e.g. what are the other 99% of granule neurons doing? Surely there is a situation in which they are active, no?). But the findings were robust and raise many questions, namely: How does the same population of DG neurons activate different populations of downstream CA3 neurons, during different experiences?

Until now I had been in denial, fixated on trying to understand what types of behavioral experiences might activate different populations of dentate gyrus neurons. But maybe now it’s time to face the data.

The consensus, both in vitro (e.g. here and here) and in vivo (here), seems to be that if DG neurons are sufficiently active they can reliably activate CA3 neurons. Can a single population of DG neurons account for the amount of CA3 activity seen in the behaving animal? Well, 1% activation of the total DG population (1 million neurons) is 10 000 DG neurons. Each DG neuron contacts about 10 CA3 neurons. So if all active DG neurons activated all their downstream targets, then you’d expect about 100 000 active CA3 neurons – a third of the population. Indeed, about 20% of CA3 neurons are active when a rat explores a novel environment. So it’s possible. But it’s probably unlikely.

One reason it’s unlikely is that it doesn’t explain how different populations of CA3 neurons are activated by different experiences if it is the same population of DG neurons that are always driving them. In other words, since DG neurons are relatively hard-wired to CA3 neurons, how could a given DG neuron activate a CA3 neuron under some conditions and not others? One answer is that maybe it doesn’t – quite a while ago, McNaughton et al. showed that, even when the DG is lesioned, CA3 neurons are still able to selectively encode spatial locations as a rat traverses the environment, probably due to direct inputs from the cortex. And so perhaps the primary function of the DG is not to selectively activate different CA3 populations. However, the DG could certainly shape activity within CA3 or insert unique information into the CA3 network. How?

One possible mechanism, which may be dead obvious to electrophysiologists, is frequency itself. Leutgeb et al. found that frequency of activity is how DG neurons encode information and so frequency of activity may also be the way DG neurons transmit information to CA3 during different experiences.

It has been known for some time now that the output of DG neurons, the mossy fiber axons, show extraordinary frequency-dependent synaptic facilitation. Basically, as a DG neuron fires more action potentials over shorter periods of time, the amount of neurotransmitter it releases onto CA3 neurons increases (thereby increasing the likelihood a CA3 neuron will in turn fire action potentials and be recruited to participate in memory encoding). This means that at low firing rates, a DG neuron will activate some CA3 neurons and, at higher firing rates, it will recruit different or at least additional CA3 neurons.

Wouldn’t this cause a problem where, as DG firing rates increase, it is not different populations of CA3 neurons that become activated, but more populations? Well, it is known that some DG neurons increase their activity, and others decrease their activity, as an animal has different experiences, so the net activity in CA3 could remain constant, while still activating different CA3 populations. However, the DG-CA3 circuitry is certainly complicated enough to allow for other mechanisms. For example, while the dentate gyrus projects to CA3, and it is connections between these hippocampal regions that are thought to encode memories, DG neurons actually contact more inhibitory interneurons than CA3 neurons. Furthermore, there is a wide variety of synaptic connections between DG neurons and interneurons and these connections can be made weaker or stronger in a state- and frequency-dependent manner. Suffice it to say, by firing at different frequencies, it is plausible that a given DG neuron could activate different populations of interneurons, which in turn could inhibit different populations of downstream CA3 neurons, making them less likely to participate in memory encoding.

This ties in loosely to a peculiarity of the dentate gyrus that, until now, has just been a source of pretty histological images (to me) – the variability of calbindin expression in dentate gyrus neurons. Calbindin is a protein that binds calcium, it acts as a buffer, and gives DG neurons their property of facilitation (briefly: A single action potential in a DG neuron will travel down the axon and trigger the opening of calcium channels in the synaptic terminal at a CA3 neuron. Calcium is necessary for neurotransmitter release and subsequent activation of the CA3 neuron. Calbindin will bind this small amount of calcium, thereby preventing neurotransmitter release and CA3 activation. However, as the number and frequency of action potentials increases, calbindin will fail to effectively “mop up” the extra calcium and neurotransmission will proceed.). If you look at the picture at the top of this post, you can see that the amount of calbindin varies greatly in DG neurons. Immature DG neurons, identified by PSA-NCAM expression, are devoid of calbindin (arrows point to clear examples) and even when they are quite mature (10w of age) 40% will still be devoid of calbindin (see my data in this montage). Lastly, calbindin expression can be modified by experience. So the variable and modifiable expression of calbindin might be yet another mechanism by which DG neurons are capable of shaping activity in CA3 neurons. Indeed, at least one study, from Robert Sapolsky’s lab, has shown that genetically altering calbindin expression in the dentate gyrus dramatically influences DG-CA3 physiology and impairs memory.

Thanks to A.P. for posing the question.

Reference

Leutgeb JK, Leutgeb S, Moser MB, & Moser EI (2007). Pattern separation in the dentate gyrus and CA3 of the hippocampus. Science, 315 (5814), 961-6 PMID: 17303747