Will it be possible to watch a film of our dreams?

Dreams are complex mental experiences that occur during sleep and are believed to be related to the brain's processing and consolidation of memories and emotions. New technology could make it possible to decode brain activity. This would possible imply that we could one day watch a movie of our dreams.

A recent study, publised in Nature (07 January 2022), introduced a novel experimental paradigm for neural decoding, using synthesized yet hyperrealistic stimuli, and developed the HYPER model as an implementation of this paradigm. The model was able to decode fMRI recordings into the best reconstructions of perceived face images to date using a generative adversarial network (GAN) that synthesizes photorealistic faces from latent vectors. The results suggest that unsupervised deep neural networks can effectively model neural representations of naturalistic stimuli and that the GAN latent space approximates the neural face manifold. However, the reconstructions produced by the model contain biases, such as a preference for young, western-looking faces without eyeglasses, and may be affected by the problem of feature entanglement, where manipulating one feature in latent space affects other features as well. The study suggests that using a modified version of the GAN called StyleGAN, which addresses the problem of feature entanglement, could be a potential next step for improving the model's ability to reconstruct unfamiliar features. The study also highlights the importance of using a large number of participants in future studies to better understand the neural basis of face perception. While the results of the study are promising, the authors caution that this does not necessarily mean it will be possible to eventually watch a film of our dreams.

Dado, T., Güçlütürk, Y., Ambrogioni, L. et al. Hyperrealistic neural decoding for reconstructing faces from fMRI activations via the GAN latent space. Sci Rep 12, 141 (2022). https://doi.org/10.1038/s41598-021-03938-w

Quiz:

1. What is the goal of neural decoding?
   A. To map brain responses to sensory stimuli via a feature space
   B. To map sensory stimuli to brain responses via a feature space
   C. To map brain responses to sensory stimuli without using a feature space
   D. To map sensory stimuli to brain responses without using a feature space

2. What is the main advantage of using a feature space in neural decoding?
   A. It allows for the testing of alternative hypotheses about the nature of neural representations
   B. It makes the direct stimulus-response transformation more data efficient
   C. It allows for the reconstruction of natural images, movies, and faces
   D. It makes the direct stimulus-response transformation more complex

3. How is the stimulus-feature mapping modeled in neural decoding?
   A. By a linear transformation
   B. By a nonlinear transformation
   C. By a composite function of linear and nonlinear transformations
   D. By a composite function of nonlinear and linear transformations

4. How is the feature-response mapping modeled in neural decoding?
   A. By a nonlinear transformation
   B. By a linear transformation
   C. By a composite function of linear and nonlinear transformations
   D. By a composite function of nonlinear and linear transformations

5. What is the relationship between feature representations of stimuli and neuroimaging measurements in neural decoding?
   A. They have a linear relationship
   B. They have a nonlinear relationship
   C. They have no relationship
   D. They have a composite relationship

6. How have supervised deep neural networks been used in neural decoding of visual perception?
   A. To classify perceived, imagined, and dreamed object categories
   B. To reconstruct perceived natural images, movies, and faces
   C. To model neural representations
   D. To generate latent vectors

7. How have unsupervised deep neural networks fared in modeling neural representations?
   A. They have been more successful than supervised deep neural networks
   B. They have been less successful than supervised deep neural networks
   C. They have been equally successful as supervised deep neural networks
   D. Their success has not been established

8. What is a generative adversarial network (GAN)?
   A. A type of supervised deep neural network
   B. A type of unsupervised deep neural network
   C. A type of network that pits a generator network against a discriminator network
   D. A type of network that maps latent vector samples to unique data samples

9. How does a GAN work?
   A. The generator network maps latent vector samples to unique data samples that appear to have been drawn from the real data distribution
   B. The discriminator network maps latent vector samples to unique data samples that appear to have been drawn from the real data distribution
   C. The generator network maps real data samples to latent vector samples
   D. The discriminator network maps real data samples to latent vector samples

10. What is the main advantage of using GANs in neural decoding?
    A. They can bring neural decoding to the next level
    B. They can model neural representations more effectively than supervised deep neural networks
    C. They can reconstruct natural images, movies, and faces more accurately
    D. They can classify perceived, imagined, and dreamed object categories more accurately
    
    

Answers:

1. A. To map brain responses to sensory stimuli via a feature space
2. A. It allows for the testing of alternative hypotheses about the nature of neural representations
3. B. By a nonlinear transformation
4. B. By a linear transformation
5. A. They have a linear relationship
6. A. To classify perceived, imagined, and dreamed object categories
7. B. They have been less successful than supervised deep neural networks
8. C. A type of network that pits a generator network against a discriminator network
9. A. The generator network maps latent vector samples to unique data samples that appear to have been drawn from the real data distribution
10. A. They can bring neural decoding to the next level