Title Page

Introduction

Introduction

Comparing behavior across individuals

Introduction

Goal

We want to evaluate how similar is the Spontaneous Brain Activity across individuals

  1. record Spontaneous Brain Activity → multiple individuals + whole-brain + neuron-scale
  2. find a shared representation of this activity across fish
  3. build of vocabulary of brain states
  4. compare the structure of the sequence of states

Introduction

comparing brain activity has been done for a long time
recording=fRMI + stimuli or task → link activity with stim/task → maps identifying the functions of brain regions → compare maps across individuals using registration
brains are different but share large scale functional organization

Introduction

Even if we can find similar large brain structure that are specialized at doing different things, why do we expect brain activity to be the same. The brain controles the behavior, and it's easier to understand what an animal does than how an animal thinks.
My favorite example : how flies clean themselfs. Behavior = stereotyped actions (vocabulary) + stereotyped structure of sequence
you can find a shared representation of the behavior

Introduction

this is a very simple example but you can generalize it to much more complicated behavior.
For example human movements. Infinite possibilities but in practice we can represent most behaviors with just a few joints (250 dof → 30 dof). We can simplify this even more : not every position equiprobable (forbiden + unlikely) + transitions
if we can find a shared representation space for behavior, and behavior is controled by the brain → we should be able to find a simplified reprentation of brain activity which explains the behavior.

Introduction

spont def + problem of comparison with SA + finding networks of SA shared across humans (ICA diag)
problem : not at scale of neurons

Introduction

doing whole brain analysis in small animals where neurons can be identified
C. elegens : same neurons can be identified in multiple individuals. maps to latent space representation of behavior.
Very simple brain → we would like to study vertebrates

Introduction

Zebrafish

Adult
Larva
  • Rich behavior
  • Transparent
  • Small brain (~100k neurons)
  • Genetic tools (GCaMP, ...)

Introduction

Light Sheet Microscopy

Light Sheet

Introduction

Light Sheet Microscopy

Restricted Boltzmann Machines

Theory : A Probabilistic Model

Geoffrey Hinton

visible configuration

$$\mathbf v=(v_1,\dots,v_N)\in\{\text{on},\text{off}\}^N$$

hidden configuration

$$\mathbf h=(h_1,\dots,h_M)\in\mathbb{R}^M$$
$$ \begin{aligned} P(\mathbf v,\mathbf h) &= \frac{1}{Z}\, e^{-E(\mathbf v,\mathbf h)}\\ E(\mathbf v,\mathbf h) &= -\sum_{i=1}^{N}g_i v_i +\sum_{\mu=1}^{M}\mathcal{U}_\mu(h_\mu) -\sum_{i,\mu} w_{i\mu} v_i h_\mu \end{aligned} $$
parameters $ \begin{cases} &g_i\\ &\mathcal{U}_\mu\\ &w_{i\mu} \end{cases} $

Restricted Boltzmann Machines

Theory : A Generative Model

same statistics ⇒ $\langle\mathbf{v}\rangle$, $\langle\mathbf{v}\mathbf{v}\rangle$, $\langle\mathbf{v}\mathbf{h}\rangle$, $\langle\mathbf{h}\rangle$, $\langle\mathbf{h}\mathbf{h}\rangle$
maximize $\mathcal{L} = \langle \log P(\mathbf{v}) \rangle_\text{data}$
learn parameters $ \begin{cases} &g_i\\ &\mathcal{U}_\mu\\ &w_{i\mu} \end{cases} $

Restricted Boltzmann Machines

from zebrafish larvae

Thijs van der Plas

Jérôme Tubiana

Restricted Boltzmann Machines

Zebrafish Brain as a Composition of Cell Assemblies

Thijs van der Plas

Jérôme Tubiana

cell assembly = group of co-activating neurons, usually spatially localized

Restricted Boltzmann Machines

Zebrafish Brain as a Composition of Cell Assemblies

Thijs van der Plas

Jérôme Tubiana

Restricted Boltzmann Machines

provide Degenerate Representations

Goal : Find a Common Representation for the Spontaneous Brain Activity of multiple fish.

  • Hidden Units ≈ known Functional Networks
  • Functional Networks are shared by all individuals

⠀

⇒ multiple RBMs shoud provide the same representation

RBMs are stochastic ⇒ variable solutions

Restricted Boltzmann Machines

Summary

Goal : Find a Common Representation for the Spontaneous Brain Activity of multiple fish.

Spontaneous Brain Activity ⇒ no alignment

RBMs representation ⇒ composition of cell assemblies

RBMs representation ⇒ degenerate

⇒ We needs a method which explicitely aligns the latent space across individuals

Aligning Latent Space across Individuals

Two Methods

Neuronal Models With Shared Hidden Space

At the single neuron scale ?

⠀

Problem : neurons are not comparable across individuals

Hypothesis : cell assemblies are shared across individuals

Multi-fish Neuronal Model

Latent-aligned RBMs : Method

Multi-fish Neuronal Model

Latent-aligned RBMs : Method

Jorge Fernández

⠀

de Cossío Díaz

  • Converge faster (1/10th)
  • Converge more reliably

Multi-fish Neuronal Model

Hidden Units map to Stereotypic Neuronal Populations

Multi-fish Neuronal Model

Hidden Units describe a Shared Space

Multi-fish Neuronal Model

Translating Activity from one Fish to Another

  • Shared latent space across multiple individuals
  • Generative Model

⇒ Translate configuration through the latent space

Multi-fish Neuronal Model

Translating Activity from one Fish to Another

Multi-fish Neuronal Model

Translated Activity is Probable under the Recipient Model

Multi-fish Neuronal Model

Summary

  • statistical model of whole-brain spontaneous neuronal activity → RBMs
  • RBMs representation = composition of cell assemblies
  • align the space of representation across multiple fish
  • translate activity from one brain to another

only static

⠀

description

What about the dynamics?

Dynamics of Spontaneous Activity

Shared space of Representation

⠀

Dynamics of Spontaneous Activity

Segmenting the hidden space into a vocabulary of brain states

Dynamics of Spontaneous Activity

State sequence decoding from Neuronal Data

$$P(s\mid\mathbf{v}_t) = \mathbb{E}\Big[ P\big( s\mid P(\mathbf{h}\mid\mathbf{v}_t) \big) \Big]$$

Dynamics of Spontaneous Activity

States Capture Large Scale Feature of Brain Activity

Dynamics of Spontaneous Activity

Across Individuals ?

  • Are states stereoypical ?
  • Are states used similarly ?
  • Are states sequences stereotypical ?

Dynamics of Spontaneous Activity

States represent Stereotyped Brain States

Dynamics of Spontaneous Activity

States are Used Similarly by All Fish

$$P(s) = \frac{1}{T}\sum_{t=1}^T P(s\mid\mathbf{v}_t)$$

Dynamics of Spontaneous Activity

Markovian Transition Rates are partially Conserved across fish

$$P(s\to s') = \frac{ \sum_{t=1}^{T-1} P(s\mid\mathbf{v}_t) \cdot P(s'\mid\mathbf{v}_{t+1}) }{ \sum_{t=1}^{T-1} P(s\mid\mathbf{v}_t) }$$

Dynamics of Spontaneous Activity

Markovian Dynamics - sub-structure and hub-states

Dynamics of Spontaneous Activity

Markovian Dynamics - sub-structure and hub-states

Conclusion & Perspectives

Goal : Is Spontaneous Brain Activity stereotypical across individuals?

  • Spontaneous Activity is challenging to compare between individuals
  • RBMs provide a representation of whole brain activity as a composition of cell assemblies
  • Method : align this representation across multiple zebrafish larvae
  • Method : define shared brain states and compare their temporal organization
  • Spontaneous Brain Dynamics is partially conserved across individuals
  • Joint training of RBMs (instead of Teacher/Student)
  • Integrate a temporal model directly into the RBM (eg. HMM)
  • Joint neuronal and behavioral studies to improve model interpretation

Shared Behavioral and Neuronal Representation

Zebrafish Reorientation

Ethics

SciComm

Thanks

Supplementary

Section Title

Smaller subtitle 1

$$\mathcal{U}(x) = \frac{1}{2}\gamma_+ x_+^2 + \frac{1}{2}\gamma_- x_-^2 + \theta_+x_+ + \theta_-x_-$$

Section Title

Smaller subtitle 2

hey hey heey
introduce contrastive loss → intuition first
The model fails when negatives are too easy
segue: hardness-aware sampling

Section Title

Smaller subtitle 3

Anything goes here: text, images, charts…