PGI-15 Tutorial

Learning in RNNs and SNNs

02.11.2023 | Emre Neftci, Susanne Kunkel, Jamie Lohoff, and Willem Wybo

Spike-Timing Dependent Plasticity

Bi and Poo 1998

Jesper Sjostrom and Wulfram Gerstner (2010), Scholarpedia, 5(2):1362.

$W$: Learning Window
$t_i^n$: $n$th spike time of post-synaptic neuron $i$
$t_j^f$: $f$th spike time of pre-synaptic neuron $i$

STDP Visualized

STDP lacks credit assignment

STDP updates only depend on pre-synaptic and post-synaptic activity
No direct influence by task performance

<h2>Generalized Hebbian Learning</h2>
  Generalized Hebbian learning: Introduce dependence on pre-synaptic and post-synaptic activities, and the weight itself:

$$
\begin{split}
  \frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) &= F(w_{ij}, \nu_i, \nu_j)\\\\
  \frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) &= a_0(w_{ij}) + a_1^{pre}(w_{ij})\nu_j +  a_1^{post}(w_{ij})\nu_i + a_2(w_{ij})\nu_i \nu_j + \dots \\\\
\end{split}
$$

<center>
  <table>
    <tr>
      <td>Pre ($\nu_j$)</td> 
      <td>+</td> 
      <td>0</td>
      <td>+</td>
      <td>0</td>
    </tr>
    <tr>
      <td>Post ($\nu_i$) </td>
      <td>+</td> 
      <td>+</td> 
      <td>0</td> 
      <td>0</td>
    </tr>
    <tr class=fragment >
      <td>$\frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) \propto \nu_i \nu_j$</td><td>+   </td><td> 0 </td><td> 0 </td><td> 0 </td>

</tr>
    <tr class=fragment >
      <td>$\frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) \propto \nu_i \nu_j - c$</td><td>+   </td><td> - </td><td> - </td><td> - </td>
    </tr>
    <tr class=fragment >
      <td> $\frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) \propto (\nu_i - c)  \nu_j$ </td><td>$\pm$ </td><td> 0 </td><td> - </td><td> 0 </td>
    </tr>
    <tr class=fragment >
      <td>$\frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) \propto \left(\nu_{i}-\langle\nu_{i} \rangle\right)\,\left(\nu_{j}-\langle\nu_{j}\rangle\right)$</td><td>+   </td><td> - </td><td> - </td><td> + </td>
    </tr>
    <tr class=fragment >
      <td>$\frac{\mathrm{d}}{\mathrm{d} t} w_{ij}(t) \propto \left(\nu_{j}-w_{ij}\nu_{i}\right)\nu_i$</td><td>$\pm$   </td><td> - </td><td> 0 </td><td> 0 </td>
    </tr>
  </table>
  </center>
  <p class=ref>Gerstner and Kistler, 2002</p>

The Concept of Locality

For computation to occur on a physical substrate, information much be spatially and temporally local.

Neftci et al. 2019

Considerations about STDP

Rate-based models are consistent with STDP
Spike-time dependence depends on synapse location wrt soma
The exponential fit of STDP is for computational convenience
Update in original model is relative
STDP is not derived from computational requirements

STDP is a measurement, not an accurate mechanistic model!

Discrete Euler Integration

$$ \begin{align*} U^{t+1} & = \beta U^t + (1-\beta) W S^t_{in} - S^t \tag{Membrane Potential}\\ S^t &= \Theta(U^t-1) \tag{Spike & Reset} \\ \beta & = \exp(\frac{t}{\tau_{mem}}) \tag{Time Constant}\\ \end{align*} $$

Limits of Leaky I&F: Towards More Complex Neurons

In many tasks, SNNs consisting of Leaky I\&F do not perfiorm better than ANNs, and are more expensive to implement
Just as with ANNs, we need to use more complex neuron models to improve performance (Vanilla RNN $\rightarrow$ LSTM)

Spiking LSTM, Adaptive Thresholds, Dendritic Compartments, etc.

Bellec et al. 2018

Vecoven, Ernst, Drion, 2021

Limits of Leaky I&F: Towards More Complex Neurons

In many tasks, SNNs consisting of Leaky I\&F do not perfiorm better than ANNs, and are more expensive to implement
Just as with ANNs, we need to use more complex neuron models to improve performance (Vanilla RNN $\rightarrow$ LSTM)

Synaptic delays, graded spikes

Kretzberg et al. 2001

PGI-15 Tutorial

Learning in RNNs and SNNs

Spike-Timing Dependent Plasticity

STDP Visualized

STDP lacks credit assignment

The Concept of Locality

Considerations about STDP

Discrete Euler Integration

Limits of Leaky I&F: Towards More Complex Neurons

Limits of Leaky I&F: Towards More Complex Neurons

The End