Continual learning for real

Continual learning aims at learning neural network models from non-i.i.d. data or for a sequence of tasks. In this project, we plan to lift continual learning from toy setups on small-scale datasets to more realistic settings. In particular, we plan to tackle some open challenges of traditional continual learning methods, including larger scale real world settings, realistic domain shifts, learning from streams of data, and avoiding explicit storage of examples. To this end, we plan to better exploit the knowledge implicitly encoded in the model.