![]() It’s through metalearning techniques that MNM learns to remember: It learns how to read from and write to memory, as opposed to using hard-coded read/write operations like most existing computational memory mechanisms.įigure 1: Metalearned Neural Memory (MNM) employs a feedforward network for memory and a recurrent neural network controller for writing to and reading from the memory. Methods from these fields can discover update procedures that optimize neural parameters from many fewer examples than standard stochastic gradient descent. Fortunately, recent progress in few-shot learning and metalearning has shown how we might overcome this challenge. Deep networks require abundant data and many steps of gradient descent to learn. We propose a new model, Metalearned Neural Memory (MNM), in which we store data in the parameters of a deep network and use the function defined by that network to recall the data.ĭeep networks-powerful and flexible function approximators capable of generalizing from training data or memorizing it-have seen limited use as memory modules, as writing information into network parameters is slow. In a paper published at the 33rd Conference on Neural Information Processing Systems (NeurIPS), we demonstrate how to use a deep neural network itself as a memory mechanism. In the fields of natural language understanding and processing, for example, memory is crucial for modeling long-term dependencies and building representations of partially observable states. Memory is equally important in deep learning, especially when the goal is to create models with advanced capabilities. Memory allows us to efficiently store the information we encounter and later recall the details we’ve previously read, whether that be moments earlier or weeks, to piece together the full narrative. The ultimate goal is to understand the story, and memory is the reason we’re able to do so. Consider the simple example of reading a book. It grounds us in the current moment, helping us understand where we are and, consequently, what we should do next. Memory is an important part of human intelligence and the human experience.
0 Comments
Leave a Reply. |