Learning Layer Paper Reading Club - Week 31 - The Big World Hypothesis

This week's paper:The Big World Hypothesis and its Ramifications for Artificial Intelligence Link: https://openreview.net/pdf?id=Sv7DazuCn8 Abstract: Most of the theory behind reinforcement learning and much of the practice behind deep learning quietly assumes the agent can, in principle, get the world right: enough capacity to represent the state, enough data to converge on the correct value function, enough time to find the optimal policy. Khurram Javed and Richard Sutton argue that for the problems we actually care about, this assumption is false and will stay false. The big world hypothesis says the world is many orders of magnitude larger than the agent. The agent never fully perceives the state, cannot store a correct value for every situation it meets, and has to live with approximate solutions. The paper makes the case that this is not a temporary limit that more compute will erase. As agents get bigger, the environments we point them at get bigger too, and the interesting problems are always the ones at the edge of what the agent can handle. The second half of the paper works through what follows if you accept this. Learning stops being a search for a fixed answer and becomes a matter of continually tracking a world that always looks non-stationary from the agent's point of view. That changes what a good algorithm is, what forgetting means, and how we ought to evaluate systems, since benchmarks with a fixed, learnable optimum measure exactly the regime the hypothesis says does not matter. What is the Learning Layer Labs Paper Reading Club? An initiative from https://www.learninglayer.ai, a lab with the goal of reducing AI anxiety in the world. What is the format? Discussion based. Expect a low pressure environment to share insights and opinions with the group. What are the group goals? Stay on top of AI research and improve understanding of AI fundamentals + math. Who is welcome? Everyone! Try to put in at least some time on the paper and come prepared with questions or things you'd like to discuss, but it's ok to just show up! Learning Layer Labs team: