Inmanyreal-worldscenarios,rewardsextrinsic totheagentareextremelysparse,orabsentaltogether. Insuchcases,curiositycanserveas anintrinsicrewardsignaltoenabletheagent toexploreitsenvironmentandlearnskillsthat mightbeusefullaterinitslife.Weformulate curiosi