Gittins Index §

Date created: 2021-10-19

It’s a way to optimise your choices in an Explore Exploit situation. By taking into account Discounting, the lowered value of something if it occurs in the future rather than right now, he was able to develop the solutions beyond the Win-stay, Lose-shift heuristic.

One quirk in the Gittins Index is that it puts an inherent value on exploration, so new experiences are rarely valued at the expected value 0.5 but rather much higher. From 0.7 with a 10% discounting, up to 87% with 1% discounting.

Exploration in itself has value, since trying new things increases our chances of finding the best. So taking the future into account, rather than focusing just on the present, drives us toward novelty.

---

References §

• Link to website, bibtex from Zotero or note with book/blog/etc summary