BEGIN:VCALENDAR
VERSION:2.0
PRODID:-//UC Irvine Donald Bren School of Information & Computer Sciences - ECPv6.3.4//NONSGML v1.0//EN
CALSCALE:GREGORIAN
METHOD:PUBLISH
X-WR-CALNAME:UC Irvine Donald Bren School of Information & Computer Sciences
X-ORIGINAL-URL:https://ics.uci.edu
X-WR-CALDESC:Events for UC Irvine Donald Bren School of Information & Computer Sciences
REFRESH-INTERVAL;VALUE=DURATION:PT1H
X-Robots-Tag:noindex
X-PUBLISHED-TTL:PT1H
BEGIN:VTIMEZONE
TZID:America/Los_Angeles
BEGIN:DAYLIGHT
TZOFFSETFROM:-0800
TZOFFSETTO:-0700
TZNAME:PDT
DTSTART:20240310T100000
END:DAYLIGHT
BEGIN:STANDARD
TZOFFSETFROM:-0700
TZOFFSETTO:-0800
TZNAME:PST
DTSTART:20241103T090000
END:STANDARD
END:VTIMEZONE
BEGIN:VEVENT
DTSTART;TZID=America/Los_Angeles:20241017T160000
DTEND;TZID=America/Los_Angeles:20241017T170000
DTSTAMP:20260711T102641
CREATED:20241008T151945Z
LAST-MODIFIED:20241008T152327Z
UID:23124-1729180800-1729184400@ics.uci.edu
SUMMARY:Decoding Game: On Minimax Optimality of Heuristic Text Generation Strategies
DESCRIPTION:Abstract: Decoding strategies play a pivotal role in text generation for modern language models\, yet a perplexing gap persists between theory and practice. Surprisingly\, strategies that should intuitively be optimal\, such as Maximum a Posteriori (MAP)\, often perform poorly in practice. Meanwhile\, popular heuristic approaches like Top-k and Nucleus sampling\, which employ truncation and renormalization of the conditional next-token probabilities\, have achieved great empirical success but lack theoretical justification. This talk introduces the Decoding Game\, a theoretical framework that attempts to reconcile this gap by recasting text generation as a two-player zero-sum game. In this game\, the Strategist aims to produce text that aligns with the true distribution\, while Nature acts as an adversary\, distorting the Strategist’s target distribution. Our analysis reveals that the adversarial Nature imposes an implicit regularization on the likelihood\, with truncation-renormalization methods emerging as first-order approximations of the optimal strategy.
URL:https://ics.uci.edu/event/decoding-game-on-minimax-optimality-of-heuristic-text-generation-strategies/
LOCATION:Donald Bren Hall\, Irvine\, CA\, 92697\, United States
ATTACH;FMTTYPE=image/jpeg:https://ics.uci.edu/wp-content/uploads/2024/10/jason-klusowski-resized.jpg
END:VEVENT
END:VCALENDAR