Many AI systems operate in real-world settings that require making sequential decisions under constraints--such as limited capacity or restricted access to information--while still providing learning or inferential guarantees. This talk presents two ongoing projects with this structure and focuses on the mathematics behind such problems. First, in a queueing model motivated by hospital discharge decisions, I show that capacity constraints can induce censoring and biased learning of patient risks, and that simple discharge-up-to policies avoid these issues and enable consistent learning. In a second setting, I study algorithm auditing under limited or strategic data access, using betting-based martingale methods to design risk-limiting procedures with valid guarantees. Together, these studies illustrate the use of tools from stochastic processes to design sequential decision rules for AI systems that remain robust under real-world constraints. This talk is based on joint work with Kara Schechtman, Pengyi Shi, Angela Zhou, Inioluwa Deboraj Raji and Ian Waudby-Smith.