Python developers are used to working with APIs that have documentation, versioning, and some expectation of stability. Websites offer none of that — yet many Python systems depend on web data every day.

This talk reframes web scraping and extraction as an API design problem under extreme uncertainty. Instead of focusing on selectors or parsing techniques, we’ll focus on how to design Python-facing interfaces that can survive change.

We’ll explore topics such as optional fields, backward-compatible schema changes, defensive parsing, and meaningful error semantics. We’ll also discuss how breaking changes affect downstream users, especially when your data feeds analytics pipelines, dashboards, or automated systems.
By the end of the session, attendees will have a clearer mental model for treating web data as a volatile dependency and practical strategies for designing Python APIs that protect users from inevitable change.