Shalabh Chaturvedi

Shalabh is a long time Python developer who has worked on various web, large scale and distributed systems. He works on Dagster Cloud, a managed data orchestration platform. He likes questioning established boundaries in software architectures.


Profile image nbpy-2023/question_uploads/shalabh_9r2p2KC.jpg Twitter handle

chatur_shalabh

Mastodon/Fediverse handle

@shalabh@discuss.systems


Sessions

07-29
14:40
25min
Ship your Python code faster with PEX
Shalabh Chaturvedi

A lot of Python code is shipped around in Docker images. But did you know there is another way to ship Python code to production that is... better in some situations? This talk covers our exploration into making code deployment faster for our users (our product runs user code in our cloud). We will cover the limits we faced with Docker builds and container provisioning, various options we explored, and why we settled on PEX.

PEX is a tool that packages Python code and dependencies into a single file. It provides various features such as determinism (given the same input files, you get a bit-for-bit identical output file), isolation (a PEX file runs in the bundled environment, isolated from the system’s site-packages), composition (multiple PEX files can be combined to form new environments), and more. We used a number of these feature to great effect in our solution. By using PEX in addition to Docker we were able to reduce the time a developer waits for code to deploy from 3-4 minutes to about 30 seconds.

Shipping code isn't the only use for PEX. Once we see how PEX files work, we will also go over a few fun tips and tricks that are handy for everyday Python development.

The Barn