Author: Emma Smith
Created at: 2025-11-17 21:38
Number: 29
Clean content: Chris Angelico: More seriously though: Rust has been a focus for a lot of controversy and risk, with massive open questions such as the impact of a potential Ken Thompson style compiler hack . Will CPython lose trust by becoming dependent on a single specific compiler that might be subject to such a hack? I’m not familiar with the risks you speak of. As David Hewitt points out, there is a work in progress implementation using gcc, so while there is currently one compiler, that will not remain the case. Paul Ganssle: I guess the question is a dual one: can we write Rust in a way that it will not cause build times to explode and if we cannot, is the plan to keep the scope of Rust in CPython to a level where building is still very accessible? I mostly will defer to @workingjubilee ’s excellent answer and their expertise. I will add however that build times are dear to my heart, so I think good devguide documentation on setting up a dev environment that is configured for fast incremental builds will go a long way in helping. danny mcClanahan: But I am personally convinced that if CPython were to integrate rust (possibly even just at phase 1, with only external module support), we (CPython and pypa contributors, of which I am only the latter) would necessarily have to figure out a more structured way to thread ABI info through cargo This is a really interesting perspective, thanks Danny! I will say getting cargo to fit into CPython’s current build system was a little tricky. That being said I think the ABI question may be less important until Rust code is exposed to users, which will be after we’ve had a lot of experience working with cargo ourselves and can be planned independently of the initial integration, in collaboration with PyPA. David Hewitt: Very excited to see this initiative and as a proponent of Python, Rust, and the two together, eager to be involved. Very excited to have you join us! We greatly value your expertise on Rust and Python interop. David Hewitt: At the same time, CPython will need safe higher-level Rust APIs to get the benefit of Rust. PyO3 has a lot of prior art on the high-level APIs (and hard lessons learned); I think the right approach here will be similar to what attrs did for dataclasses - the Rust APIs implemented by CPython can pick the bits that work best. This is right along the lines of what I was thinking, so glad we are on the same page David Hewitt: emmatyping: What about platforms that don’t support Rust? gccrs is an alternative implementation of Rust for GCC backend, which is not yet at feature parity but an important target for Rust for Linux. If CPython was using Rust, I would hope that efforts for non-llvm platforms may be helped by this. Excellent point! We should make sure to note this in the PEP. David Hewitt: emmatyping: What about Argument Clinic? PyO3’s proc macros function a lot like argument clinic - we could potentially reuse parts of their design (and/or implementation); PyO3 might eventually even depend upon any implementation owned by CPython. I would recommend this choice as the more idiomatic way to do codegen in Rust. Absolutely, I think this would be a great path forward. I would love if the code could be shared across PyO3 and CPython! David Hewitt: Technical opening - Rust for Linux has unsurprisingly become a major strategic focus for the language. I would hope that Rust for CPython would have justification for carrying similar weight in the focus of the Rust project should there be friction where Rust (and cargo etc) do not currently meet CPython’s needs. Yes, there will likely be a few things upstream that may need some work but probably (hopefully!) less than Linux! James Gerity: I think it’s worth being more explicit about this. I understand the general point, but having references to some recent issues that would have been avoided would strengthen the value proposition of the proposal. If you look at issues labeled type-crash you will see a number of issues, such as Use-after-free due to race between SSLContext.set_alpn_protocols and opening a connection · Issue #141012 · python/cpython · GitHub or heap-buffer-overflow in pycore_interpframe.h _PyFrame_Initialize · Issue #140802 · python/cpython · GitHub or JSON: heap-buffer-overflow in encoder caused by indentation caching · Issue #140750 · python/cpython · GitHub . There are many more. James Gerity: ”What will we do about doubling the number of programming languages in the core” feels important to address up-front. Absolutely, I hope that thorough devguide coverage and good tooling will go a long way in making this experience pleasant. I also think having a team of experts will be useful. James Gerity: It seems to me that an eventual PEP should address ”Why not put that development effort towards RustPython ?” in the Rejected Ideas section Thanks, will add this to the list of rejected ideas to add. I also want to cover other language choices, and a few other things. James Gerity: Does the interaction between PEP 11 support tiers and Rust support tiers merit adjustment of CPython ’s policy? Having 2 additional dimensions to keep track of feels complicated. Hm, what adjustment did you have in mind? Steve Dower: Then you force me to factor in my experience in Rust, my experience in mixed-language codebases, and my experience in teams, and come down as a firm -1 on the entire proposal. I’m sorry to hear that Steve. I’m happy to chat further about your experiences at some point, there are definitely wrong ways of integrating new languages into existing code bases. David Hewitt: For what it’s worth, PyO3 has a mechanism for carrying panics as Python exceptions through stack frames, but I would agree that for sake of binary size and simplicity, an abort would be good enough (the Rust panic hook could be configured to call into Python’s existing fatal exit machinery). That’s good to know! I think we will have to evaluate this (as with many things) and decide based on what our experience finds out. Steven Sun: You can notify me if new Rust code or docs requires review. I support if we can start with some extensions with C/Python fallback. Thanks Steven! It’s been great to see a number of people excited about contributing to CPython in Rust if it were added. Steven Sun: Based on my experience, I want to highlight that certain common practice do not align with Rust. For example, fork() is not usable for many Rust libraries (states may be incorrect after fork). https://stackoverflow.com/questions/60686516/why-cant-i-communicate-with-a-forked-child-process-using-tokio-unixstream fork() + threads is sadness pretty universally, it has been an issue for CPython before, so it’s an issue I’m well aware of. Thanks for bringing it up! Dmitry: Another benefit that you might want to mention in the PEP is how IIUC this would open the door for the possibility of shipping with Python modern Rust-based tooling for dep management, formatting and linting, with all the performance, correctness and community enthusiasm that comes with it. That’s a good point, I’ll make sure to note that in the PEP. I misunderstood this post I think, see my comment below.
