« BackCommitting to Rust in the Kernellwn.netSubmitted by todsacerdoti 10 months ago
  • p1necone 10 months ago

    "Torvalds said that it is not necessary to understand Rust to let it into a subsystem; after all, he said, nobody understands the memory-management subsystem, but everybody is able to work with it."

    Chuckled a bit at this line, anyone have context on how true this is?

    • j16sdiz 10 months ago

      "Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future. Kroah-Hartman said that the Rust developers can take responsibility for the maintenance of the abstractions they add."

      This need some very good expectation management.

      • j16sdiz 10 months ago

        For most driver or subsystem, maybe you don't need to know how mm works.

        Rust is different. The kernel Rust teams are trying to encode some safety invariant. If any of those mismatch with the C side, it breaks. Those invariant need some non trivial knowledge of rust to understand

        • syndicatedjelly 10 months ago

          It’s an opinion, but it sounds very good from the perspective of treating the relationships between system and subsystems as an interface to be managed.

          • raggi 10 months ago

            It's very probably true in the totality of "as expressed in a real build for all configurations and architectures", too much variation of behavior to have the whole map in mind at once. You can work through it potentially, and I'm sure a few come close, but others will have things top of mind that experts don't.

            • klysm 10 months ago

              It's true in the sense that nobody understands it well enough to avoid writing memory safety bugs.

            • saagarjha 10 months ago

              Aside: LWN is basically what ChatGPT summarization as advertised itself to be, except it's actually good and coherent and useful. I can trust Jonathan to summarize the conversation in a way that is mostly sane and reasonable in a way that I can never hand off the job to generative AI. It's an important area where humans seem to excel over computers.

              Also, just to have some content that is actually on-topic: is anyone actually shipping upstream Linux Rust code yet? I understand that some stuff is slowly merging in but I'm not sure if it's actually being exercised yet.

              • rc00 10 months ago

                > I can trust Jonathan

                Given the clear bias in the writing and the coverage, how do you reconcile this? LWN isn't a news site staffed with reporters any more than Fox News is and it is much more closely defined by being a personal infoblog these days. Phoronix, Linuxiac, and DistroWatch are a few examples closer to actual news without trying to lead the reader. The Register, even with snark, is more representative of a news site these days.

                I used to be a fan of Jonathan and LWN but in recent times, he seems less interested in being a reporter and more interested in being a Rust evangelist. That isn't a source I can trust.

                • patmorgan23 10 months ago

                  I believe the kernel graphics drivers for the Apple M1 are written in rust, and are upstreamed.

                • jsheard 10 months ago

                  On a semi-tangent, does anyone happen to know how Microsofts push to use Rust in the Windows kernel is coming along? They rewrote some components in Rust and rolled them out to production about a year ago but it has seemed pretty quiet on that front since then, unless I missed something.

                  • tsujamin 10 months ago

                    I noticed the other morning that they’ve either re-written or are growing the win32k driver in rust. This was either in Server ‘25 or vNext though I can’t remember.

                    Given win32k implements a good chunk of the kernel-mode graphics and windowing system it’s a pretty good place to start that effort.

                    edit: win32kbase_rs.sys was what it was called, and I’m pretty sure it was 2025 I pulled it from but it might be on earlier versions too

                    • steveklabnik 10 months ago

                      They haven’t spoken publicly about it. As a Windows user, I am very intrigued!

                      • pjmlp 10 months ago

                        While they are rather quite on that front, Rust is now the official systems language for new projects in Azure infrastructure, with C#, Java and Go as alternative is a managed language is also possible.

                        "Decades of vulnerabilities have proven how difficult it is to prevent memory-corrupting bugs when using C/C++. While garbage-collected languages like C# or Java have proven more resilient to these issues, there are scenarios where they cannot be used. For such cases, we’re betting on Rust as the alternative to C/C++. Rust is a modern language designed to compete with the performance C/C++, but with memory safety and thread safety guarantees built into the language. While we are not able to rewrite everything in Rust overnight, we’ve already adopted Rust in some of the most critical components of Azure’s infrastructure. We expect our adoption of Rust to expand substantially over time."

                        From https://azure.microsoft.com/en-us/blog/microsoft-azure-secur...

                        Several key projects have been migrated to Rust, or started from Rust althogether.

                        => Azure Boost

                        https://learn.microsoft.com/en-us/azure/azure-boost/overview

                        "Rust serves as the primary language for all new code written on the Boost system, to provide memory safety without impacting performance. Control and data plane operations are isolated with memory safety improvements that enhance Azure’s ability to keep tenants safe."

                        => OpenHCL, Azure's para-virtualization

                        https://techcommunity.microsoft.com/t5/windows-os-platform-b...

                        "OpenHCL is a para-virtualization layer built from the ground-up in the Rust programming language. Rust is designed with strong memory safety principles, making it ideally suited for the virtualization layer."

                        => Security processor Pluton firmware (used by XBox, Azure and CoPilot+ PC hardware)

                        https://learn.microsoft.com/en-us/windows/security/hardware-...

                        Post from David Weston, Microsoft's vice president of OS security regarding the Rust rewrite and TockOS adoption, https://x.com/dwizzzleMSFT/status/1803550239057650043

                        => CoPilot+ UEFI firmware

                        https://techcommunity.microsoft.com/t5/surface-it-pro-blog/r...

                        "Surface and Project Mu are working together to drive adoption of Rust into the UEFI ecosystem. Project Mu has implemented the necessary changes to the UEFI build environment to allow seamless integration of Rust modules into UEFI codebases. Surface is leveraging that support to build Rust modules in Surface platform firmware. With Rust in Project Mu, Microsoft's ecosystem benefits from improved security transparency while reducing the attack surface of Microsoft devices due to Rust’s memory safety benefits. Also, by contributing firmware written in Rust to open-sourced Project Mu, Surface participates in an industry shift to collaboration with lower costs and a higher security bar. With this adoption, Surface is protecting and leading the Microsoft ecosystem more than ever."

                      • newpavlov 10 months ago

                        >Changing C interfaces will often have implications for the Rust code and may break it; somebody will the have to fix the problems. Torvalds said that, for now, breaking the Rust code is permissible, but that will change at some point in the future.

                        I think this is the main technical change needed from the Linux kernel. It needs a layer of quasi-stable well documented subsystem APIs, which ideally would be "inherently safe" or at least have clear safe usage contracts. And it's fine for these interfaces to have relaxed stability guarantees in the early (pre-1.0, if you will) experimental stages. Changing them would involve more work and synchronization (C maintainers would not be able to quickly "refactor" these parts), but it's a familiar problem for many large projects.

                        It's the only reasonable point from the infamous tantrum by Ted Ts'o during the Rust for filesystems talk, everything else, to put it mildly, was a really disappointing behavior from a Linux subsystem maintainer.

                        • w10-1 10 months ago

                          > It needs a layer of quasi-stable well documented subsystem APIs

                          I think the Rust developers weren't even asking for that. They just want the C developers to sign up to some semantics. But the C developers know the function interface has evolved to be merely functional: it works, but with few invariants, riddled with caveats and few cross-function guarantees. It can't be hoisted into meaningful semantics, no less a type system, particularly across 10 filesystem API's.

                          Rust developers should focus on drivers in subsystems with stable API's, instead of trying to stabilize what decades of work has failed to.

                          • BD103 10 months ago

                            For those curious, this is the link[0] to the filesystems talk with the relevant timestamp. A bit more was discussed in this[1] article as well about Wedson Almeida Filho leaving.

                            [0]: https://youtu.be/WiPp9YEBV0Q?t=1529

                            [1]: https://lwn.net/Articles/987635/

                            • josephcsible 10 months ago

                              Why do we need quasi-stable anything within the kernel? Wouldn't a much better long-term solution be changing the rule to "break whatever APIs you want, but you have to fix all of the in-tree Rust uses too"?

                            • brotchie 10 months ago

                              Thought that ThePrimeTime in his video https://youtu.be/62oTu9hjxL4?si=E98WZ0zJSNUC8TEH&t=287 hit the nail on the head re: Rust versus C.

                              Max level C programmers, have designed their programming style around control down to the absolute bit. C derives control from absolute control over behavior.

                              Max level Rust programmers have complete mastery of types. Rust derives control from types.

                              Seems somewhat philosophically incompatible.

                              • samsartor 10 months ago

                                It is a good video, but I'm not at all convinced that a philosophical "authoritarian" vs "anarchy" difference between Rust and C actually exists. C programmers work through all sorts of constraints and rules on how to correctly use a system, same as anyone else. Heck, there are hundreds of pages of kernel documentation explaining how developers are expected to use the various locking subsystems. Does that make C authoritarian? I don't think so... that's just a fact of programming. The details matter.

                                IMO the only real cultural difference in Rust is that you are expected to explain those constraints through the language of the type system, not just in English. That's a lot of work up front but it also gives you way more automation down the line (eg checking that you used a mutex correctly via rustc rather than through emails to Linus Torvalds). Some people definitely take it too far, and blow up their code with endless incomprehensible traits. But the islands aren't incompatible, it just takes work and skill to bridge them.

                                • raggi 10 months ago

                                  This is less true in "how the language works", and more true in "code I have read in these languages follow these patterns". The latter I agree with, the prior less so. Rust is more type heavy sure, but there are plenty of type shenanigans in rich enough C ecosystems, the kernel has plenty of vtable structures which have varying level of richness and type complexity in their own C expression. Ever worked with the addr types in the BSD sockets API - super messy types that can be a real pain for FFI for example, heavier typing exists in C - not "higher order" and so on, and yes you _can_ do that in rust, but do you want to debug it in kernel use cases, maybe not. Rust kernel code may look different in the end from a lot of other Rust code in a similar way to some C code in the kernel being quite different from other C code elsewhere.

                                  • asveikau 10 months ago

                                    Good C code will select a rigid set of patterns, carefully chosen to maximize safety, and stick to them. It makes a lot of the bug prone patterns stand out.

                                    This is somewhat like what a higher level compiler can produce, but more manual. It's reliance on code smell instead of a type checker.

                                    • pjmlp 10 months ago

                                      "C derives control from absolute control over behavior."

                                      Many C devs like to think C is like Assembly, then they discover it is an high level systems programming language like every other.

                                      • 01HNNWZ0MV43FF 10 months ago

                                        IDK, I thought well-written c and c++ had the same notions of ownership and lifetimes, just not enforced by the compiler

                                      • ristos 10 months ago

                                        Can anyone explain to me why these two issues aren't considered deal breakers for introducing Rust into the kernel?

                                        1. It doesn't map almost 1:1 to assembly the way C does, so it's not inherently clear if the code will necessarily do what it says it does. That seems questionable for something as important as a kernel and driver.

                                        2. Only one real Rust compiler, that's a recursive compiler, which reminds me of the Trusting Trust problem: https://dl.acm.org/doi/abs/10.1145/358198.358210

                                        • jcranmer 10 months ago

                                          > 1. It doesn't map almost 1:1 to assembly the way C does, so it's not inherently clear if the code will necessarily do what it says it does.

                                          As someone who works on a C compiler, I will tell you that Rust maps marginally better 1:1 to assembly than C does. No major C compiler goes 1:1 to assembly; it all gets flushed into a compiler IR that happily mangles the code in fun and interesting ways before getting compiled into the assembly you get at the end. Rust code does that too, but at least Rust doesn't pull anything silly on you like the automatic type promotion that C does.

                                          If C maps 1:1 to assembly in your view, then (unsafe) Rust does; if Rust doesn't map 1:1 to assembly, nor does C. It's as simple as that.

                                          • steveklabnik 10 months ago

                                            1. C doesn’t actually do that. Rust is the same as C in this regard.

                                            2. The Linux kernel doesn’t use standard C, it uses many gcc specific extensions. By this point, clang also supports those extensions and can compile the kernel, but that took work, and upstream has never tried to be only standard C.

                                            • ozgrakkurt 10 months ago

                                              This complexity issue is very similar to memory issues of older languages. Most rust people say it is ok or you can avoid it etc. but they don’t understand people just tend to go to the path of least resistance and most times this means a lot of traits and generics. Would be super cool to have something like zig but with borrow checker

                                            • aero-glide2 10 months ago

                                              Even though I don't use Linux in my projects, Rust on Linux is extremely important to convince others to use Rust. I hope it succeeds.

                                              • iLemming 10 months ago

                                                Even though I don't write any Rust, I still think Rust on Linux is important. I rather learn and deal with Rust than with C/C++, and I too, hope it succeeds.

                                              • deivid 10 months ago

                                                Not exactly rust-in-kernel, but I've been using Aya[0] to write ebpf programs in Rust and it is quite nice. The verifier is still a bit trigger-happy with some constructs, but it is manageable.

                                                [0]: https://aya-rs.dev/

                                                • duped 10 months ago

                                                  Totally unrelated comment to my previous one

                                                  I think it's folly to encode the semantics of APIs in the Rust type system and memory model and that's the impedance mismatch that has people riled up. unsafe code isn't incorrect code, and trying to add abstraction where there wasn't one before is encoding principles where they didn't previously exist and should be an obvious problem.

                                                  I've written a lot of systems-type Rust using unsafe and I think the design pattern of -sys bindings and then a higher level safe wrapper is mostly incorrect because callers should always use the -sys bindings directly. It's more workable and doesn't suffer from changes that the detractors complain about.

                                                  • ozgrakkurt 10 months ago

                                                    Totally agree with this. In performance critical code this is a problem always. Even zstd bindings in rust is slow compared to directly using sys version. People don’t want to acknowledge this but it is the truth. Not sure why this comment is flagged.

                                                    I have been writing performance critical code for production in rust for several years now. And generics/traits usage and needless atomics/copy/abstraction is very much an issue. Biggest concrete one is memory allocation api and work stealing everywhere kind async apis. It is understandable these make sense for general development but they are just a problem when doing performance critical and low level code

                                                  • duped 10 months ago

                                                    What is the value in relying on distro authors to publish rust compiler versions, when the bespoke release channel for rustc is kept much more up to date?

                                                    • cozzyd 10 months ago

                                                      How can you build a new kernel with distro tools otherwise?

                                                    • strken 10 months ago

                                                      Does Rust support the same range of architectures as C? Or is there some way to compile Rust to C to support weird obscure systems?

                                                      Forgive me if this is something commonly known by the people involved. It just strikes me as the most obvious objection to Rust in the kernel and I haven't seen any explanation of what's going to happen.

                                                      • steveklabnik 10 months ago

                                                        Currently Rust is only acceptable in drivers. These are inherently platform specific, so there’s no issue with platform support. If Rust doesn’t support the platform, the driver won’t be written in it.

                                                        Rust’s platform support is better than you might assume, but it is missing some platforms the kernel itself supports, so until that’s resolved, it can’t be in the core of things.

                                                        • boulos 10 months ago

                                                          Rustc lowers to LLVM IR, so it can target anything LLVM can. For a long while, that meant there were obscure architectures that were excluded, but IIRC coverage has improved and the Linux kernel has decided to drop support for them.

                                                          The main push has actually been from the BSD family pushing clang (and thus LLVM) to support a broader swath of less common architectures.

                                                          • BD103 10 months ago

                                                            Check out the Rust's documentation page on platform support[0]. You'll be able to find the full list of supported platforms, as well as the target tier policy, and specific target requirements and maintainers.

                                                            [0]: https://doc.rust-lang.org/rustc/platform-support.html

                                                            • raggi 10 months ago

                                                              rustc does not yet no, there's a lot of hope in the gcc rust frontend to help bridge the gap, but also there is increasing support in llvm for more architectures and things like this should drive interest in accelerating those projects.

                                                              • Palomides 10 months ago

                                                                rust doesn't support alpha, parisc, or super-h, but afaict there are at least nominally functional back ends for the other platforms modern linux runs on

                                                              • debo_ 10 months ago

                                                                As a side note, does anyone know what the backup plan is if Linus is suddenly no longer able to lead in his current capacity? I think it's likely there is one, but am not sure what it is.

                                                                • arp242 10 months ago

                                                                  In a previous interview (several years ago) Linus said "there are at least a dozen people who can take over tomorrow if I get hit by a bus", or something to that effect. I don't think there's a specific concrete plan.

                                                                  Realistically, what will probably happen is that all the core maintainers will end up in a big room and discuss what to do next. Maybe they will do something with white smoke from a chimney. It's also very possible the nature of the leadership will change, rather than a simple s/Linus/…/.

                                                                  • johnny22 10 months ago

                                                                    ATM Greg KH would probably take over (since he's already done it once before). I doubt he's the last in the line of succession though

                                                                    • klysm 10 months ago

                                                                      No, but as long as it doesn't land in Microsoft's hands I'm happy. edit: Not Oracle either

                                                                    • pjmlp 10 months ago

                                                                      Regardless of what upstream decides, downstream on ChromeOS and Android land, it is already being used.

                                                                      • Waterluvian 10 months ago

                                                                        I’ve read many times that a goal is to see if the use of Rust can work out in the kernel.

                                                                        What does this experiment for? What exactly is the evaluation? What are some example findings, both pro and con?

                                                                        • steveklabnik 10 months ago

                                                                          You can find the home of the project here: https://rust-for-linux.com/

                                                                          Linus has been talking publicly over the last few years about how it’s getting harder and harder to find maintainers. There are various reasons for this, but C is part of the problem. If there are newer technologies that would be appropriate for the kernel, they should be investigated.

                                                                          At the same time, more and more downstream users of the kernel have been adopting Rust, and more kernel devs have been checking out Rust and have liked what they’ve seen. Many also don’t, to be clear, but the point is that a group of people were willing to put in the work to give Rust a real shot, and so did some initial work to sketch it out.

                                                                          Linus agreed to give it a try, but there are some caveats: Rust code cannot hold up improvements to the C code. Changes to the C are allowed to break the Rust, in other words. Then, it was discussed that the right first place to try Rust would be in driver code. This has the advantage of being a real project, but also sidesteps another issue: today, Rust doesn’t support every platform Linux does. Drivers are platform specific, so you just simply only write drivers for platforms Rust supports.

                                                                          So far, there have been some successes: some example drivers have been written. There’s also been some struggles: there are some experimental features of Rust and its standard library that the code relies on. But the Rust project has committed to trying to get those finished sometime soon, so that’s not a permanent issue. There have also been some… skepticism of both Rust and the experiment by some members of the kernel development community, and that’s caused some problems. Hopefully those can be worked out too.

                                                                        • IshKebab 10 months ago

                                                                          > he also is unable to work with Rust until a suitable compiler is available in RHEL

                                                                          What an arse. Best to ignore these people.

                                                                          • cozzyd 10 months ago

                                                                            Why can't all symbols exported to modules maintain a C ABI, obviating (if I understand correctly) the genksyms problem?

                                                                            • duped 10 months ago

                                                                              (by my reading) the problem is that determining what the C ABI is relies on a C parser. If a module depends on an API, genksyms checks if the ABI has changed and won't load the module as a result (or more broadly, it could also use content addressing to look at the body of the function).

                                                                              It seems like the solution is to use DWARF info to determine that but it has to be backwards compatible because previous implementations relied on the parsed C code and not the DWARF symbol info.

                                                                              • kelnos 10 months ago

                                                                                Because then when you want to write a module in Rust, all your interfacing with the rest of the kernel will be done through C function calls, with C semantics, and C expectations of ownership. There's really little point to adding Rust abstractions to the kernel, then, at least when it comes to modules.

                                                                              • joelignaatius 10 months ago

                                                                                [flagged]

                                                                                • slater 10 months ago

                                                                                  HN automatically changes some titles, to counteract against SEO'd clickbait titles.

                                                                                  dunno about the link, but the part after "si=" isn't required, e.g.:

                                                                                  https://youtu.be/_wc7ujflrnI

                                                                                  works fine

                                                                                • stonethrowaway 10 months ago

                                                                                  With the Rust team falling apart, who is actually long term going to maintain it? I want to use it but I just see drama.

                                                                                  • kelnos 10 months ago

                                                                                    One person left. "Falling apart" is a bit hyperbolic, no?

                                                                                    • mynameisash 10 months ago

                                                                                      Who says the Rust team is falling apart?

                                                                                      • mardifoufs 10 months ago

                                                                                        Not sure I'd agree they are falling apart but just to be clear, I think you're referring to the rust Linux team, not the actual rust language team, right?

                                                                                        • arp242 10 months ago

                                                                                          The Rust team is not "falling apart". The article specifically says it's not. One person left. Many people leave projects for many different reasons.

                                                                                          • renewiltord 10 months ago

                                                                                            Same thing will happen as happened when OpenBSD team fell apart.

                                                                                            • stonethrowaway 10 months ago

                                                                                              Not sure why the downvoting. It’s a serious question. Who is ensuring that this project has a future and isn’t going to collapse tomorrow?