this post was submitted on 03 Sep 2024
430 points (97.4% liked)
Linux
48331 readers
489 users here now
From Wikipedia, the free encyclopedia
Linux is a family of open source Unix-like operating systems based on the Linux kernel, an operating system kernel first released on September 17, 1991 by Linus Torvalds. Linux is typically packaged in a Linux distribution (or distro for short).
Distributions include the Linux kernel and supporting system software and libraries, many of which are provided by the GNU Project. Many Linux distributions use the word "Linux" in their name, but the Free Software Foundation uses the name GNU/Linux to emphasize the importance of GNU software, causing some controversy.
Rules
- Posts must be relevant to operating systems running the Linux kernel. GNU/Linux or otherwise.
- No misinformation
- No NSFW content
- No hate speech, bigotry, etc
Related Communities
Community icon by Alpár-Etele Méder, licensed under CC BY 3.0
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Not an expert in both the languages but I heard that C developers are trained to use memory smartly, sometimes even reuse a range of allocated memory for completely different purpose to save cycles freeing and reallocating. But for Rust developers, everything is about making sure when one should get the hand away from the memory, and whose memory is allowed to be touched.
Sounds to me like sharing rides that maximise economically but we may have some oops moments sitting on someone's laps vs absolute private rides to make sure no one in your family will be harmed but we have to make sure everyone gets a car only when needed.
It is quite interesting to see how it will work out eventually...
Kernel coders are an entirely different breed, and when I worked with a few of them they were just stunning. The smartest man I know on the planet so far coded on the Unix kernel -- the one that IBM forced back to Novell who'd already fired their staff after selling it, and thus shelved it and killed Unix. He is and was amazing.
So yes, I can confirm that Kernel devs know how to manage their memory -- they use very little, they allocate and free it, and they build very small, tight, optimized kernels by knowing how the optimizer will do things and how to hint it to do what they know needs to happen.
Yeah, it's a skill. Yeah, it takes skilled people. I'd like to one day find out that really big training wheels will let anyone build code that well, but I've seen the goal and I don't expect we're there yet.
Let the kernel be built by kernel devs.
They are amazing but at the end of the day they are still humans and they can make mistakes. In the YouTube video referenced one of the C devs is heavily against rust.
Decided to go look for CVEs from code the guy manages (Ted Ts'o) I found these
CVE-2024-42304 — crash from undocumented function parameter invariants
CVE-2024-40955 — out of bounds read
CVE-2024-0775 — use-after-free
CVE-2023-2513 — use-after-free
CVE-2023-1252 — use-after-free
CVE-2022-1184 — use-after-free
CVE-2020-14314 — out of bounds read
CVE-2019-19447 — use-after-free
CVE-2018-10879 — use-after-free
CVE-2018-10878 — out of bounds write
CVE-2018-10881 — out of bounds read
CVE-2015-8324 — null pointer dereference
CVE-2014-8086 — race condition
CVE-2011-2493 — call function pointer in uninitialized struct
CVE-2009-0748 — null pointer dereference
Do you see a pattern in the type of error here? It's pretty much entirely memory related and right in the wheelhouse of something rust would just outright not allow short of just slapping everything into unsafe blocks.
The Old Guard is not perfect, and they are acting as a barrier to new talent coming in. Sometimes change is good and I'm heavily in the camp that rust one of those times. Linus seems to agree as he allowed the code into the kernel which he would never do lightly or just because it's fomo
But on the other hand you can’t expect some smaller and smaller subset of the population to primarily just learn C and meet the criteria of a kernel dev.
I absolutely agree with all your points, and most rust devs would agree, but the general idea is that over time that energy (which would have been spent tweaking malloc and such) should be spent on the rust compiler and memory management systems, which is already magic as someone who as written a lot of c, c++, and spent the better part of a year learning rust. (I’m no expert of course, but I have a pretty decent grasp on the low level memory management of both the Linux kernel and the rust compiler).
So that over time the effort that would be spent on memory management and kernel functionality can be properly divided. Rust not being efficient somewhere in catching memory faults or managing memory? Fix it. Someone writing unsafe rust code? Fix it.
I think at the end of the day everyone wants the same thing which is a memory safe kernel, and I think that rust Is being shoehorned into kernel projects too early in places where it shouldn’t be, but I also think there is unnatural resistance to it just because it’s different elsewhere to "how it's always been done."
I guess the question is, what happens to the kernel when all the people who learned on C are gone? The majority of even the brightest new devs aren't going to cut their teeth on C, and will feel the same resistance to learning a new language when they think that there are diminishing returns to be had compared to what's new and modern and, most importantly, familiar.
I honestly get the hostility, the fast pace of technology has left a lot of older devs being seen as undesirable because the don't know the new stuff, even if their fundamental understanding of low level languages could be a huge asset. Their knowledge of C is vast and valuable, and they're working on a project that thrives because of it. To have new people come to the project and say "Yeah, we could do this without having to worry about all that stuff" feels like throwing away a lot of the skill they've built. I'm not sure what the solution is, I really don't think there are enough new C developers in the world to keep the project going strong into the future though. Maybe a fork is just the way to go; time will tell which is more sustainable.
Unfortunately there are a lot of problems created by using C in the kernel, and having all of this done manually. Many kernel vulnerabilities including several severe ones have been due to issues with memory management. Even the whitehouse has spoken on these issues related to C. Rust has been proven to be comparable to C in terms of performance, sometimes even faster. So it doesn't make a great deal of sense to keep using C for new projects.
That all being said Rust has had its own issues. There was a recent vulnerability in older versions of cargo the Rust package manager for instance. It's a somewhat new language so obviously teething issues are to be expected, and it might be too soon to use Rust for mission critical systems. It's also a harder language to learn and understand, so that makes adopting it more difficult especially for very experienced C developers like those who work on the Linux kernel. It might be better to wait and see what other languages like Zig and Carbon manage to do, but those are even newer and will take more time to actually be production ready.
Expecting C programmers to like a compiler-based approach to memory safety is like expecting petrolheads to like a car purely because it's electric. They have always viewed compiler based memory safety techniques as guard rails for novices. In their view, good bowlers don't need guard rails at the bowling alley. It's a massive massive clash of cultures and the rust folks come into the discussion with an assumption that C devs would leap with joy at the chance to automate memory management. Rust and C are complete opposites, but rust programmers seem to assume that just because rust is fast C programmers will love it.
That might be true but it's not what happened at that specific conference. I beg you watch the clip to see what happened. Also fuck programmers with the attitude you describe. It's been proven wrong over and over again with so many C memory safety vulnerabilities.
I saw the clip previously. The rust guys are absolutely assuming that the C guys would go for something because (a) the compiler guarantees it's memory safe (b) the semantics would be encoded in the type system. They demonstrate this using rust terminology and algebraic data types. Algebraic data types are the bees knees, (but not with that syntax and clumsiness), and compiler guarantees are the bees knees, but that's not how a C programmer who's middle aged sees the world, it just isn't. Your typical middle aged C programmer grew up telling pascal programmers that automatic array bounds checking is for wimps and real men use pointer arithmetic and their programs run five times as fast. They were always right because their programs did really run significantly faster, but now rust comes along and its fast and safe. Why wouldn't C programmers like it? Because the speed was the excuse and the lack of guardrails was the real reason they liked C.
I said it's a massive culture clash that the rust folks didn't realise they were having because they just assume that "memory safe" wins people round, whereas C folks value their freedom from automatic compiler-based safety, and here you are, sounding like a rust person, saying it isn't a culture clash at all and that the rust folks are right about memory safety and the C folks are just being irresponsible.
They aren't asking C devs to write Rust code, which is what the guy being a heckler was claiming. Why don't they want to right Rust? For exactly the reasons you describe. The thing is though that's not currently being asked of them, all they actually want is the documentation to create that code themselves.
You really don't have to explain any of the culture clash to me lol. I've written both C/C++ and Rust. My C and C++ coding skills are demonstrably better (or at least used to be, it's been a while) than my Rust skills. Why? Because of how complex those guardrails are. The difference is I have the self awareness to know that my lack of Rust skills doesn't mean that the language is bad, or that C is a safe language to use. Rust tutorials could be improved. Perhaps an easier to use language like Zig might be more useful for some people. I feel like it's a good compromise between safety and ease of use. Rust though is still incredibly progressive for the industry, and will improve systems security, maintainability and reliability going forward if only people would stop getting in the way.
TL;DR: Vast culture clash that rust guys didn't perceive and C guys hated and false assertion that "you don't need to learn rust" based on inexplicably naive lack of understanding that maintenance might be necessary.
If someone builds a rust api on top of your C code inside your project, you have exactly five choices: (1) preserve the assumptions the rust code is making (2) only change your code if you have a rust expert to collaborate with handy (3) edit the rust code yourself (4) break the rust assumptions leading to hard to find bugs (5) break the build. The C guys hated all five of those options, and the rust guys told them they didn't need to worry their pretty little heads about it. ON, they weren't as dismissive as that, but they either didn't understand those as issues or didn't care about them or dismissed them.
The rust guys were asking the C guys to tell them the semantics so that they could fix the type signatures for their rust functions and the C guys were reluctant to do that because they wanted to be able to change the semantics of that turned out to be useful to them. They didn't want to commit so something that was documented in a way they weren't familiar with because they felt that even if they wanted to, they couldn't ensure their code was compliant with this specification going forward because they didn't understand the rust type signature fully. (They got hung up on the self argument and launched a rant against OOP.)
The rust guys knew instinctively that the Result return type meant that the operation could fail and could tell from the two arguments to that both in what ways it could fail and every kind of answer it could produce if it succeeded, but the C guys found almost none of that obvious. This was for just one function in the rust API, but it also radically changed the way of doing it. This one rust call replaced the whole algorithms of ask, check answer, if none, check this and that, otherwise do this blah blah blah. The C guys are used to keeping everything lean and simple with a single purpose and were being asked to think of a while collection of procedural knowledge and edge cases with a handle everything monolith. But they were audibly reluctant to commit to that being all the edge cases because they don't think of all of those tests as one thing and instinctively wouldn't write something that checks for all of the edge cases because (a) in a lot of circumstances the code they're writing only needs to know that there was a problem and will give up quickly and move on and (b) they want to be able to freely choose to add other edge cases in the future like they normally do without having to worry about the rust code breaking.
They weren't complaining that they were being asked to write rust, they were complaining that they didn't want to learn rust, and they were complaining this because they could see that to preserve all the rust API type signatures they would have to understand them, the expectations around them and memory safety principles, so that a rust programmer in the future wouldn't have to change the rust type signature.
The rust guys would have gained a lot more traction by just asking the C guys to keep a bunch of comments up to date detailing the semantics and error checking procedures, and promising to edit their rust API if the C code changes, but I suspect they didn't ask for that because they know that no guarantees come from a comment and they want to be sure that the rust code works across all the possible scenarios and in rust culture, that is always documented in the type system where it can be enforced.
The rust guys spoke like it was self evident that having a monolithic API with a bunch of stuff guaranteed by the rust compiler was best, but seem not to have realised that this is a massive culture clash because the C guys come from a culture of rejecting the idea of compiler guarantees anyway (because they have long had confidence in their ability to hand optimize their code to be faster than some prescriptive compiler's output and look down on people who choose to have the guardrails up).
They felt like they were being asked to help write an interface definition in a monolithic style that they have always rejected, to achieve goals that they have long resisted, in a language that they find alien, with no guarantees for them that the rust guys were going to stick around to agree and implement the rust changes necessary if they changed the C code, and with no confidence that they understood what would count as a breaking change at the rust level.
This perceived straightjacket made them particularly cross. They complained about the inability to change their C code and its semantics and the need to learn enough rust to understand quickly what not to change, but they didn't want to not change things and would need to edit the rust API at the same time as editing the C code if they didn't want the rust build to break, and then there would be even more downstream changes from that, so realistically they would need not only to be able to understand the rust type signatures, they would need to be able to edit both the type signatures and the functions themselves, and basically maintain all the downstream rust, and they would want to be sure they were writing efficient rust, well aware that it took them decades to get to the level of extreme efficiency they write in pure C, a much simpler language.
The rust guys said "Just tell us what your code means so we can write our type signatures", but the C guys didn't want to help create for themselves a prison whose walls were of a strange and intricate design they found hard to perceive, made out of materials they didn't have experience working with. They felt like the first guys were asking them how all the doors, windows, chimneys, air vents etc of the house that they built by hand would ever be used, so they could encase it in a stainless steel shell and make it part of a giant steel city. The C guys said "but I might want to build an extension or a wider garage!" They claimed that the C guys didn't have to learn how to weld or manufacture steel sheets, and that their house would be much safer, but for some reason this didn't win the C guys round to the plan, and there's a bunch of people online calling the C guys tech luddites for not liking the whole thing and saying that they were incorrect that they needed to learn rust just because the rust guys made that claim, but that claim is actually completely incorrect unless you think that it's OK to stop the project compiling with your pull request or you think that changes to the C code should be banned wherever a rust API is built on top of it.
I could be being daft but I thought this is more or less what the Rust guys were asking for. Tell us the current symantics of the system, and if it changes in future let us know what the new semantics are and we will fix the Rust code accordingly.
I do understand what you mean though about enforcing restrictions on what the C guys can do without breaking the Rust code. I think you run into situations wherever two languages meet. The way most projects handle this is the upstream releases a new version, or a release candidate of a new version with their breaking changes documented and then downstream updates their stuff accordingly when they get time. Obviously this is one project, but I imagine it's possible for the C guys to update stuff in a pull request and then drop an email in LKML to the Rust guys so they know stuff needs fixing. None of this seems that hard to me.
Ultimately though everything here is Linus decision. Either your in or your out. If Linus says yes to Rust doing whatever then that's what's going to happen. Likewise if he says no, then it's not going to happen that way. Until he weighs in no one can really say how this will end.
Personally though I disagree with the C guys. Safety features are important and should be used where it is practical to do so. Until now C has had the justification that it's still the fastest language and by a significant margin. Now a somewhat safe language like Rust exists with the same speed and capabilities I don't think we can afford to continue ignoring safety for the sake of a few bruised egos. If this was a proper industry like aviation safety would always come first, and if that means adopting new technologies and forcing people to adapt. I can understand if C devs have a hard time adapting, I don't expect it to happen overnight. The expectation though should be they should learn some Rust eventually, even if it's just enough to know the type signatures and what not that they might break with their changes to C code. Kernel devs are supposed to be some of the smartest computer people out there. If they can't learn even that small amount of another language then should they really still be kernel developers?
Oh yeah, rust has to win, but I think this was an empathy-free paradigm war masquerading as an innocent request for information. I think trying to bolt rust into Linux is a strategic error. It's going to cause quite a lot of unnecessary friction and an awful lot of unnecessary technical complication and will be absolutely riddled with complexities and ways of doing things that are inherently unsafe. Instead build a posix compliant OS as rust from the bottom up and it'll knock the spots off Linux and will be rock solid. It'll take well over a decade but it'll be far, far better.
That's already being attempted in the form of Redox OS. Though I don't think it's 100% POSIX compliant. Linux has so much inertia though, and Linus seems all for including Rust in Linux.
That is honestly a decent analogy. So, on what rides is it ok if something goes wrong and a young family member is killed? Rust says, it is never ok so we won’t let you do it.
To use your analogy though, the issue is the driver feeling quite confident in their skills and rating the risk as low. Then a tire blows on a corner. Or somebody else runs a red light. Or, there is just that one day when an otherwise good driver makes a mistake. History tells us, the risk is higher than the overconfident “good” drivers think it is.
In particular, history shows that 70% of the real world injuries and fatalities come from passengers without seat belts. So, instead of each driver deciding if it is safe, we as a society decide that seat belt use is mandatory because it will prevent those 70% of injuries and fatalities ( without worrying about which individual drivers are responsible )
Rust is the seat belt law that demonstrably saves lives regardless of how safe each individual driver thinks they are. It is a hard transition with many critics but the generation that grows up with seat belts will never go back. Eventually, we will all realize just how crazy it was that they were not always used.