I feel like there's a lot of misunderstanding of this issue in the software community, because primarily, supply chain risk isn't a software or engineering issue. It's a governance issue.
Someone doesn't have to be a bad actor for a project to have supply chain risk. Nor do all who evaluate supply chain risk have the same security posture and evaluate risks the same as others might. The DoD likely has a very different set of risks they evaluate against for their security posture than you do.
Most supply chain risks are not an indictment of somebody's code or somebody's character. A lot of one person projects are risky just because they're only one person. Having a bus factor of one is a supply chain risk in and of itself.
And while most people don't prepare for war while choosing their packages, it's not unreasonable for a military to do so. During a war, the ability for people to govern themselves and their own projects often changes dramatically, even in democratic countries. It is entirely routine for countries to require cooperation by the force of law in war time, even the US can and has forced private companies to cooperate with war efforts. This is probably not in the security posture calculation for most of us. But it is for some.
Huh? The DoD would not have used the package if they hadn't read every line, locked it down for updates, and were ready to patch it themselves if needed. Can you really imagine in a war they'd be like "damn, if only there were a second person we also don't trust at all to do this work for us cause otherwise we'd just be SOL"
I don't know where you're working, maybe you work in some secret lab where everything is air-gapped and not even the pigeons are allowed within a mile of the facility. In which case, what the hell are you doing commenting on a public message board?
That is absolutely not how DoD works. The vast majority of code is contracted out. Nobody from DoD side is reading any of the code. It's all a series of affidavits and audits for configuration management process. Vendors assert everything's cool. Failed audits lead to fines or revocation of access. And the audits check up on documentation and config. They don't dig into code.
At no point in time is anyone, anywhere, in this process reading every single line of code. Not even A single line of code. I doubt they even read the Software Bill of Materials we're supposed to generate, because I've never heard any feedback on any of it.
Doesn't change the fact that they can just fork it if it ever matters though...
You're missing the point of a supply chain risk assessment. Yes, you can fork a project to maintain it yourself. But, for an organization to do this, they need to allocate resources, e.g. time and money. This is part of the risk you are assessing for in a supply chain risk assessment.
>It’s not until I change downloads to 1 billion downloads that we see 1 package maintained by 1 person, and 9 packages maintained by more than 1.
Which one is that?
Has anyone seen any stats on what happens to a single maintainer project when said person is hit by a bus (or meets some other demise)? With that many data points, there should be enough of them by now to study it.
Is the project taken over by another, single developer? Is it replaced by a similar project? Does it just go away?
Here is one data point.
I bought ASIO Link Pro (software) something like 10 years ago to help route virtual audio devices on my system. The author sadly died and eventually the license key server went offline rendering it unable to start. His nephew looked into it and eventually made the tool free after a year or 2.
I stopped using it after the license server went offline because I still had to record videos. I ended up solving my problem with hardware, but that tool was extremely helpful when I used it for years. It was around $40 at the time. It's one of the few pieces of software I've purchased and felt really happy about it.
Unless something changes in the underlying infrastructure, most packages don't need active maintenance after achieving their objective.
If there is a major change (e.g., Python 3, React Native new arch), they are replaced/forked.
I would love to see a diligently researched episodic series, every episode covering the transition of a popular open-source library/tool/app/site from one maintainer to the next.
And that's why I don't run Netflix.
No, but I would happily pirate that.
I think this is in the realm of a YouTube series. I mean, what's stopping you from doing it?
Closest example I could think of would be Hans Reiser/Reiserfs. It's a more sordid story than just getting hit by a bus, though. Ultimately the project just died.
I don't think this is a good example though as the "sordid" part also made the project toxic for anything that might have otherwise chosen to take it on.
I think this is one thing that people fail to consider: if the code is open source, though it may take time to understand, worst case scenario you can just fork it.
> So while NPM has over 4 million single person projects, they have about 900,000 maintainers for those 4 million single person projects. This will be an important data point at the end.
Am I missing something or was it not, in fact, an important data point at the end?
And even in projects that are maintained by more than one person, it's usually just a single person responsible for most of the commits.
This is the exact reason I decided to avoid 11ty for my personal website and instead went with Jekyll [0].
If they had done an activity check they would have seen that half of all projects have zero maintainers.
software once "perfected" (working well enough long enough) needs NO maintenance. No cleaning. No calibrating/tunning.
updating is a systemic issue, not a per-project matter
Definitely varies with language/runtime/library choice. I have no problem using a clojure library that hasn’t been touched in 5 years. But back when I had a gatsby site (static site generator for react) I would end up in the dependency hell after literally a month of not touching it
Under a microscope, maybe.
But if you had a "perfect" piece of software that used Log4j in 2020, it wouldn't have been perfect for long.
Unfortunately, there's a lot of reasons that software needs maintenance, even if it was thought to be perfect when it was originally written.
Hardware changes. The software landscape changes. Dependencies are deprecated, or are found to have their own problems. Vulnerabilities are discovered. Vulnerabilities are found that aren't even the fault of your software, maybe they are a flaw in the hardware your software runs on, and the only way to fix it is via a software mitigation. These are all real things that happen to otherwise perfect software.
Ironically if you didn’t upgrade from 1.x you didn’t get the new features or the bug you’re referring to
2.x had been out for about six years by the time the vulnerability was discovered.
That is a hysterically wrong statement.
It is true of Solitaire, Minesweeper, Calculator, and Notepad, and probably about the same number of programs on other OSes. (Notepad has recently had an important expansion of functionality, but it didn't NEED that change.)
It's also true of some dinosaurs I have on my system, that copy DVDs and so forth.
It's not true of most other applications, nor can it be true, unless the app works in a sealed, unchanging environment.
Even then... Voyager 2 recently required a software upgrade, IIRC.
The point is everything require maintenance, the degree at which it does require it depends on how dependent you are on it and how resilient the system itself is.
You are but going to fundamentally be in distress if solitaire and minesweeper is not running, if your monitoring SW for some important infrastructure starts to exhibit some issues, you might want to take a look or two...
Maybe we need a Linux distro based on "inactive" software and look how reliably it performs.
I was once forced to use older (but not deprecated) LTS Ubuntu and I hated it. New software come out and you're gonna want to use them (often forced to use them), and they of course use newer dependencies. I had to do the distribution maintainer job and package a bunch of software myself.
What sort of work do you do?
I only use LTS distributions, and this is not a problem I have encountered, so I wonder what accounts for the difference in our experiences.
s/inactive/stable/
Well, when you talk about a distribution there's a different issue.
The entire Linux ecosystem is constantly shifting with each package releasing new versions, and therefore everything else must be updated to accommodate the changes in the dependency tree.
You could get away with some stuff being only stable versions, but things like mesa, x11, chrome, etc... would still be constantly changing as would their dependency trees.
I find it more concerning that the DoD uses node.
I might be wrong but npm etc feels like a very large attack surface.
Why?
The DOD is one of the world's largest organizations. There are people there who do things like publish newsletters and put up webpages for people like boy scouts to arrange tour bases. It is totally fine to use Node for things like that.
Those systems are not connected to the systems that fire missiles. If the sign up page for the 4th of July fireworks announcement gets vandalized, it isn't really an issue.
The DoD is a huge organization, so I'd guess they use almost everything.
> The DoD is a huge organization
That's an understatement if there ever was one.
Woah that’s insane, I didn’t realize it was THAT big. And that’s not even counting the zillions of contractors and consultants. I live in the DC area and I know a ton of people who work for places that contract for the DOD, and only like 2 people who actually work there
I think I'm even more amazed that Walmart has almost as many employees as the DoD.
The DoD is very efficient at finding something they are getting for free and convincing everyone it's in their best interest to pay a team of contractors for it.
The city of Troy kind of got fucked that one time by free shit.
Come on, tell me you don't want a pony!
I've heard good things about work done by this guy Linus. I'm pretty sure that I've used his work.
I think he comes from a country that borders Russia, so should we be worried?
I've done OSS for decades; mostly by myself, but sometimes, in teams of volunteers.
If anyone has any experience, working in teams of volunteers, it can be ... challenging.
It can definitely work, but not as often as you'd think. If it works, there's usually some "BDFL," or a common goal that has everyone on the same beam. In my case, it was usually the latter.
(Off topic.)
Not only that, but Linus's parents were politically active communists and young Linus was a pioneer (like a boy scout but for communists). His father also lived in Moscow for several years on two separate occasions.
I don't think Russia (or China, either) has been truly communist, in a long time.
Not sure there are any real communist nations left. It's one of those ideologies that looks good on paper, but falls apart, as soon as humans get added to the soup.
Idealists never seem to account for base human nature.
>> It's one of those ideologies that looks good on paper, but falls apart, as soon as humans get added...
Name an ideology where this doesn't happen.
True, dat…
Your parent comment labeled themselves off-topic but I'd say they were still pretty on it, but you're like way too off-topic. The point isn't whether some country or some people are real communists or not, but that an individual shouldn't be harassed for maintaining open source software and can somehow be linked to some rival of the West.
Fair point.
But, to be fair, the note about not taking human nature into account, applies everywhere.
I think that we've all seen very smart people fail to account for human nature, and things go badly.
Open source/free work is very human, and I have found it important to keep human nature in mind, as I work.
> Linus was a pioneer
Being a Young Pioneer or joining the Komsomol was not officially mandatory, but it functioned as a gatekeeper for any kind of advancement. Party membership operated the same way.
So, by themselves, they don't tell you whether the person in question is a communist.
Not in Finland which has never been a communist country. His parents were just political activists who forced young Linus to participate in that as well. Linus has said that the experience made him very apolitical person.
Ah, whoops! I mistook Linus as the name of the Russian developer in question, whose name is actually Denis.
(The whole first name thing in software circles kind of irritates me.)
Well, my use of his first name was kind of a joke. I apologize for the confusion.
There's very few folks in software that can be recognized uniquely, by their first name, but he's one.
Linux is a well supported project with a lot of maintainers and support, it isn't a one-man project by Linus.
Not anymore, but that was not always the case. He just has an extremely strong will, and a force of personality, that was able to shepherd the project through its nascent challenges.
I have founded fairly important projects (nowhere near on the scale of his work, though), but I don't have the force of personality he does, so tossing the keys to a new team, and walking away, is what worked for me.
Of course, and likewise it wasn't always the case that Linux was a well trusted operating system with a robust supply chain. It took a lot of time, people, and investment to prove that out. The organizations who cared about their technology supply chain, didn't adopt it until that was the case. These are some of the forces that built companies like Red Hat.
I feel like you’ve missed the sarcasm here and zeroed in on correcting PC. Good old HN
Open Source is just a guy, and The Internet is just his computer.
the west or those with largely liberal viewpoints who think in black and white vs seeing the world as grey are gonna cost the west a lot.
we already saw this - with 'cancel' mafia.
because russia or i.e putin invaded ukraine doesn't mean the whole russia is bad. or you shouldn't interact with russia at all. no one stopped interacting with usa after they invaded iraq.
just because russia doesn't give a shit about lgbtq rights doesn't mean russia is a bad country. likewise just because china runs an explicit authoritarian system - it doesn't mean its a country - china bad.
trump and his idiotic gvt kinda recognize this - but they're also doing it the wrong way.
anyways - trade with enemies / friends alike as long as they're benefits to be realized.
This reminds me of the observation that adding people to a project doesn't necessarily increase productivity that much...
Huh, I just checked stats on ecosyste.ms
It looks they consider as maintainer only those people who listed on package.json, not a real number of contributors on github or anything.
So all conclusions in this post is based on wrong assumption and incorrect data interpretation. That's all you need to know about it.
I think you could list random people on github in your package.json to looks cool in eyes of stats cultists.
that and, i would argue that npm in particular is filled with lots of small projects and only very few large ones simply by the nature of the ecosystem. it is the wrong place to look. something better would probably be to eg count the contributors on github, or, on npm, analyze project dependencies and distinguish projects that are directly downloaded vs those that are loaded as a dependency. arguably, dependencies can be replaced by the developers of the project using it, so a developer of a dependency disappearing is less dramatic than if you use that project directly.
technically speaking, if you have a large project with many contributors, every contributor is often still only responsible for one small part of the project. linux kernel drivers and subsystems most have their dedicated developers. and very few of them each.
leftpad was a minuscule project that could have been created by anyone. Yet its deletion caused chaos. There are certainly load bearing projects of moderate complexity that are still single person efforts.
right, but the problem here was the deletion of the module, not the disappearance of the maintainer. in the later case the module would have remained, and if it would stop to work because of some incompatibility in a future js, people would replace it
You could also imagine leftpad was using some security compromised library (eg log4j). If the project is of moderate complexity and there is nobody behind the wheel to maintain it, what happens to the ecosystem?
[Relevant xkcd.](https://xkcd.com/2347/)
It's interesting to see the periodic rediscovery of "capitalism + technology relies on unpaid, voluntary labour", or as the author puts it, "Open source, the thing that drives the world, the thing Harvard says has an economic value of 8.8 trillion dollars".
The one flaw that I see in the author's analysis though is that they don't seem to account for whether the packages accounted for by their source have dependents or monthly downloads. There's *a lot* of dead code out there. When excluding abandoned packages, I bet the picture is still grim, but it might be less so.
half way down the page:
> So now, let’s look at the number of maintainers for projects with over 1 million downloads this month.
Fair point, I glossed over that part a bit fast.
It does go in the direction I thought it would though. I'd be curious to see (or to take) a look a little deeper at what those thousand of packages are.
You can frame the "unpaid voluntary labor" as "creative work" and it would start making a whole lot of sense. "Creative work thrives despite being unpaid in capitalist society."
I’d state that as capitalism + technology provides enough surplus money and time that people can work on hobbies
The title of the register article is completely disgusting
> Putin on the code: DoD reportedly relies on utility written by Russian dev
then in the article:
> Hunted Labs told us that it didn't speak to Malinochkin prior to publication of its report today, and that it found no ties between him and any threat actor.
Yeah, the subtle way to plant an idea. It's a crime again to a person have "certain nationalities".
Yeah, it is pretty amazing but not surprising. The Register has taken to a certain kind of sensationalism as of late.
I found this interesting:
> "Every piece of code written by Russians isn't automatically suspect, but popular packages with no external oversight are ripe for the taking by state or state-backed actors looking to further their aims," Smith told us in an email. "As a whole, the open source community should be paying more attention to this risk and mitigating it."
Uh, I guess? The nature of open source is supposed to be that the dev provides the effort and the code, and that's where the guarantee stops. It is up to the people who uses it to implement and ensure security. People treat OSS like it is a business product that must have drop-in replacement ready at all times.
The modern nature of development is perhaps my biggest gripe as a professional. There is little care given. Projects begin with importing dozens of other packages and libraries that we never look at, let alone fully understand. And it is normalized.
Aren't Russian developers on average more susceptible to the "wrench attack" though?
Not necessarily, Australia has a law allowing the government to compel software devs to add backdoors and gag them to prevent people hearing about the backdoors.
https://scarff.id.au/blog/2023/state-actors-can-add-a-backdo...
Many of them don't live in Russia.
Some of the best engineers that I've worked with (in the US and Europe) are Russian. I've also been quite impressed with other former Iron Curtain developers. A lot of Chinese folks I've worked with have been good.
I know that some nations are known for threatening the relatives of expats, to get them to work on their behalf. Not very nice.
But state-sponsored Russian (or other nations, as well) is definitely something to be concerned about. I suspect a number of folks are concerned about the influence of American programmers. The CIA is known for using fairly innocuous employees of NPOs. My father was one.
> Many of them don't live in Russia.
Well Malinochkin does. His GitHub profile says he is located in a suburb 30 minutes from the Kremlin.
Of course, there's a lot of smart software engineers in major cities all around the world.
The FSB is looking for people they can recruit, even here, on HackerNews, too. Look at the HN news history. You will find stories about Russian history or culture. In comments, some people are expressing their fascination with Russia or its culture. This is how FSB identifies potential sympathizers, who are easy to recruit. Most likely, some of those, who expressed their sympathy under such news articles a year or two ago, are already recruited by FSB.
they would probably still fake their identity to hide their tracks.
The title of the register article is completely disgusting
They're also from Great Britain which seems to have the most irrational hatred for everything Russian.
Another case of power law distribution being all around us. I wonder how many commits of the 1M+ downloads projects maintained by more than one person, were done by just one person?