Summary: it's cultural. Rust likely inherited the practice from Nodejs, who inherited it from Ruby. I think in Rust online spaces in particular there is also this undercurrent of "you're not smart enough to use certain parts of the language, so download libraries that handle that stuff for you."
The language also takes backwards compatibility very seriously, anything that goes into std "must be maintained forever". It is also argued that a large stdlib means the maintainers have less time to work on the language itself.
I understand not every language can have Go's amazing stdlib, but I would much prefer Pyhton's approach where every now and then some package/function from the stdlib gets deprecated/removed. Rust's 3rd party ecosystem is the worst thing from the language, worse than the compile times.
The "npm-ness" of Cargo (centralized and standardized dependency management used at every opportunity) is generally pitched as one of the primary developer experience advantages over C++.
The thing is, just having a centralized standardized dependency manager doesn't in itself force your ecosystem into this dangerous "micro dependency" pattern. Java'd Maven is also a centralized standardized dependency manager, as is Python's PyPI. However, neither the Java nor the Python ecosystem have devolved into having dependency libraries with 4 methods in them becoming used and required by major libraries.
i'd like to welcome you to the hell that is c/c++ dependency management. Make? cmake? qmake? conf? autoconf? configure? autotools? submodules??? AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
I'd just select a tool, vendor the libraries I need to the codebase and call it a day. Did these for Eigen, liboption++, Catch2, Easylogging++ (now archived and not maintained anymore, sadly).
Build a simple makefile, and you're off to the races.
None of those (other than submodules which is just vendoring) does dependency management. They are more configuration management than anything. C, Python, Perl (and maybe ruby?) relies on flag switches and environment variables to find all the necessary files and modules for compiling/running a script. Cmake and autoconf just configure those.
With NPM and Rust's focus on project's level dependencies, there's no longer emphasis on API stability. Instead we have breakage every months, forcing everyone on the upgrade treadmill. It's easier to audit C library because they focus mostly on security updates instead of redesigning the API for the nth time.
All of those can (and some do) wget source code from the internet and run it if you kick off their build pipeline. People just tend to use these tools differently.
Nothing is preventing anyone from distributing Rust crates as standard system packages, like some distros do with Python.
I think Debian is actually shipping rust libs in their repositories: https://wiki.debian.org/Rust
Now Rust developers can also enjoy having to deal with popular packages that are years out of date/missing from popular repositories/getting deleted.
Of course they can be made to do anything but that's not what they are designed for and as a result they also build a culture that frowns upon doing that. Debian and other Linux distributions would have never had the policy do handle dependency resolution themselves if contemporary build systems automatically fetched everything automatically rather than being designed to use dependencies provided by the user.
Rust as a language is mostly alright in my opinion. The problems I have with it are similar to the problems I have with C++, and it makes up for them in other really compelling ways.
Where it loses me is Cargo and everything surrounding it. I'm essentially forced into an extreme where I just never use anything in the Rust ecosystem, or I have to deal with insane dependency graphs that have the density and microstate complexity of a neutron star.
As a person who doesn't like Rust very much, no, it doesn't suck. Some of its features make some folks very excited for very right reasons, and evokes "Silver Bullet Syndrome" in others for all the wrong reasons.
People weaponizing Rust rewrites with permissive licenses is another problem, but it's not about the programming language itself.
The languages that have a poor standard library support have this issue and other languages encourage you to import tons of libraries to fix the problem.
This is why Javascript and Typescript suffer from this the most and has little to nothing to do with "popularity" and likely 9/10 of these npm packages import an external library.
Golang on the other-hand is just as popular and has a stronger standard library which people build against and it is encouraged to use its standard library rather than rolling your own or importing another package to solve the problem.
Rich official standard library vs "import tons of libraries" are not the only two options.
Java's standard library had arguably also been poor for a very long time and "import tons of libraries" just had not been practical for most of that time because the tooling and ecosystem for that did not exist yet.
The solution was apache-commons and guava. Two large libraries with everything the developers heart desired and well maintained by large organizations.
For Rust be probably will never have anything exactly like that because requirements from no-std development to fully fledged backend service are too diverse, but there is still room for a small number of well
maintained backed by reputable developers convenience libraries in my opinion.
Namespacing. Mandatory namespacing. It's an essential prerequisite for any kind of reputation building and managememt, before you even get into the gritty technical or security details.
And it has to be mandatory. Top-level package names will always have more cachet. Developers are suckers for good package names, literal or imaginative. Plus it helps address, but by no means completely solves, name and typo squatting.
I understand people and groups can run their own crates.io-like repository, but that's a tangential aspect. Even if this were ubiquitous, you'd still want mandatory namespacing. You want provenance, or at least intended/nominal provenance, to be as transparent as possible, not implicit or buried. By no means a complete solution, but an important foundation for better technical and culture patterns.
It would make it easier to, as a matter of policy, to stick to components from teams like boost or Apache or Google, i.e. groups with known reputations, reputations for having long-term, active involvement and maintenance than crates put out by single individuals or startups. And if and when a project was compromised, it's more likely to be discovered in a timely manner.
To build reputations (good or bad), an ecosystem needs branding--a reliable way to connect products to an identifiable group with a history. Reputations are crucial if you're just downloading blobs of code in a fast-paced environment, because you're not directly analyzing the code, certainly not every update, and usually not even initially. It's why you don't download software from random websites to run.
Linux and BSD distributions that packaged code were trusted (to varying degrees) to vet code through their package maintainers. People apt install packages with m
less trepidation and risk than if they pulled random code off Github. The distro reputations tell you something, at least something much more credible that what availability on crates.io and github.com tell you.
Technical security processes, which are also lacking, are no substitute for social processes. They're complimentary; you need both. But the most rudimentary and important prerequisite for building more secure social processes is completely absent in the Rust crates ecosystem.
I don't want to see a large standard library for Rust. If something is added to the standard library, then it is very difficult to change it afterwards because backwards compatibility.
It would be better to have blessed crates in crates.io. The Rust core team would release or audit them. If the blessed crates need breaking changes, it can be done by increasing their major semantic version number. That can't be done to the standard library.
Actually, there could be a "trust" level for crates: 1. blessed crates by the Rust core team, 2. trusted developers, 3. untrusted developers. Or something like that..
I don't think this is true. People like to scapegoat the JS standard library but in reality most JS implementations have pretty featureful standard libraries, especially browsers. I've never felt the need to use any third-party libraries in the javascript projects I've worked on (except for maybe `ws`).
Go has a stronger standard library for certain use cases like basic CRUD web applications, but a lot of the Go standard library is also extremely low quality (flag, container, image, json, log, math, path, regexp, sync, time). Many of these aren't usable outside of toy use cases and have weird edge cases all over the place. Over time many will probably get new incompatible versions just like json. The container package in Go is the worst collection library in any mainstream programming language by a huge margin. It's actually astounding how bad it is.
> a lot of the Go standard library is also extremely low quality
Now adjust your definition of "a lot" to include all other languages instead of apparently arbitrarily deciding that "a lot" means at best 10 and pretending that "weird edge cases all over the place" isn't normal.
I also wonder what your issue is with most of those, especially since e.g. regexp is excellent in the context of this thread - it runs in linear time, so it's not possible to craft a malicious input that will make it slow down to a crawl.
> Over time many will probably get new incompatible versions just like json.
What a bizarre statement to make.
FYI: "The encoding/json package is now backed by the v2 implementation.". You get all the benefits possible from the v2 implementation for free while maintating backwards compat and if you want to upgrade to something better, v2 is right there.
Do you have some magical suggestion how this could have possibly been handled better? And no, not having json in the stdlib isn't a viable path, that's just a cop out. But I guess since e.g. Rust and Java don't have json in their stdlibs at all, you can't say their json packages are bad, how clever and smart!
Another frequently made suggestion straight from la-la land is just writing perfect code from the get-go. Brilliant, can't believe nobody's ever thought of that.
Meanwhile in the real world, you've been able to reap the benefits of the solid encoding/json for over a decade now, and with v2 you get some nice free backwards compatible improvements and have a clear path to upgrading to v2. Perfectly handled IMO.
I don't think it's just that, though I agree they are clearly correlated.
The reason I don't think it's a sufficient explanation is that there is a clear history of large, 3rd party libraries being created exactly to supplement poor standard libraries. C++ has Boost, Java has Apache Commons (though Java also has a pretty huge standard library), arguably we could even say C has Posix/Win32/Cocoa.
I believe there is some deeper cultural reason why certain language ecosystems coalesce large utility libraries, while others prefer myriad tiny dependencies.
The important difference IMO is in the small vs big libraries culture.
Lots of languages have a bad stdlib but don’t fall into the trap of having thousands of micro libraries.
The reason people do it is because it brings clout and money. Just look for articles defending micro libs: the popular ones are by people who make a living on donations, due to maintaining 1000+ packages.
And collaborating in larger libs/stdlib is hard. Plus: Rust, Node, all have a lot of visibility.
You need a good stdlib culture to avoid it (like Go did).
The reason is simple: the package manager works well. Helps if the package manager is standardized and there is a "default" registry for open source projects.
I remember the days where I had to manually put the Spring .jar files into my project. No way I am doing that for 100s of dependencies.
I've never seen a non trivial Python or Go project without external dependencies. The dependency tree of comparable Go and Rust projects seem comparable, IMO.
Pytorch has 9 transitive dependencies for cpu-only execution, or 29 to bring in cuda.
candle, the most popular rust ml library I found in a cursory search, has 119 for cpu only, and 150 to bring in CUDA.
I guess it's taste whether that's comparable
this would have been a way more satisfying dunk if nvidia hadn't split the cuda functionality needed by pytorch into 19 (!) packages on pypi but such is life.
I expected somebody to pull out an example "disproving" my point. It wouldn't be hard. I can point out rust projects that have a ton of external dependencies, and then point at similar projects with very few.
But a Python example doesn't really count, in my mind -- Python is pre-GitHub so tends to have small numbers of large external dependencies, like C++ and other older languages.
Ok! I had parsed your comment wrong. I see now that you claimed that python and go projects always had at least one dependency, and that go and rust projects had similar dependency counts, but you were not claiming that python and rust had similar dependency counts. Thanks for clarifying, with that I agree my comment was not a good counterexample.
Maybe it wouldn't matter either way if people stopped putting untrusted executable code into basic build processes.
You could get rid of a huge portion of the go standard library and have a massive ecosystem of leftpads, but the fact that `go build` can easily be run in off-line mode and doesn't execute any code other than the go toolchain itself, means you have a fundamentally more trustworthy ecosystem overall.
Rust, on the other hand, might keep you from a subset of buffer overruns, but it will gladly run obfuscated, untrusted code in your name when you wouldn't otherwise expect it to.
As an industry, we need to somehow stop making this mistake.
I do not think this is a "mistake". This is the result of different priorities.
Rust has done nothing to make me safer (although I agree that memory safety is desirable) and the supply chain issues make me less safe.
Still parts of the industry want it. If your business is putting locked down app stores and media-consuming devices in everybody pockets, than memory safety is much more important than for the rest of us.
> the fact that `go build` can easily be run in off-line mode and doesn't execute any code other than the go toolchain itself
You can trivially run Cargo in offline mode, via the --offline flag. Nothing about this capability, in either Go or Rust, results in a more particularly trustworthy ecosystem.
C has a piss poor standard library and doesn't have this problem.
Make dependencies annoying to use and it forces people to be more disciplined about them and if you do insist on a package manager you should not support transitive dependencies at all, packages should be self contained.
Transitive dependencies aren't the problem. C projects often have transitive dependencies twelve packages deep.
The problem is that these languages have repositories are both free for all and have no consumer side vetting out of the box.
Linux distributions have maintainers that vet the packages and that's why you can blindly download transitive dependencies with your OS package manager.
But you can't do the same with the AUR. The AUR workflow requires you to actually read the PKGBUILD.
There are two reasons you might want to use a crate, The first being that you want to use a good solid implementation that you know someone has spent more time doing and works better than almost any solution you could integrate. The second is you don't want to spend time implementing that.
Writing macros in rust is a pretty horrible experience but it's not difficult
The horrible practice is not writing libraries, it's having individual small packages like this crate that are handled by individual small authors, and ending up with huge webs of small dependencies.
The better practice is for bigger projects to form collecting such small utilities into larger utility libraries that have some organization, security practices, code reviews, etc. Good examples are Java's Apache Commons or C++'s Boost.
Rust is actively incorporating more functionality into the stdlib. The functionality of this crate has been in std since 2024. The ecosystem is just slow to update (not everything is maintained, etc).
Why do so many languages fall into this horrible practice?