By Joe Brockmeier
\nOctober 6, 2026
Chromium, the open-source upstream project for Google's Chrome web browser, is the browser of choice for many Linux users. It has also gained a reputation as being difficult for Linux distributions to package and build: Chromium has a complex build system, the project bundles many of its dependencies, and it has frequent releases. All of that, plus user complaints, has led the maintainers of the Gentoo Chromium package to give up on trying to maintain the package.
\nGentoo focuses on allowing users to build their own software from source using the Portage software-management tool. Gentoo packagers create ebuild files, which are text files written in a Bash-like syntax that contain the information needed by Portage to build the software. Users then build packages themselves using emerge, which is the command-line interface for Portage. A recent ebuild for Chromium illustrates the complexity of the package.
Gentoo does offer prebuilt binary packages for some software via its Binhost project. However, building from source gives users the flexibility to declare USE flags to set compile-time options or change a package's configuration. For example, a user may wish to specify which optional libraries are linked with a package, or whether to install the accompanying documentation. With Chromium in particular, a user might employ the -bundled-toolchain USE flag (which tells Portage not to use the bundled toolchain) in order to use their system's version of Clang to compile Chromium rather than the bundled version included by the upstream project. That is optional for Gentoo users on x86-64 systems, but -bundled-toolchain is required for users on other architectures, since Chromium's bundled toolchain is only shipped for x86-64.
It is worth noting that Chromium is currently not available as a binary package from Gentoo due to problems with building the package with proprietary codecs.
\nMany distributions, including Arch Linux, Debian, and Fedora, use the term "orphan" to refer to packages that have been abandoned by their maintainer; the developer announces that a package has been orphaned, and other contributors have the opportunity to claim maintainership of it if they wish to do so. Gentoo's terminology is a bit more dramatic: a package maintainer announces "last rites" for a package on the gentoo-dev-announce mailing list to notify the project that it will be unmaintained, and as an opportunity to allow other developers to claim maintainership.
\nAfter a developer has declared last rites, the package's ebuild is masked in Gentoo's ebuild repository to ensure that users will not accidentally install a package that is no longer being maintained. Users who have the package installed and attempt an upgrade will receive a warning that it has been masked. Chromium package maintainer Sam James declared last rites for the Chromium package on September 24, after a lengthy conversation in a bug report filed by Roman Žilka.
\nOn September 9, Žilka had complained that the latest stable version of the Chromium ebuild was outdated:
\n\nThe latest stable www-client/chromium (and google-chrome and possibly others) is 151.0.7922.169, which contains a catastrophic 602 known vulnerabilities of all severities. Measures need to be taken by somebody qualified that this doesn't keep happening in packages as important as these.
He had attached an ebuild for a newer version of Chromium to the bug report, but noted that it would not work if a user tried to compile Chromium with the -bundled-toolchain USE flag.
James replied, ""Chromium is horrible to maintain as a source package. It is easily the package in Gentoo that burns through maintainers the most."" Complaining about its maintenance isn't helpful, he said, and added that the maintainers have come close to giving Chromium last rites several times over the years due to the difficulty in keeping it up to date. He also noted that he had mentioned this to Žilka before, and said that it would be welcome if he wanted to assist with the package.
\nIndeed, Žilka had filed a similar bug in August complaining about the number of vulnerabilities present in the Chromium ebuild; James had replied then that the package had always been difficult to maintain, ""and now they've added Go"" as a dependency as well. Prior to that, Žilka had shown up in a bug report filed by Matt Jolly about Chromium vulnerabilities to complain that ""the update process needs fixing, so that events like these don't happen again in packages as critical as these!""
\nJolly had replied that the maintainers are a team of volunteers who do their best, but ""build failures, human factors ('I messed up assigning the bug and automation didn't pick it up'), and my own limited free time are the biggest causes of delays"". The best way to change the situation, he said, would be for Žilka to volunteer his help.
\nThere are various third-party repositories, called overlays, that provide additional and alternative ebuilds for Gentoo. On September 11, Nick Reale said that he had tried ebuilds for a newer version of Chromium from the Bentoo and Fireburn overlays, but he couldn't get either one to work. A few days later he reported success with Bentoo's ebuild and a specific set of USE flags. There was some follow-on discussion from Reale and François Valenduc about various incantations they were able to use to compile an updated version of Chromium on Gentoo from other ebuilds. On September 18, Valenduc commented that version 153.0.8010.47 of Chromium was out, and that version was already available in Arch Linux, openSUSE, SUSE, and Ubuntu. ""We are still waiting for Gentoo despite all the proposals made here.""
James replied that none of the maintainers have had a chance to look into the problems with the Chromium package yet. He said that the patches and ebuilds that had been submitted with the bug report were ""useful and appreciated, but it's not as simple as just grabbing it and committing it"". Users would complain, he said, if the maintainers had not tested the ebuilds and it broke on their systems.
\n\nIt'll get done when it's done, but it's more likely to be last-rited if nobody is interested in handling it in the way these things have to be done (which goes beyond just attaching ebuilds).
Valenduc said that he was not able to maintain the package, ""but taking the ebuild from the bentoo overlay was not that difficult [...] It's just frustrating to see no updates in Gentoo."" James replied that just because the ebuild worked for Valenduc did not mean it had all USE flags working. He added that no one had submitted a patch to the Gentoo ebuild, which meant that the maintainers would need to reverse-engineer the changes by diffing the ebuilds and work out what had been modified. ""All of that is a non-trivial amount of work and while having some ebuild to start with is useful for that, it doesn't mean one can just git add && git commit && git push it."" Valenduc acknowledged that he could not test all possible USE flags, since it took between 15 and 16 hours to compile Chromium on his laptop.
On September 24, Žilka commented: ""This is ridiculous"". A few minutes later, James replied ""OK then"", and issued the last rites for the package.
\nMaciej S. Szmigiero asked if it would be possible to keep the ebuild in Gentoo's repository and just declare that it was maintained at "best effort" level. ""Removing buildable/patchable Chromium is a huge loss for a Gentoo desktop."" James said that he did not think that was possible. He had already explained that Chromium was being maintained by volunteers, and that explanation had not been accepted.
\nPeople were free to maintain the ebuild in another repository, he said, but that meant they would have to do the work themselves or accept someone doing it to a lower standard. He pointed out that the ebuilds from the Bentoo overlay had been cited in the discussion, but creation of those ebuilds seemed to be ""heavily automated"" and had missed some fixes in the Gentoo version. ""Copying ebuilds works fine until it doesn't.""
\nJames added that someone could still pick up the Chromium package before the end of the last-rites period on October 24, or that the package could be restored in the future. He did not want to see that, though, unless there was a plan for sustainable maintenance. ""Chromium has a history of burning out maintainers in Gentoo because of the high rate of churn upstream, though some of the contents of this bug don't help motivation at all either.""
\nMatt Whitlock wondered if the developer-burnout factor could be reduced by eliminating most of the USE flags for Chromium, or if the maintainers stopped unbundling third-party libraries.
\nBasically, what I am asking is: can we give up all of the flexibility to which we have grown accustomed in www-client/chromium, in exchange for a package that is actually maintainable but that is still built from sources by a toolchain that is not distributed as opaque binaries?
He added later that part of the problem was that the upstream project developers ""do not test their builds with any of their allegedly 'optional' features disabled"". That left the responsibility with the downstream package maintainers. ""In the case of Chromium, keeping all the toggle switches working (despite chronic neglect from upstream) is a monumental, nigh-insurmountable task.""
\nThat was a good question, James said, but part of the recent complexity with Chromium was the addition of Rust and Go dependencies. The packagers had to do some wrangling to ensure that the Go build did not require network access, which is forbidden with Gentoo ebuilds, but that was not the only problem. He said that Google developers seem to keep using experimental Rust features that break on new releases as well. On top of that, ""yet another problem is that upstream's CI to produce release tarballs isn't reliable"". One of the Gentoo Chromium maintainers had sent patches to fix that, to little avail. ""It took ages to get [the patches] reviewed, and it's broken again a few times since. They didn't seem to notice or care that it either wasn't producing tarballs or was producing incomplete ones"."
\nJolly added some of his observations about problems with maintaining Chromium for Gentoo. He said that maintaining Chromium can take more than six hours each week, and that was counting only compilation time on a fast PC with ""an absurd amount of RAM"". Compilation time is even longer for 64-bit Arm machines.
\n\nIt's a commitment to volunteer at least a business day's worth of your free time each week when you include rebasing patches, cherry-picking fixes, iterating on
ebuild ... clean compile, plus trying to identify and fix any new bugs, working out how much you violate Gentoo policy by not unbundling more of the browser, spending time on automation to save time and mistakes later, etc. There's immense pressure to keep builds up-to-date, especially in the age of hundreds of weekly browser CVEs.
He included a ""pre-emptive FAQ"" in his reply about the last rites for Chromium, and what it would require to restore the package. He said that there were no insurmountable technical issues to maintaining Chromium, but it would need multiple maintainers with a broad set of skills—including familiarity with C, C++, Go, JavaScript, and Rust. ""There are also a huge number of NIH [not invented here] Google tools that you'll [have to] develop skills for and never be able to use anywhere else - you have been warned"."
\nUsing Google's toolchain would be simpler, but it would also restrict the ebuild to x86-64, he said. ""We'd lose arm64, ppc64, and the RISC-V overlay - better to just rip the band-aid off now as it doesn't materially make the situation better"". He added that he would be willing to review commits and offer advice to anyone who wanted to maintain the source ebuild, but he felt that at least three people would be needed. ""History has shown that one maintainer isn't working, and we've had the release cycle shorten twice since I started maintaining the package in 2023; it was bad then, and it's worse now.""
\nThe release cadence for Chromium is aggressive, to say the least. Chromium has multiple channels including stable, beta, and development. There is a new major branch of the browser roughly every two weeks via the stable channel, and a new version is shipped every week.
\nFor now, it appears that Gentoo users will need to search elsewhere for a suitable Chromium ebuild, choose a different browser, or find a different distribution method to acquire Chromium. For example, Gentoo users could install the Flatpak from Flathub, but it is an unverified package built by a third party—not by the Chromium upstream project. The Flatpak is also of no help for users who are not on x86-64 or 64-bit Arm systems.
\nIt is technically possible for Chromium users to build and compile the browser on their own using the upstream's instructions, but it does not appear that the Chromium project cares a great deal about making that easy to do. It does have a page on getting the code but the links to instructions for Linux (as well as macOS and Windows) return a 503 error. Searching for instructions turns up this page, though it is undated so it is unclear if it is the most up-to-date. Assuming the instructions are current and correct, building Chromium using those directions is quite an ordeal. Users can download pre-built binaries from the project, but only for x86-64 systems.
\nOther distributions continue to supply Chromium packages, of course, though their approach does not provide the same flexibility as Gentoo's. Fedora's package (which is described as ""A WebKit (Blink) powered web browser that Google doesn't want you to use"") is at version 154.0.8037.92—one version behind the upstream (154.0.8037.97) release that was published on October 1. Likewise, the package in Debian's security repository is the same version (currently) as Fedora's and the information page shows that there are at least 11 known CVEs addressed in 154.0.8037.97. Keeping up with Chromium is not an easy task.
\nPerhaps Gentoo will find a team that is willing and able to take on the burden of maintaining the Chromium ebuild up to the standards that Gentoo users expect. Whatever happens, it seems unlikely that Chromium will become any easier to maintain in the interim.
\nto post comments
\nPosted Oct 6, 2026 14:38 UTC (Tue) by sam_c (subscriber, #139836) [Link] (12 responses)
\nThank you for covering this, it's challenging and we didn't want it to come to this, but it felt like the writing had been on the wall for quite some time.
\nOn bundled dependencies: one additional issue is that if we were to go "fully (Google) native" and eschew system libraries in favour of Google's bundled copies, the costs would in some way become greater: longer build times, unhappy users, greater portability issues (consider non-amd64), and losing out on features like distcc compatibility which in part presently mitigates the build time suffering.
\nThat is, at a point, are we only performing a "from source" build on the face of it, but not to the standards users have reasonably come to expect? But even this is rather taxing to maintain.
\nFrom my survey of other distributions, it appears they're often also behind by at least one major version (perhaps made worse by the release cadence change), and they don't have the permutations to handle that we do: it only has to build on the set of architectures $DISTRO supports, and it only has to build with one set of build options on a particular builder machine, rather than on all machines from users. I think we're all suffering.
\nPosted Oct 6, 2026 14:52 UTC (Tue) by collinfunk (subscriber, #169873) [Link] (5 responses)
\nThanks for your work in Gentoo, Sam.
\nI wonder, is packaging other browsers (I guess, only Firefox, really) similarly challenging, or are these issues unique to Chromium? From my perspective as an outsider, it seems Firefox is equally massive and riddled with security issues. But perhaps they make it easier to build and to release less frequently?
\nAnecdotally, I have tried to make the projects I work on easy for downstream maintainers. But, then again, I don't work on anything the size of Chrome.
\nPosted Oct 6, 2026 15:52 UTC (Tue) by sam_c (subscriber, #139836) [Link] (2 responses)
\nThank you Collin!
\nFirefox is also challenging but Mozilla seems to care at least a bit about downstreams being able to consume Firefox's sources: you don't have the disappearing or incomplete source tarballs, they're generally receptive to patches, and they seem to help at least sometimes (I can't say always because I'm not as familiar with it, but it doesn't mean they aren't) when folks get stuck. Firefox increased its release cadence a few years ago and maybe even more recently again, but I believe the changes on the build side are usually quite small. The biggest problems we have there involve some small components which sometimes break with newer Rust (SIMD I think).
\nqtwebengine benefit from Qt's efforts to wrap it in something sane, so is more tolerable than Chromium. But it's often behind Chromium by a fair bit, though they do backport (some? again, not certain if it's all; not claiming it is definitely not all) security fixes.
\nwebkit-gtk is also mostly okay though it does take a significant time to build (same for all of these) and we have to build several variants right now for some ABI changes, which is unfortunate.
\nPosted Oct 6, 2026 18:03 UTC (Tue) by sam_c (subscriber, #139836) [Link] (1 responses)
\nI was under the impression (I think wrongly?) that anonymous comments could be made with subscription links but it looks like not.
\nPosting juippis's comment on his behalf then:
\n\nHey, sam asked me to fill in as Gentoo's current Firefox maintainer. Firefox is a lot of work, but I'd like to share few points why I believe it's easier than Chromium. Note that I have close to 0 experience packaging Chromium, I've just aided with some updates previously.
\nI could write a long detailed post about everything I do with each Firefox release, but since I'm a guest here, I'll try to keep it short.
\nTLDR as a whole is:
\nDoing the version bumps properly takes a lot of time - mostly because I test with multiple USE flag combinations, gcc/clang, and simply testing pgo itself takes like an hour to build once. It can be hard, because Gentoo carries few custom patches, e.g. to enable different system-* libraries to be used instead of bundled ones, and those regularly break / need patch rebasing. But upstream is rather welcoming, and bigger issues affecting the whole Linux ecology are usually fixed fast by upstream. Upstream also has a CI building using Ubuntu. We usually find toolchain / system-lib related issues amongst the first ones due to the nature of Gentoo being a rolling-release distro, with newer toolchain available. Some of the Firefox developers seem to use Gentoo, too. Therefore I'd say the relations between upstream and other distros is stronger with Firefox, especially since Firefox is often the default browser in other distributions.
\nGentoo offers "two channels" - Firefox-157.0 with testing keywords, and Firefox-153.4.0esr with stable keywords. Thunderbird is identical; Thunderbird-157.0 and Thunderbird-153.4.0esr. Thunderbird releases come immediately after Firefox and sometimes requires TB-specific patches to be added. Firefox recently switched to 2-week release cadence, where it used to be 4 weeks, so the work is now doubled each month. It currently takes me ~20 hours every 2 weeks to do everything related to version bumps (both Firefoxes, both Thunderbirds, Spidermonkey, security bugs...) and I've even skipped a step recently, and am pushing ESR's straight-to-stable (after testing on stable system) to save some time. Most of the hours go into running build-tests, which can be done by leaving the pc to run on its own. There's I'd guess ~3-4 hours of "active" work (without bugs) whenever major releases happen. So there's one less active channel to monitor than what Chromium currently has, but I think handling Thunderbirds make it equal here.
\nGentoo also offers different variety outside use flags: you have musl, libcxx, different architechtures (arm, x86, riscv) that all require their own quirks. Luckily the power users with these setups know to provide patches, which I then test on glibc and add them if it's ok. Any rust-related issues are somewhat hard to debug, and you can bet there's plenty. Chromium has similar challenges there, but I'd say the nature of Chromium's bundled toolchain vs. system toolchain makes it harder for everyone even with "normal" setup.
\nOverall Firefox takes a lot less time to build than Chromium which makes it more bearable. And as sam said, the build system changes affecting us between major releases are usually rather minimal. As the sole Firefox maintainer in Gentoo, I also do feel like there's a bit more community support for Firefox. Often times when a bug for Firefox in Gentoo is filed, I'm not even the first one to reply there. And through collaboration there's been cases where we discovered a bug in GCC directly, which was then fixed in GCC (by GCC developer sam) :)
Posted Oct 7, 2026 11:10 UTC (Wed) by rwky (subscriber, #119998) [Link]
\nAs a Firefox Gentoo user I want to say thank you for all the effort put in! I've not had a firefox build fail once (though I have had it OOM when I forget to close stuff). I've not used Chromium (I use the google chrome bin package instead when I'm forced to use Chrome for slack huddles). But I do use qt-webengine in some packages and whenever I see an update I do groan knowing that it'll take hours to compile.
\nI can understand giving chromium last rites considering how time consuming qt-webengine is it must be a nightmare to compile/test. I'd rather it was shelved and people were encouraged to switch to more developer friendly options.
\nPosted Oct 6, 2026 15:59 UTC (Tue) by NHO (subscriber, #104320) [Link] (1 responses)
\nQuick run of cloc tells me that chromium has 50% more code in ebuild than firefox, and looking at git blame tells me that changes in code responsible for building firefox happened in versions 81, 105, 121, 130-133 (rust-related), 137 (testing) and some in 150. In last ten versions, only chages in ebuild are dependency-related (as in, new versions happened and it stopped depending on old libraries and rust).
\nThis is not precise information, and I haven't looked at chromium ebuild like that.
Posted Oct 6, 2026 16:02 UTC (Tue) by NHO (subscriber, #104320) [Link]
\nI have not looked at firefox patchset, mind, only on raw ebuild.
\nPosted Oct 6, 2026 16:02 UTC (Tue) by DemiMarie (subscriber, #164188) [Link] (5 responses)
\nWould a package that worked as follows be sustainable?
\nThat starts to sound a lot like Canonical’s Chromium snap, though.
\nTrivalent would be a good choice, except it only ships RPMs and is only tested on Secureblue.
\nPosted Oct 6, 2026 16:25 UTC (Tue) by NightMonkey (subscriber, #23051) [Link] (1 responses)
\n"Trivalent would be a good choice, except it only ships RPMs and is only tested on Secureblue."
\nThat would make it not a good choice for Gentoo. :)
\nThe odd tone of this message makes me think it is LLM-generated. Apologies if it isn't. But, if it is... please don't do that here! :)
\nPosted Oct 6, 2026 17:00 UTC (Tue) by ttuttle (subscriber, #51118) [Link]
\nFWIW, I didn't get an LLM vibe, mostly because it was very succinct. Some people just really like bulleted lists!
\nPosted Oct 7, 2026 20:49 UTC (Wed) by NAR (subscriber, #1313) [Link] (1 responses)
\nIn this case why bother with building it and why not simply use the binary downloaded from Google (if it exists at all)?
\nPosted Oct 8, 2026 8:14 UTC (Thu) by smurf (subscriber, #17840) [Link]
\nA lot of reasons … known provenance. Less (if any) bundled libraries that already exist on the system. Wider arch support. Debuggability. Removal of anti-features. And so on.
\nPosted Oct 11, 2026 2:15 UTC (Sun) by Kangie (guest, #161139) [Link]
\nUnfortunately no.
\nGentoo ebuilds must run offline - it's a requirement for air gapped networks (etc.)
\nCross compilation is reasonable - it's how upstream produce binaries, but at that point we're talking about building a full set of Chromium binary CI infrastructure.... which is on upstream to provide; we'd happily consume the binary but they don't want it distributed that way to compete with Chrome.
\n\nAutomate not only checking when a new version is released, but the entire fetch-build-upload process. This means that updates are shipped with no human intervention
Part of distro packaging is doing the testing and validation before pushing out a package to your users; you can automate and abstract a lot of that but eventually you need to test your package (or risk shipping a horrible regression).
\nPosted Oct 6, 2026 15:40 UTC (Tue) by Curan (subscriber, #66186) [Link] (1 responses)
\nThe two major browsers are beasts. And nasty as hell to package.
\nI really hope the same fate will never reach Firefox in Debian and I can't thank Mike Hommy enough every day for his work. And often overlooked: the same applies to Thunderbird and its two maintainers (Carsten and Christian).
\nThough, admittedly, part of the issue seems to be the flexibility of Gentoo here.
\nPosted Oct 7, 2026 8:48 UTC (Wed) by gspr (subscriber, #91542) [Link]
\nHear, hear! The Firefox packagers are true unsung heroes.
\nPosted Oct 6, 2026 15:44 UTC (Tue) by cen (subscriber, #170575) [Link] (52 responses)
\nI firmly believe that distributions with strict anti-AI policies will eventually be left behind (Gentoo seems to be one of them) while distributions with sane AI policies will be able to keep pace with the modern craziness that is software packaging.
\nI am not saying distros should start vibe coding but when you pair an experienced package maintainer with a frontier AI agent the maintenance burden becomes much less painful and a lot of things can be done that were previously not possible. Because as it was mentioned in the article, experts in C++, rust, golang and packaging are not just behind every corner but your friendly (or evil, depending who you ask) agent knows all of that and is approaching senior dev levels if not there already.
\nI've already seen first hand certain distro packages being able to advance past years of deadlock and failure with some help from AI and that is a good thing.
\nPosted Oct 6, 2026 15:48 UTC (Tue) by sam_c (subscriber, #139836) [Link] (3 responses)
\nSomeone is free to do that in another repository with LLM assistance. So far, the only attempts seem to be done somewhat blindly with LLM assistance rather than "expert use".
\nIt also doesn't help the build time issues or lack of cooperation or interest from Google with say, the tarballs missing for periods of time and having to produce our own.
\nMatt Jolly already had used LLMs to write tools for handling some aspects of Chromium, but not the ebuilds and not patches themselves, but e.g. release notifications and aggregating fixed vulnerabilities. I think it extended the time for which keeping Chromium was viable. So maybe you're right in that regard, but not right enough.
\nPosted Oct 6, 2026 17:29 UTC (Tue) by Baughn (subscriber, #124425) [Link] (2 responses)
\nThe Gentoo experts don't want / aren't allowed to use AI. Of course the results would be as you see, but that doesn't have much bearing on how it could work.
\nPosted Oct 6, 2026 17:46 UTC (Tue) by sam_c (subscriber, #139836) [Link]
\nWe have plenty of expert users, though. I'm not sure with this thinking that I could say anything to refute this claim.
\nPosted Oct 11, 2026 2:41 UTC (Sun) by Kangie (guest, #161139) [Link]
\nI can say with some degree of certainty that using LLMs wouldn't fix the current batch of build issues.
\nI don't have the depth of Rust or C++ knowledge to dig into Crubit and work out how to make it work in a distro context with only stable and released LLVM and Rust for example, and I couldn't trust the output without a lot of validation (and probably spending a lot on tokens for what's supposed to be volunteering.)
\nThere's definitely a place and time for the technology, but it's not the solution to every single problem. I'll continue to use LLMs where they make sense, but they're not a silver bullet either; they don't save me from 6+ hours of compilation a week, or from having to actually think about problems.
\nPosted Oct 6, 2026 16:31 UTC (Tue) by NightMonkey (subscriber, #23051) [Link] (6 responses)
\nWho will pay for the token use in a volunteer-run distro?
\n"I've already seen first hand certain distro packages being able to advance past years of deadlock and failure with some help from AI and that is a good thing."
\nWhich ones? And how did "AI" help in those cases?
\nAn important aspect to consider is also that a distro is not just it's output. It is also the combined knowledge and wisdom of it's maintainers, and the 'pipeline' that brings on new maintainers. If an LLM is doing all the 'under the hood' work, then the "knowledge" becomes centered within the LLM, not the maintainers, if you're not careful.
\nCheers.
\nPosted Oct 6, 2026 16:46 UTC (Tue) by smurf (subscriber, #17840) [Link] (2 responses)
\n\nWho will pay for the token use in a volunteer-run distro?
Most likely, the volunteers. Or their employer. Or some other company who donates GPU time + power.
\nDebian has already acquired one sponsor for inference; there's no reason other distros couldn't do that also.
\n\ncombined knowledge and wisdom of its maintainers, and the 'pipeline' that brings on new maintainers
That's one aspect. The other is that, frankly, reconciling patches and writing packaging rules, no matter which distro, is a rather tedious process, and I for one am very happy about the fact that the LLM in my data center handles those chores for me.
\nYou still need some expertise, to determine when (not if …) the LLM's output is pure AI slop, of course. But (a) I am not sure that slogging your own unaided way through the process is the best way to gain that, (b) recognizing slop and knowing the intricacies of exactly which stanza does what are not the same skill set.
\nNB: *its. (Both times.) Please.
\nPosted Oct 6, 2026 18:18 UTC (Tue) by ballombe (subscriber, #9523) [Link] (1 responses)
\nIf you need a LLM to write packaging rules, then upstream must be doing something wrong...
\nPosted Oct 7, 2026 7:37 UTC (Wed) by taladar (subscriber, #68407) [Link]
\nI mean lets be honest, it is Chrome, upstream is doing pretty much everything wrong that they could in terms of making packaging easier/harder for downstream packagers.
\nPosted Oct 6, 2026 20:00 UTC (Tue) by cen (subscriber, #170575) [Link] (2 responses)
\n\nWho will pay for the token use in a volunteer-run distro?
\nNot sure if this is relevant but I guess the same process that pays the infrastructure - sponsors, donations, out of your own pocket (personally). Volunteering is already not free, nothing is.
\n"I've already seen first hand certain distro packages being able to advance past years of deadlock and failure with some help > from AI and that is a good thing."
\nWhich ones? And how did "AI" help in those cases?
One random example I saw on a mailing list, Debian's gradle package has been stuck at version 4 for years with many people trying to update it and giving up. Recently someone threw AI at it and managed to get a few minor versions further than before. I'm sure there are and will be many more examples out there in the future.
\n\nAn important aspect to consider is also that a distro is not just it's output. It is also the combined knowledge and wisdom of >it's maintainers, and the 'pipeline' that brings on new maintainers. If an LLM is doing all the 'under the hood' work, then the >"knowledge" becomes centered within the LLM, not the maintainers, if you're not careful.
\nCompletely agree. I think we are doomed in that regard. Old guard will start losing knowledge due to over reliance on AI and the convenience factor while the new guard will never acquire the same level of knowledge unless they really care for some reason. Some say that art of software development will slowly be lost which I think is entirely possible. When AI becomes good enough, we become the assembly line workers replaced by robots.
Posted Oct 6, 2026 20:54 UTC (Tue) by pizza (subscriber, #46) [Link] (1 responses)
\n\nNot sure if this is relevant but I guess the same process that pays the infrastructure - sponsors, donations, out of your own pocket (personally). Volunteering is already not free, nothing is.
Excellent, now we'll have even fewer volunteers to maintain all of this crap.
\nMission accomplished, I guess?
\nPosted Oct 6, 2026 21:53 UTC (Tue) by rahulsundaram (subscriber, #21946) [Link]
\n\nMission accomplished, I guess?
Indeed, the goal isn't to help volunteers necessarily. The goal is to make the LLM selling companies more valuable. If volunteers benefit by the currently heavily investor subsidized subscriptions, that is just a temporary cost that LLM selling companies are willing to tolerate to boost their profile that can come out of their growing marketing budget. Nothing to see here, folks.
\nPosted Oct 6, 2026 17:24 UTC (Tue) by notriddle (subscriber, #130608) [Link] (8 responses)
\nBut why is compiling Chromium so difficult in the first place? According to sam_c, Firefox isn’t as bad. Other projects that I would expect to be comparable, like OpenJDK, have caused plenty of drama around bundling, but they still manage to ship working release tarballs.
\nPosted Oct 6, 2026 18:17 UTC (Tue) by smurf (subscriber, #17840) [Link] (2 responses)
\nProbably because the Google devs don't bother, they have their own infrastructure with precompiled dependencies and whatnot.
\n\nworking release tarballs
Tarballs are not source code. Not since the last decade. A git tag or commit ID, and all that hangs off it, is source code. (Source code: the thing you touch when you modify it.)
\nPosted Oct 7, 2026 6:50 UTC (Wed) by pbonzini (subscriber, #60935) [Link] (1 responses)
\n\nSource code: the thing you touch when you modify it.
No. The preferred form for making modifications. You don't modify the whole commit graph, nor do you need it: it's perfectly possible to make modifications after "git clone --depth 1".
\nSo tarballs are perfectly fine as source code. And if they were not, all Linux distributions would be violating the GPL/LGPL.
\nPosted Oct 7, 2026 7:08 UTC (Wed) by mb (subscriber, #50428) [Link]
\nIt will not be too long until somebody claims that removing an AGENTS.md from a forked GPL project is a license violation, because it takes away the preferred form of modification. ;)
\nPosted Oct 9, 2026 5:11 UTC (Fri) by ssmith32 (subscriber, #72404) [Link] (3 responses)
\nBecause The Googly Way is to have a gianormous monorepo that bundles all dependencies in the build. It's a little nuts, but it's what they do there. They're so deeply committed, that when it started blowing up on them, they concluded that the correct path forward was to spend a bunch of time/money developing a special build system (Bazel) to handle the situation, instead of, you know, doing it like most people do.
\nAs you traverse the industry, most ex-Googlers realize it's just an odd quirk of the company, but a few believers try to carry it over to new Company X... And it rarely works out well. Although I think redfin managed to duplicate the idea, for various definitions of success. And Microsoft Office also did at some point, although, I don't know if the reported day-long builds count as "successfully executed".
\nI had one manager (at old company) that kept trying to convince our team to do it. He kept excitedly reporting how his old company had embraced a monorepo, again and again. Maybe he thought the blank looks he got in response was an indication that he just needed to repeat himself again, and people would go "ah-hah, I see it now". No one ever did.
\nPosted Oct 9, 2026 10:05 UTC (Fri) by taladar (subscriber, #68407) [Link] (2 responses)
\nDoesn't help that some people apparently now also use the term monorepo to refer to repositories with a couple of closely connected libraries or programs (e.g. all Rust crates related to tracing) while those old style monorepo companies really were insane enough to put all software they wrote into a single repo. A lot of people don't realize the true horror of the latter approach since they assume when the term is mentioned they mean the former.
\nPosted Oct 9, 2026 23:19 UTC (Fri) by Cyberax (✭ supporter ✭ , #52523) [Link] (1 responses)
\nI worked in companies that do tons of packages (one even had its own internal packaging systems) and with companies that do monorepos.
\nOn the balance, monorepos were better from the organizational standpoint. Packages tend to result in dependency hell on every upgrade, with all the issues of ownership, maintenance, etc. Monorepos centralize dependencies, making it easier to do tree-wide changes.
\nI'd say that the main issue with monorepos is tooling. Git is just not designed for that.
\nPosted Oct 11, 2026 14:20 UTC (Sun) by Wol (subscriber, #4433) [Link]
\n\nI'd say that the main issue with monorepos is tooling. Git is just not designed for that.
This is why, when I designed the project management system I was working on many moons ago, it was an explicit design goal that the only thing the parent project knew about the subproject was the standard task data. To the parent, the sub was an opaque blob that filled in the standard "this job" task info, and in return read the standard parent task data to find out what resources it had available.
\nLooking at the fun with SHA-1 / SHA-256 in git, I now realise that was probably one heck of an ask (but the company nearly sank before I found that out).
\nIf we want a monorepo, we really need to provide some way of telling git "this directory contains a repo with its own git structure, DON'T TOUCH". And some way of exporting the status of that sub-repo up that simply says "this is the live head", with a matching mechanism (maybe manual) of updating the sub-repo to its parent to report "this is the upstream live head" and maybe triggering the subrepo to do its own update/rebuild. Problem is, I think this is going to be a rather tricky problem ...
\nCheers,
\nWol
Posted Oct 11, 2026 2:24 UTC (Sun) by Kangie (guest, #161139) [Link]
\n\nBut why is compiling Chromium so difficult in the first place?
Recently ex-downstream package maintainer here.
\nThere are a few reasons.
\nPosted Oct 7, 2026 8:52 UTC (Wed) by gspr (subscriber, #91542) [Link] (31 responses)
\nIf you need an AI to maintain the package going forward, what's the preferred form of modification for said packaging (e.g. for the ebuild or for debian/*)? Something, at the very least prompts and "dialog", probably also the model's weights, are now needed to continue modifying it in the future. This seems antithetical to the basic principles of most distros.
\nPosted Oct 7, 2026 12:40 UTC (Wed) by stijn (subscriber, #570) [Link] (30 responses)
\nAI can just help with overcoming a little hurdle in some arcane config, macro or glue language, or point out a difficult to spot pitfall. It is the same as spending a few hours trawling stackoverflow answers or related, but usually compresses into seconds or minutes and a much better interactive experience. Just as you would not need to include your web search experience and parameters, one need not include a prompt. I am ignoring a few elephants in the room, e.g. the demise of stackoverflow traffic as AI usage dominates, freedom of access, control of resources; this is just about the role of a prompt. For better or worse, AI agents are now like highly informed colleagues with generally good taste (not infallible of course). Any learnings can be included in the code / script / documentation. In these cases, AI lowers access and activation thresholds and eases what can be frustrating parts of work.
\nPosted Oct 7, 2026 20:26 UTC (Wed) by NightMonkey (subscriber, #23051) [Link] (29 responses)
\n\nIt is the same as spending a few hours trawling stackoverflow answers or related, but usually compresses into seconds or minutes and a much better interactive experience.
It is not the same as that.
\nIn the case of a few hours trawling Stackoverflow, you might learn some useful things you didn't know - even about things tangential to your problem at hand. In those minutes with an LLM, you often only learn that the LLM "knows" and your own understanding is not increased. You also don't get the social and 'gamified' experience of stackoverflow. (And yes, you also don't get slapped by Stackoverflow users, either.) And, of course, you might be inspired to actually answer questions that other humans have, and yours has a chance of being accepted as a good answer.
\nLLMs certainly do not improve the experience of those who answered questions on the StackExchange sites... who now get zero credit for their work. :(
\nFor better or worse, StackExchange is a community of humans. LLMs, as used by big-tech capitalists, are just knowledge vacuums that abstract away the humans who actually shared the knowledge.
\nPosted Oct 8, 2026 8:27 UTC (Thu) by kleptog (subscriber, #1183) [Link] (28 responses)
\n\nIn the case of a few hours trawling Stackoverflow, you might learn some useful things you didn't know - even about things tangential to your problem at hand. In those minutes with an LLM, you often only learn that the LLM "knows" and your own understanding is not increased
Really? That's not my experience at all. I learn new things from LLMs all the time. I learned this yesterday from a bash script it generated:
\nnext=$(printf "%02d" $((10#$h + 1)))
\nDid you know that bash can interpret numbers in different bases? I didn't. The LLMs sometimes come up with different solutions than I would have but it's always fascinating to see what it comes up with. Most of the time they just produce the code I would have written, just faster.
\n\nLLMs certainly do not improve the experience of those who answered questions on the StackExchange sites... who now get zero credit for their work. :(
Is that the only purpose of SO, so people can get credit? SO only worked in so far as Google could correctly map your question to a SO question, which it did pretty poorly most of the time. That's why it took hours.
\nIt seems like you are arguing: you should use crappy tools because in the time wasted you learn more than using helpful tools. Instead of just giving people a hammer you give them a pile of junk and they have to figure out which objects will work best on a nail.
\nPosted Oct 8, 2026 11:26 UTC (Thu) by pizza (subscriber, #46) [Link] (5 responses)
\n\nIs that the only purpose of SO, so people can get credit?
That's one of the main reasons people answer stuff on SO.
\nWTF do you think the LLMs have been getting their answers?
\nPosted Oct 9, 2026 8:24 UTC (Fri) by taladar (subscriber, #68407) [Link] (3 responses)
\nI can't speak for others but the gamification has been a major reason for me to never try to answer anything on SO.
\nPosted Oct 10, 2026 1:33 UTC (Sat) by mathstuf (subscriber, #69389) [Link] (2 responses)
\nI don't think it was the gamification itself (for me). It was the enforcers of the rules looking for those seeking Fake Internet Points™ by any means to enact punishment. I think they cried wolf far too often, on balance.
\nI've had a better answer to a common SO question for quite some time that I just never bother posting because the current answer is so up-voted to drown my better one out and I don't feel like arguing about it. For those curious, it's that the common "pipe the output of set after running vcvarsall.bat to get the environment you need" fails to account for newlines in values. An environment variable with the value foo\\nPATH=hahaha will happily overwrite PATH much to everything after that's sadness. Discovered when an MR description injected environment variable settings into CI that ruined things. The solution is to invoke PowerShell and dump env: as a JSON document and then recover it on the outside as JSON. Go forth and karma farm if you want.
Posted Oct 12, 2026 7:11 UTC (Mon) by taladar (subscriber, #68407) [Link]
\nThe enforcement certainly didn't help but for me gamification acts as the kind of extrinsic motivation that completely kills any intrinsic motivation and since I don't care about SO points that means no motivation to do anything there is left.
\nPosted Oct 12, 2026 19:55 UTC (Mon) by madscientist (subscriber, #16861) [Link]
\nI helped people on SO for years (over 2800 answers). Then, in spring 2025 a few of my answers were edited in ways that I felt reduced their accuracy and usefulness, so I reverted the change. Then someone reverted my revert, and I undid it again. I should point out that my answers had been edited before and I never objected, even when I didn't fully agree with the edit. But this change really annoyed me (honestly I can't even remember the details anymore).
\nI had a discussion with some sort of moderator about this, who then suspended me for a week for arguing with them.
\nI've never even logged into my account again, much less answered questions there.
\nPosted Oct 11, 2026 6:01 UTC (Sun) by ras (subscriber, #33059) [Link]
\n\nThat's one of the main reasons people answer stuff on SO. WTF do you think the LLMs have been getting their answers?T
They are definitely biased towards "official" sources - man pages, documentation, RFC's, Python PEP's and the writings and reasoning of people whose names appear on those documents. For example, if you ask an AI (ChatGPT, Claude, Gemini - I tried them all) to give you the references they used for "does bash allow numbers in bases other than 10" they to the man refer you to https://www.gnu.org/software/bash/manual/html_node/Shell-... Stack overflow is definitely ranked below sources like that. I'm suspect it's ranked below most Wikipedia articles too.
\nMy guess is what they get from sites like Stack Overflow is how themselves clearly to software engineers. Don't underestimate the value of that - how to express themselves clearly to mathematicians was notable by absence is the recent Mathocalypse paper dump.
\nPosted Oct 8, 2026 15:38 UTC (Thu) by NightMonkey (subscriber, #23051) [Link] (18 responses)
\nYou do understand that, by definition, LLMs do not create anything new. Nothing. They cannot "innovate". They only synthesize 'answers' from the corpus of data that has been scraped for them. You think 'it' is teaching you something. It is not. The original author(s) of those ideas are teaching you. But you just don't see that because the LLM service owners don't care about plagiarism, or even copyrights.
\nPosted Oct 8, 2026 16:02 UTC (Thu) by marcH (subscriber, #57642) [Link] (2 responses)
\nHuman "innovation" is mostly made of (re) combining existing things too. That's why there are often several people inventing more or less the same thing at the same time (one per country/culture...)
\nThere are differences but good luck drawing a line that simple and clear between the two.
\nPosted Oct 8, 2026 16:07 UTC (Thu) by NightMonkey (subscriber, #23051) [Link] (1 responses)
\nYes, sure, I agree with your point about innovation - it is just humans describing what exists in nature, albeit sometimes in a novel way. However, it isn't always useful to try to draw a line. It can sometimes be more useful to consider the contradictions together as a system or a dynamic.
\nHowever, LLMs do not innovate in that way. They do not 'describe nature' - they are statistical engines that work with probabilities. They don't have a consciousness to make observations with. And the problem with assigning them agency in this way is that it makes it much easier to pull the wool over people's eyes about how they are being used by humans.
\nPosted Oct 8, 2026 16:20 UTC (Thu) by marcH (subscriber, #57642) [Link]
\nI don't think Nature matters much here.
\nI agree LLMs are not conscious but I don't think anyone mentioned that either before. Another tangent?
\nTrying to go back to the initial topic(s): yes LLMs can totally outclass search engines in some contexts. And yes you learn much less when someone gives you the answer as opposed to struggling - but you don't learn nothing either. What's new here? Derek Muller has a great talk about how no technology has been evolving this learning trade-off despite all of them promising it.
\nPosted Oct 8, 2026 16:09 UTC (Thu) by marcH (subscriber, #57642) [Link]
\nLLMs have been solving old maths problems left and right. If that's not "creating" anything, then most mathematicians have not been creating anything either...
\nWhat is creativity? There is simply no formal definition. People often agree that A is more creative than B but that's it; it's relative.
\nPosted Oct 8, 2026 17:13 UTC (Thu) by zdzichu (subscriber, #17118) [Link]
\nOnce you've read the dictionary, everything else is just a remix. Authors just synthesize words in different order, they're not creating anything new. Right?
\nPosted Oct 8, 2026 21:54 UTC (Thu) by Cyberax (✭ supporter ✭ , #52523) [Link] (6 responses)
\n\nYou do understand that, by definition, LLMs do not create anything new. Nothing. They cannot "innovate".
From the recent Scott Aaronson: https://scottaaronson.blog/?p=10169
\n\nThe UGC proof invents a completely new bizarre code with a noise test. It’s some crazy recursive construction.
\nIt’s not the long code, not the short code – some alien craziness
This seems to me like it's creativity.
\nPosted Oct 9, 2026 10:17 UTC (Fri) by paulj (subscriber, #341) [Link] (5 responses)
\n\nFrom the recent Scott Aaronson: https://scottaaronson.blog/?p=10169
To quote Scott quoting a mathematician describing the proof, as the mathematical community tries to understand these proofs:
\n"It feels like something written by someone who’s on psychedelics. So much unclear and doesn’t make sense. Lots of name dropping of previous work without discussing why it can be used despite impossibility results"
\nIt doesn't seem to be a fact that LLMs have solved loads of old, hard maths problems. Currently it sounds like mathematicians are trying to figure it IF they have OR if LLMs have just spat out a bunch of features learned from prior work it was trained on, composed together in the usual "plausible looking bullshit" way that LLMs excel at.
\nWhat is the most probable here?
\nPosted Oct 9, 2026 10:31 UTC (Fri) by stijn (subscriber, #570) [Link] (1 responses)
\nMaths solving AIs have left 'plausible looking BS' far in their wake. A lot of these proofs come with Lean programs. They could be difficult to verify (e.g. was the problem correctly encoded), but there is no question that AIs are actually solving leading edge maths problems. Coding and maths both lend themselves well to AI as it allows discovery and self-learning via testing. Code can be run and Lean programs can be verified in a productive feed-back loop.
\nPosted Oct 10, 2026 12:51 UTC (Sat) by malmedal (subscriber, #56172) [Link]
\nPerhaps somewhat paradoxically I actually find it strengthens their case when they issue retractions and corrections: https://github.com/openai/math/blob/main/history.md
\nPerfection is not(yet) attainable so the lack of corrections would be highly suspicious.
\nPosted Oct 9, 2026 10:42 UTC (Fri) by paulj (subscriber, #341) [Link]
\nA counterpoint from a comment further down by Scott: "Dana tells me she now mostly understands the proof of the UGC, and is amazed by the new ideas in it, and wants to give talks about it soon.".
\nWell, if so, it's definitely a brave new world we're heading into.
\nPosted Oct 9, 2026 23:05 UTC (Fri) by Cyberax (✭ supporter ✭ , #52523) [Link] (1 responses)
\n\nIt doesn't seem to be a fact that LLMs have solved loads of old, hard maths problems. Currently it sounds like mathematicians are trying to figure it IF they have OR if LLMs have just spat out a bunch of features learned from prior work it was trained on, composed together in the usual "plausible looking bullshit" way that LLMs excel at.
LLM produced the proof in Lean, so it can be automatically verified to be correct. And it is, nobody is arguing with it.
\nBut the way it is produced definitely looks like the LLM was on psychodelics. It definitely is a result of very alien creativity.
\nPosted Oct 10, 2026 12:28 UTC (Sat) by smurf (subscriber, #17840) [Link]
\nLLMs have a really large memory; both short term (basically their context window) and long term (their weights, i.e. a fuzzy image of the training corpus).
\nHumans can keep something like seven ideas at a time in their head … AIs, a couple of thousand or so; we can think about maybe three of these … an AI, 16, or however many attention vectors OpenAI trained their model with.
\nAs a human, I basically have to be on psychedelics to even being able to imaging having 16 hands with which to juggle a thousand balls, each different. So, no wonder that's what the output reads like.
\nPosted Oct 9, 2026 6:50 UTC (Fri) by smurf (subscriber, #17840) [Link]
\n\nYou do understand that, by definition, LLMs do not create anything new.
I understand that your definition does not fit the real world.
\n"Real" innovation often happens when people try some random s**t, just to see what the effects are, and then we keep whatever works and toss out (or even actively advocate against) what does not.
\nGuess what? LLMs have this thing called "temperature" which causes them to do exactly the same thing.
\nPosted Oct 9, 2026 11:45 UTC (Fri) by kleptog (subscriber, #1183) [Link] (4 responses)
\n\nYou do understand that, by definition, LLMs do not create anything new.
None of the code I write is very new, otherwise the answers wouldn't be on SO. It's all of the form: read from interface X, do some data mapping, write to interface Y. It's so stupid an LLM can write it. Not everybody is working on cutting edge software you know.
\nHowever, I'm not sure I agree that "LLMs can't create anything new" because "new" depends on your point of view. Is mathematics created or discovered? That's a philosophical point, but if you're in the "math is discovered" camp, then mathematicians do not ever create anything new. That doesn't mean it's not useful.
\n\nYou think 'it' is teaching you something. It is not. The original author(s) of those ideas are teaching you.
This makes no sense. I'm learning something, does it really matter where it came from? In the example above it says it in the bash(1) man page if you look for it, I'd just never considered looking for it.
\nHalf of knowledge is knowing it is possible. One you know something it possible it changes the way you approach other problems.
\nPosted Oct 9, 2026 12:22 UTC (Fri) by intelfx (subscriber, #130118) [Link] (3 responses)
\n\nHowever, I'm not sure I agree that "LLMs can't create anything new" because "new" depends on your point of view. Is mathematics created or discovered? That's a philosophical point, but if you're in the "math is discovered" camp, then mathematicians do not ever create anything new. That doesn't mean it's not useful.
Heh, why is this even an argument, let alone a philosophical one? Methinks the answer is blindingly obvious: laws are discovered (what are mathematical laws, if not "natural laws" of interaction of specifically constructed objects?), and proofs are created.
\nPosted Oct 10, 2026 20:02 UTC (Sat) by cyphar (subscriber, #110703) [Link] (2 responses)
\nThere's a whole field of philosophy dedicated to this topic, so a little more intellectual curiousity might be worthwhile.
\nYou appear to be in the empricist camp. The common counterargument to empiricism is that physical sciences (which are clearly hypothesis-driven) are inexplicably far more fallible than mathematics. The existence of Lean itself is probably one of the better examples of this -- you can probably use Lean alone to discover and prove new mathematical concepts but you can't do that in physics, chemistry, or biology.
\nPosted Oct 11, 2026 9:23 UTC (Sun) by intelfx (subscriber, #130118) [Link] (1 responses)
\n\nso a little more intellectual curiousity might be worthwhile
That’s why I posted (as an invitation to object), so please don’t accuse me of lack of intellectual curiosity of all things, thank you very much.
\nPosted Oct 12, 2026 8:08 UTC (Mon) by cyphar (subscriber, #110703) [Link]
\nI really am struggling to see how anyone could possible interpret
\n\nHeh, why is this even an argument, let alone a philosophical one? Methinks the answer is blindingly obvious: ...
as anything other than dismissive.
\nPosted Oct 15, 2026 15:04 UTC (Thu) by turistu (guest, #164830) [Link] (2 responses)
\n\n\n
next=$(printf "%02d" $((10#$h + 1)))Did you know that bash can interpret numbers in different bases?
Yes, it's documented in the manual.
\nSome of us even know that it's possible to inject executable via the $h variable, something that Gemini confidently says it's not possible -- e.g. by setting h='13+q[$(rm -r /just-kidding)]'.
BTW, this kind of argument from ignorance / lack of knowledge/imagination is kind of offensive and condescending and makes you sound like a shill, even if your wonder at the marvels of "AI" is genuine.
\nPosted Oct 15, 2026 16:06 UTC (Thu) by intelfx (subscriber, #130118) [Link] (1 responses)
\n\nYes, it's documented in the manual.
Cool, good for you. The person you're replying to didn't, and prompting an LLM taught him that, directly invalidating the great-grand-parent's argument.
\nThe fact that you personally happened to spend more time with said manual and did not need this specific piece of knowledge does not invalidate anything said previously.
\n\nBTW, this kind of argument from ignorance / lack of knowledge/imagination
What "argument from ignorance"?
\nAn "argument from ignorance" is claiming that something is true or false because of lack of evidence to the contrary. Saying that one learned something valuable by doing X, and therefore X is capable of teaching people something valuable, is not an argument from ignorance.
\n\nis kind of offensive and condescending
What is kind of offensive and condescending is your tone and style of communication. You are, of course, free to communicate as you wish, but if you intend to actually win someone over with your ideas or arguments, that tone and style isn't doing you or your ideas any favors.
\nPosted Oct 16, 2026 2:43 UTC (Fri) by smurf (subscriber, #17840) [Link]
\n\nprompting an LLM taught him that
The other interesting point here is that the "10#" prefix to avoid mis-octal-izing a number is really not that common. I mean, yeah we should code defensively and all that, but how often are we just plain lazy (or sloppy) and forget to quote our filenames in shell scripts?
\nThe real bug here is of course the fact that the shell word-expands its variables by default in the first place. But we don't have a time machine, thus can't tell Mr. Bourne to not do that.
\nPosted Oct 6, 2026 16:37 UTC (Tue) by smurf (subscriber, #17840) [Link] (2 responses)
\n\n"They didn't seem to notice or care that it either wasn't producing tarballs or was producing incomplete ones"
Yeah, well. Notice that their build instructions do not mention downloading tarballs. Also, why the heck would you want to?
\n\nlinks to instructions for Linux (as well as macOS and Windows) return a 503 error
They work now — probably 'cause somebody™ read this article.
\nPosted Oct 8, 2026 0:42 UTC (Thu) by tbodt (subscriber, #120821) [Link]
\nCurious - the "link to instructions for Linux [that] return a 503 error" is the exact same link as the link from "Searching for instructions turns up this page". Seems like the 503 error resolved itself in the minutes between the author trying the link and searching for the page. Guess Google is not exempt from AI scraping ddos on git hosts.
\nPosted Oct 11, 2026 2:29 UTC (Sun) by Kangie (guest, #161139) [Link]
\n\nYeah, well. Notice that their build instructions do not mention downloading tarballs. Also, why the heck would you want to?
Tarballs are the canonical way that Linux distributions get their source code for a particular release. A git clone of Chromium is massive and has overheads versus a tarball that's generated once, for example.
\nA good reason for wanting to use a tarball over a live git repo, even if you're targeting a tag, is working in an offline or air-gapped environment - you want the tarball to contain everything that you need to build the software (except for dependencies).
\nGentoo isn't alone in using tarballs, and upstream do provide them; they just don't notice when it fails to generate because their CI is entirely based on git (which is fine - it's their workflow)
\nPosted Oct 6, 2026 17:23 UTC (Tue) by maharmstone (subscriber, #174156) [Link] (8 responses)
\nThis is sad, but entirely understandable.
\nA large part of the problem is, as noted, quite how long Chromium takes to compile: three and a half hours on my beefy machine. It is the slowest package by quite literally a factor of 10. It's ridiculous how much it's ballooned in recent years.
\nPosted Oct 6, 2026 17:44 UTC (Tue) by ballombe (subscriber, #9523) [Link] (1 responses)
\nFor reference, how much time do you need to compile libreoffice ?
\nPosted Oct 6, 2026 18:01 UTC (Tue) by maharmstone (subscriber, #174156) [Link]
\n15 minutes for libreoffice, and FWIW 10 minutes for firefox
\nPosted Oct 7, 2026 13:14 UTC (Wed) by marcH (subscriber, #57642) [Link] (5 responses)
\n\nA large part of the problem is, as noted, quite how long Chromium takes to compile: three and a half hours on my beefy machine.
Chrome (and all browsers to some extend) are mini-operating systems. Operating systems are big and take a long time to compile.
\nChrome developers never compile on a single system, they always use advanced caching and remote execution to distribute the build on multiple systems. Which is one more build challenge when you are not a Google employee.
\nPosted Oct 7, 2026 14:15 UTC (Wed) by ceph3 (guest, #164777) [Link] (1 responses)
\nThere was a "joke" on Old Twitter some time ago that the easiest way to build Chromium is to become a Google employee, although workarounds exist if that's not possible.
\nPosted Oct 8, 2026 2:17 UTC (Thu) by gutschke (subscriber, #27910) [Link]
\nIt is my understanding that non-Google Chromium developers occasionally get approved for access to Google's build infrastructure.
\nPosted Oct 8, 2026 5:39 UTC (Thu) by maharmstone (subscriber, #174156) [Link] (1 responses)
\n\nChrome (and all browsers to some extend) are mini-operating systems. Operating systems are big and take a long time to compile.
No - Firefox takes 10 minutes. Chromium is not 20 times more complicated than Firefox. Or if it is, it shouldn't be.
\nPosted Oct 8, 2026 14:52 UTC (Thu) by marcH (subscriber, #57642) [Link]
\nThere is a Firefox thread above and reading between those lines, the compile time difference between Chrome and Firefox does not seem that big? Apples/oranges maybe.
\nPosted Oct 15, 2026 15:44 UTC (Thu) by turistu (guest, #164830) [Link]
\n\nChrome (and all browsers to some extend) are mini-operating systems. Operating systems are big and take a long time to compile.
I still remember a test I did in 2006 or so, where building the entire plan9 OS from scratch (kernel, user-space, compilers, etc) took less time than firefox just starting up.
\nAnd a) firefox was actually quite snappy, compared to some java-written horrors I had to run and b) half of the time building plan9 was actually spent building the imported ghostscript ;-)
\nPosted Oct 6, 2026 18:08 UTC (Tue) by q3cpma (subscriber, #120859) [Link] (1 responses)
\nFirst, thanks a lot for the brave souls that tried to make Chromium sane enough to make sense on Gentoo and it's better to stop early than after a few more burnt out maintainers if upstream is negligent to the point of hostility to building it without bundling.
\nA note, ungoogled-chromium is relatively well maintained in the overlay of one of its main devs (https://github.com/PF4Public/gentoo-overlay) and might be an alternative for desperate people.
\nPersonally, I can only hope the new wave of browsers (Ladybird, Servo) bring a bit of sanity or at least less historical cruft to our world in the coming years. Guess most of us will have to endure Firefox crumbling under the weight of its piggy-backed foundation full of suits and used car salesmen.
\nPosted Oct 11, 2026 2:30 UTC (Sun) by Kangie (guest, #161139) [Link]
\nFor now. The Crubit issue was actually just "discussed" by upstream - continuing to revert patches using it is not sustainable.
\nhttps://groups.google.com/a/chromium.org/g/embedder-dev/c...
\nPosted Oct 7, 2026 0:05 UTC (Wed) by tlb (subscriber, #178072) [Link] (4 responses)
\nWe've seen this with other Google endeavors into Open Source, most notably Android, where the codebase becomes so large, the build system so onerous and the upstream so unresponsive to downstream maintainers (GrapheneOS's troubles with monthly patches being withheld by Google as an example) that it sometimes feels like an intentional effort to make life impossible for maintainers trying to keep the promises of being "open source".
\nPosted Oct 7, 2026 10:13 UTC (Wed) by jengelh (subscriber, #33263) [Link]
\nI remember a time, 25 years ago, when "open source" was literally what it said: available source. There generally were no SCMs, and often also no free public discussion forum. Maybe the maintainer already abandoned the project by the time you found it. Which was fine, because the promise was/is just for source. Not for collaboration.
\nAny sufficiently large project might also be struggling under its own weight. (too few maintainers for too much existing code and too many submissions) If you're trying to implement/manage everything under the sun, I can see this easily happening.
\nPosted Oct 7, 2026 13:24 UTC (Wed) by marcH (subscriber, #57642) [Link] (1 responses)
\n\nWe've seen this with other Google endeavors into Open Source, most notably Android, where the codebase becomes so large, the build system so onerous and the upstream so unresponsive to downstream maintainers (GrapheneOS's troubles with monthly patches being withheld by Google as an example) that it sometimes feels like an intentional effort ...
You've just described Hanlon's razor perfectly.
\nOne very rarely ever gets a promotion by fixing the build system. Maybe a bit less unlikely at Google but still nothing like "real" developers working on the actual product as opposed to the infrastructure or test code.
\nOne or two years ago when LLMs where not as good as today, I very clearly remember many people describing them as "not great but good enough for test code".
\nDirty little secret: products don't match specifications, they only match their test coverage. If there's no test coverage or buggy coverage then it does not work.
\nPosted Oct 7, 2026 19:41 UTC (Wed) by smurf (subscriber, #17840) [Link]
\nIMHO this is not a matter of promotions. It's a matter of open and bidirectional stewardship of the source code you released into the world … a task which Google is rather bad at.
\nDon't forget that they don't even have a source repository for Chromium / Chrome / whatever. They have a source repository for their private, closed-off corner of the world, thus zero incentive (and, apparently, zero intent) to fix anything they don't need, or want, fixed.
\nI'm not exactly calling this intentional — don't attribute to malice which can adequately explained by stupidity, and all that — but …
\nPosted Oct 9, 2026 5:17 UTC (Fri) by ssmith32 (subscriber, #72404) [Link]
\nInvoking the rule of "incompetence before malicious" - see my comment above.
\nGoogle has some non-normal deeply held beliefs about monorepos and dependencies in builds. It's a long-standing thing, before any open-source efforts.
\nI suspect even some ex-googlers would be inclined to attribute it to intentions, so adverse are they to admitting their old company.. might have been wrong about something. But I've ran into several that admit it's just an odd .. quirk .. of the company.
\nPosted Oct 7, 2026 6:41 UTC (Wed) by kxxt (subscriber, #172895) [Link]
\nI have been maintaining (unofficial) RISC-V continuous delivery pipeline for electron https://github.com/riscv-forks/electron-riscv-releases and Chromium https://github.com/riscv-forks/chromium-riscv for some time.
\nSetting up a continuous integration workflow greatly helped with that. Currently my CI runs weekly and it helped me to discover bugs that affects RISC-V before it hits a stable release and gives me a chance to open a CL to the gerrit instance so that I (and potentially other packagers of chromium/electron RISC-V) don't need to deal with patching if the CL is submitted before the next major stable version of Chromium branches.
\nBut that does not eliminate the maintenance burden (for me). It just brings forward the burden so that I can release Chromium and Electron releases on schedule. For many RISC-V CLs I submitted to the gerrit in the past, most of the time reviewers are responsive and helpful. But the two CLs I opened recently seen no activity from the reviewers. I am not sure what's exactly happening, though.
\nPosted Oct 7, 2026 9:39 UTC (Wed) by Karellen (subscriber, #67644) [Link] (2 responses)
\nI'm just remembering back to the Apache OpenOffice days when, IIRC, there was the ooo-build cross-distro collaboration team that worked on common packaging issues, like maintaining patches which upstream was not interested in, and unbundling.
\nWould anything like that be useful here, or are the packaging systems too diverse (and upstream too active, unlike AOO?) for that to work?
\nPosted Oct 7, 2026 13:53 UTC (Wed) by smcv (subscriber, #53363) [Link] (1 responses)
\nI've seen Debian integrating patches from Ungoogled Chromium, so I think Ungoogled Chromium is acting as this, at least partially and informally.
\nPosted Oct 8, 2026 2:15 UTC (Thu) by dilinger (subscriber, #2867) [Link]
\nThis is correct. Patches are flowing both ways; recently we (debian) worked with u-c to rework disable-ai.patch, and also fixed a bug that occurred when cherry-picking that patch (a separate u-c patch masked the bug for u-c users, but would've hit them had they dropped the other patch).
\nFor debian (and for ubuntu, debxtra packages are available for debian's chromium patches on ubuntu), I predict that we'll be pulling in more and more u-c patches. I would encourage other distributions to do the same.
\nPosted Oct 7, 2026 10:29 UTC (Wed) by cyperpunks (subscriber, #39406) [Link] (2 responses)
\nThis is standard outcome for Google software, its very often a huge mess.
\nI really don't understand how such clever Devs can create such chaos in every project, must be nightmare to develop
\nsoftware in such environments.
Posted Oct 7, 2026 11:36 UTC (Wed) by pizza (subscriber, #46) [Link]
\n\nI really don't understand how such clever Devs can create such chaos in every project, must be nightmare to develop
\nsoftware in such environments.
It's a perfect example of Conway's Law in action:
\n"An organization's software architecture will always mirror its internal communication structure and organization chart."
\nGoogle is a total mess internally, therefore any software it produces will reflect that mess.
\nPosted Oct 7, 2026 13:24 UTC (Wed) by farnz (subscriber, #17727) [Link]
\nPart of it will be that Google has organization-wide tooling (like Blaze, Forge remote execution, Piper/CitC and more) that overcomes the chaos if you're working with Google's internal tooling and Google's internal code review systems. As a result, the huge mess is manageable because the tooling makes it manageable, not because it's a good state for a project.
\nPosted Oct 7, 2026 14:19 UTC (Wed) by mathstuf (subscriber, #69389) [Link] (6 responses)
\nPerhaps funding Igalia to alleviate things like "reliably make source tarballs" would be useful? Any Gentoo-oriented companies that might be in consideration? Though such work helps any distro, really.
\nPosted Oct 7, 2026 16:13 UTC (Wed) by mussell (subscriber, #170320) [Link]
\nThe largest deployment of Gentoo as far as I know is ChromeOS, so Google would be the best Gentoo-oriented company to help, but their ebuilds for Chrome don't use tarballs and is otherwise incompatible with Gentoo's www-client/chromium (also ChromeOS is dying). Otherwise I don't think there's that many Gentoo companies that focus on desktops that also care about Chromium support since Firefox has a similar feature set and doesn't take hours to build.
\nPosted Oct 7, 2026 17:55 UTC (Wed) by NightMonkey (subscriber, #23051) [Link]
\nSadly, there's likely few direct Gentoo-using companies out there, AFAIK. However, there are certainly downstream Linux distros that benefit from Gentoo's existence, packaging efforts and labor of its volunteers (especially in the Gentoo Security team).
\nGentoo's packaging format (ebuilds are Bash!) and documentation makes it quite easy for a person to discover how a package is created and installed and use in other distributions. I can imagine that other distribution's packagers "look over the fence" to see how to get past local packaging problems.
\nPosted Oct 8, 2026 4:15 UTC (Thu) by dilinger (subscriber, #2867) [Link] (3 responses)
\nIronically, the person who stepped in when upstream chromium's tarball generation stuff broke for a while is a Gentoo dev. Matt Jolly created https://github.com/chromium-linux-tarballs/chromium-tarba..., which Debian usually uses instead of upstream's source tarballs. They're bit-for-bit the same thing, but they have (had?) a better track record of being generated than chromium's official source tarballs.
\nPosted Oct 10, 2026 1:42 UTC (Sat) by dilinger (subscriber, #2867) [Link] (2 responses)
\nMatt actually just wrote a good summary of the many problems we/they have been having with chromium upstream:
\nhttps://groups.google.com/a/chromium.org/g/embedder-dev/c...
\nI can verify a bunch of the things he's laid out. I honestly don't understand how upstream does "release management", because it's like a bunch of the worst practices I've ever seen.
\nI'd encourage everyone who's saying things like "just use AI!" or "have $Company pay for it!" to read that email. It's not just a lack of volunteer/paid hours, it's upstream tossing things out there that simply don't work for various distributions.
\nPosted Oct 10, 2026 3:15 UTC (Sat) by mathstuf (subscriber, #69389) [Link]
\nThere seems to have been one champion back in 2014 inspired by a post by spot from 2009 (linked there). Maybe we just need 5 years for this message to provoke progress too?
\nPosted Oct 11, 2026 2:35 UTC (Sun) by Kangie (guest, #161139) [Link]
\nOh, hi!
\nPlease do contribute if you have something to add, or even a +1 on the concerns; I don't want to dogpile upstream but the situation they've left us with isn't one that can be easily solved.
\nIt's either carry ever-growing reverts or use unreleased LLVM and Rust rolls. :(
\nPosted Oct 7, 2026 18:29 UTC (Wed) by lwn4096 (subscriber, #177466) [Link]
\nGoogle's own ChromeOS and ChromiumOS are derived from Gentoo. Thank you for drawing attention to the matter, as a long time Gentoo user I wasn't aware of the bug until now. Many thanks to the maintainers of www-client/chromium for making it work for so long.
\nPosted Oct 9, 2026 5:22 UTC (Fri) by ssmith32 (subscriber, #72404) [Link]
\nBoth for maintaining it, and giving it last rites. A good example of some of the things that I think makes open-source great - a community spirit, focused on practical outcomes, and making the best of what what you have.
","headings":[{"level":3,"text":"On expectations","id":"on-expectations"},{"level":3,"text":"On expectations","id":"on-expectations-2"},{"level":3,"text":"On expectations","id":"on-expectations-3"},{"level":3,"text":"Firefox maintenance","id":"firefox-maintenance"},{"level":3,"text":"Firefox maintenance","id":"firefox-maintenance-2"},{"level":3,"text":"On expectations","id":"on-expectations-4"},{"level":3,"text":"On expectations","id":"on-expectations-5"},{"level":3,"text":"Packaging Chromium with limited manual effort","id":"packaging-chromium-with-limited-manual-effort"},{"level":3,"text":"Packaging Chromium with limited manual effort","id":"packaging-chromium-with-limited-manual-effort-2"},{"level":3,"text":"Packaging Chromium with limited manual effort","id":"packaging-chromium-with-limited-manual-effort-3"},{"level":3,"text":"Packaging Chromium with limited manual effort","id":"packaging-chromium-with-limited-manual-effort-4"},{"level":3,"text":"Packaging Chromium with limited manual effort","id":"packaging-chromium-with-limited-manual-effort-5"},{"level":3,"text":"Packaging Chromium with limited manual effort","id":"packaging-chromium-with-limited-manual-effort-6"},{"level":3,"text":"Understandable","id":"understandable"},{"level":3,"text":"Understandable","id":"understandable-2"},{"level":3,"text":"AI could help","id":"ai-could-help"},{"level":3,"text":"AI could help","id":"ai-could-help-2"},{"level":3,"text":"AI could help","id":"ai-could-help-3"},{"level":3,"text":"AI could help","id":"ai-could-help-4"},{"level":3,"text":"AI could help","id":"ai-could-help-5"},{"level":3,"text":"AI could help","id":"ai-could-help-6"},{"level":3,"text":"AI could help","id":"ai-could-help-7"},{"level":3,"text":"AI could help","id":"ai-could-help-8"},{"level":3,"text":"AI could help","id":"ai-could-help-9"},{"level":3,"text":"AI could help","id":"ai-could-help-10"},{"level":3,"text":"AI could help","id":"ai-could-help-11"},{"level":3,"text":"AI could help","id":"ai-could-help-12"},{"level":3,"text":"AI could help","id":"ai-could-help-13"},{"level":3,"text":"AI could help","id":"ai-could-help-14"},{"level":3,"text":"AI could help","id":"ai-could-help-15"},{"level":3,"text":"AI could help","id":"ai-could-help-16"},{"level":3,"text":"AI could help","id":"ai-could-help-17"},{"level":3,"text":"AI could help","id":"ai-could-help-18"},{"level":3,"text":"AI could help","id":"ai-could-help-19"},{"level":3,"text":"AI could help","id":"ai-could-help-20"},{"level":3,"text":"AI could help","id":"ai-could-help-21"},{"level":3,"text":"AI could help","id":"ai-could-help-22"},{"level":3,"text":"AI could help","id":"ai-could-help-23"},{"level":3,"text":"AI could help","id":"ai-could-help-24"},{"level":3,"text":"AI could help","id":"ai-could-help-25"},{"level":3,"text":"AI could help","id":"ai-could-help-26"},{"level":3,"text":"AI could help","id":"ai-could-help-27"},{"level":3,"text":"AI could help","id":"ai-could-help-28"},{"level":3,"text":"AI could help","id":"ai-could-help-29"},{"level":3,"text":"AI could help","id":"ai-could-help-30"},{"level":3,"text":"AI could help","id":"ai-could-help-31"},{"level":3,"text":"AI could help","id":"ai-could-help-32"},{"level":3,"text":"AI could help","id":"ai-could-help-33"},{"level":3,"text":"AI could help","id":"ai-could-help-34"},{"level":3,"text":"AI could help","id":"ai-could-help-35"},{"level":3,"text":"AI could help","id":"ai-could-help-36"},{"level":3,"text":"AI could help","id":"ai-could-help-37"},{"level":3,"text":"AI could help","id":"ai-could-help-38"},{"level":3,"text":"AI could help","id":"ai-could-help-39"},{"level":3,"text":"AI could help","id":"ai-could-help-40"},{"level":3,"text":"AI could help","id":"ai-could-help-41"},{"level":3,"text":"AI could help","id":"ai-could-help-42"},{"level":3,"text":"AI could help","id":"ai-could-help-43"},{"level":3,"text":"AI could help","id":"ai-could-help-44"},{"level":3,"text":"AI could help","id":"ai-could-help-45"},{"level":3,"text":"AI could help","id":"ai-could-help-46"},{"level":3,"text":"AI could help","id":"ai-could-help-47"},{"level":3,"text":"AI could help","id":"ai-could-help-48"},{"level":3,"text":"AI could help","id":"ai-could-help-49"},{"level":3,"text":"AI could help","id":"ai-could-help-50"},{"level":3,"text":"AI could help","id":"ai-could-help-51"},{"level":3,"text":"AI could help","id":"ai-could-help-52"},{"level":3,"text":"AI could help","id":"ai-could-help-53"},{"level":3,"text":"Tarballs?","id":"tarballs"},{"level":3,"text":"Tarballs?","id":"tarballs-2"},{"level":3,"text":"Tarballs?","id":"tarballs-3"},{"level":3,"text":"Sad","id":"sad"},{"level":3,"text":"Sad","id":"sad-2"},{"level":3,"text":"Sad","id":"sad-3"},{"level":3,"text":"Sad","id":"sad-4"},{"level":3,"text":"Sad","id":"sad-5"},{"level":3,"text":"Google build infrastructure","id":"google-build-infrastructure"},{"level":3,"text":"Sad","id":"sad-6"},{"level":3,"text":"Sad","id":"sad-7"},{"level":3,"text":"Sad","id":"sad-8"},{"level":3,"text":"Thanks and an alternative","id":"thanks-and-an-alternative"},{"level":3,"text":"Thanks and an alternative","id":"thanks-and-an-alternative-2"},{"level":3,"text":"Could this be intentional?","id":"could-this-be-intentional"},{"level":3,"text":"Could this be intentional?","id":"could-this-be-intentional-2"},{"level":3,"text":"Could this be intentional?","id":"could-this-be-intentional-3"},{"level":3,"text":"Could this be intentional?","id":"could-this-be-intentional-4"},{"level":3,"text":"Could this be intentional?","id":"could-this-be-intentional-5"},{"level":3,"text":"Maintain our own CI","id":"maintain-our-own-ci"},{"level":3,"text":"Is there cross-distro collaboration for e.g. unbundling?","id":"is-there-cross-distro-collaboration-for-e-g-unbundling"},{"level":3,"text":"Is there cross-distro collaboration for e.g. unbundling?","id":"is-there-cross-distro-collaboration-for-e-g-unbundling-2"},{"level":3,"text":"Is there cross-distro collaboration for e.g. unbundling?","id":"is-there-cross-distro-collaboration-for-e-g-unbundling-3"},{"level":3,"text":"Typical Google software","id":"typical-google-software"},{"level":3,"text":"Typical Google software","id":"typical-google-software-2"},{"level":3,"text":"Typical Google software","id":"typical-google-software-3"},{"level":3,"text":"Igalia support?","id":"igalia-support"},{"level":3,"text":"Igalia support?","id":"igalia-support-2"},{"level":3,"text":"Igalia support?","id":"igalia-support-3"},{"level":3,"text":"Igalia support?","id":"igalia-support-4"},{"level":3,"text":"Igalia support?","id":"igalia-support-5"},{"level":3,"text":"Igalia support?","id":"igalia-support-6"},{"level":3,"text":"Igalia support?","id":"igalia-support-7"},{"level":3,"text":"Ironic","id":"ironic"},{"level":3,"text":"A huge thank-you","id":"a-huge-thank-you"}]}}![Gemini confidently says it's not possible](\"https://www.pasteboard.co/sJzbaRxSJJ52.jpg\"){kind=link}