> `git history fixup` fixes an old commit that has something wrong in it, then autorebases all your branches to match.
Simplifying `git rebase -i` is a great idea, but unfortunately, that does not match my workflow.
I always keep an history of my branches. For example, when I develop a feature, the first version of the branch is `myfeature.1`, then `myfeature.2`, and so on. This is useful because I can retrieve an old version when something behaves differently in a newer one. (And no, `git reflog` will not help)
This has saved me multiple times. For example, I can determine that a problem was introduced between `myfeature.58` and `myfeature.59`. If the "feature" contains 15 commits, I do:
This lets me see the changes between the 2 versions, commit by commit (even if they don't have the same base).
I don't want all the branches containing a commit to be rebased automatically (I already try using tags for old versions instead, but this does not quite fit).
> The difference in the code is exactly one word: value.
What is unclear to me is why the decision to use a Point instance as a value or as a reference is made in the class definition rather than by the caller.
> Point[] point = new Point[10];
For the same class, I might need an array of values in one place and an array of references elsewhere within the same codebase.
It would really break the GC or the type system if you did that. If you return a single point out of a 100k long array, it has to pin the array? How do you track the GC root? Struct array elements need back pointers?
Conversely, if it auto-copies now you have to contend with runtime state changing the pass semantics?
> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design. In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability. We catalog the ways in which fork is a terrible abstraction for the modern programmer to use, describe how it compromises OS implementations, and propose alternatives.
> As the designers and implementers of operating systems, we should acknowledge that fork’s continued existence as a first-class OS primitive holds back systems research, and deprecate it. As educators, we should teach fork as a historical artifact, and not the first process creation mechanism students encounter.
> The received wisdom suggests that Unix’s unusual combination of fork() and exec() for process creation was an inspired design.
No, it was done that way so that you could launch a program that was too big to fit in memory with the parent program. The original implementation worked by swapping out the forking program to disk on a fork() call. Then, at the moment the program was swapped out but control had not returned, the process table entry was duplicated and adjusted so that there were now two processes, one in memory and one swapped out. The one in memory then got control, and could do an exec() call.
This allowed large programs to run on small PDP-11 machines. It was needed back in the era of really expensive memory. That's why.
QNX had an interesting approach. Program loading isn't in the OS at all. There's "fork", but program loading is in a library. It links to a .so file which reads the executable header, allocates memory, loads the program, gets it ready to run, and starts it. The program loader runs in user space and is unprivileged. This is probably the right way to do it.
I think fork() is more of a PDP-7 mistake than a PDP-11 mistake.
On the original UNIX system, memory was so limited that the only sane partitioning was to write the running program's memory image to disk, then reuse the running image as the child. An immediate consequence is the UNIX I/O model, where disk I/O is always synchronous (can't swap processes while waiting for disk I/O because swapping processes requires disk I/O). Anyway, as soon as the UNIX group got a PDP-11, the model broke down, because they had enough memory for multiple processes, but fork() didn't allow them to run concurrently, because their first PDP-11 didn't have an MMU. So they whined until they got one with an MMU instead of fixing their broken design.
I think GP was saying that in QNX the spawning process was responsible for dynamically linking it's child process before running it. With Linux, I think it's the spawned process taking care of it's own dynamic linking.
On QNX the process spawning is done by sending a message to the userspace process manager, which creates a new process table entry and queues up its initial thread. When its initial thread gets a timeslice its entry point may be the dynamic loader (as specified in the PT_INTERP segment) which then does all the dynamic linking as the spawned process or it might be some other entry point like with a statically-linked executable.
So on QNX, the spawned process does all the dynamic linking. The spawning process just sends an asynchronous message to the process manager and then gets on with things in a very deterministic manner as befitting a hard realtime system.
"In this paper, we argue that fork was a clever hack for machines and programs of the 1970s that has long outlived its usefulness and is now a liability"
> It links to a .so file which reads the executable header, allocates memory, loads the program, gets it ready to run, and starts it. The program loader runs in user space and is unprivileged. This is probably the right way to do it.
aiui this is what exec does, the problem outlined here is the split between process creation (expensive, kernel space, has to be done each time even if spawning the same process "template" repeatedly) and loading (cheap and in userspace).
Don’t pretty much all OSes implement process startup in userspace? On macOS, the kernel creates a process with an image of dyld and points it at dyld_start, which actually takes care of parsing the Mach-O header. I assumed ld.so does the same job on Linux.
Nope, the kernel can load static ELF binaries. ld.so is only needed for dynamically linked binaries, and in fact many Go applications (for example, as they're statically linked) ship as containers with nothing but the single binary.
You can do this on macOS too, if you're willing to break all forward/backward compatibility and make direct syscalls you can have a purely static binary. Without the LC_LOAD_DYLINKER command on the mach-o binary the kernel should just jump to the entrypoint based on LC_UNIXTHREAD. (This may not longer work on arm machines though if they actually trap on direct syscalls not through libSystem, similar to the BSDs)
Yes, it can all be done in userspace. When the "fork in the road" paper came up a while back someone linked to an example. https://grugq.github.io/docs/ul_exec.txt
But why is having a pair of separate independent operations, fork and exec, required to achieve this? A single fexec call could be implemented to work in the way you describe, no?
It's a fairly widespread idea for architectures that try to move things out of kernel mode. The Hurd does program image file loading in userspace, too, in its exec server(s).
The tricky part is setting up the initial process. The way out for that is static linking and re-use of the fact that the operating system kernel loader has to understand and be able to load (at least a small subset of) program image file formats too.
The problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1, and you need overcommit.
Now these decisions aren't objectively bad, but they have significant trade-offs and it's probably not a good idea that they're forced simply because we use fork()+exec() for process creation.
CoW is probably a good idea whether you use fork or not. Or rather, fork is probably a better option than just exec exactly because it can benefit from CoW.
At least on systems with virtual addressing. If you want to go into physical addressing, then yes, maybe it's a problem. But Linux will never touch anything with physical addressing, so I don't see what people are complaining about.
CoW is probably a good idea regardless, yeah. Overcommit is more questionable. Regardless, both ought to be argued based on their own merits. It's unfortunate that both are necessary as a consequence of fork().
I don't think fork() mandates overcommit. OpenBSD doesn't seem to even allow overcommit or have an OOM killer, memory allocations that exceed available capacity fail immediately even if the memory is not touched.
Let's say you have 1GB RAM. You're running program that occupies 600 MB. Now this program wants to launch second small program that occupies 1 MB.
You're doing fork + exec.
If you're overcommiting, fork will not reserve another 600 MB, and exec immediately after fork will cause total system usage to be 601 MB.
If you're not overcommiting, that fork will fail, because total memory consumption will be 1200 MB which is more than 1GB. That somewhat restricts program design.
> Let's say you have 1GB RAM. You're running program that occupies 600 MB. Now this program wants to launch second small program that occupies 1 MB.
> You're doing fork + exec.
This is the clear problem: you don't want another process that's a duplicate of the current one, that's just a detail of what you actually want: a 1mb process. Right now it's a badly leaky detail which you're forced to work around.
> If you're not overcommiting, that fork will fail, because total memory consumption will be 1200 MB which is more than 1GB. That somewhat restricts program design.
As I understand it, the whole purpose of vfork is to avoid that problem. So vfork does not copy memory in any way and you're not allowed to do anything but exec after vfork. But at this point question arises: why not call it `vfork_and_exec` and get rid of undefined behaviour. Or choose better name like `CreateProcessW` hehe
> As with fork(2), the child process created by vfork() inherits copies of various of the caller's process attributes (e.g., file descriptors, signal dispositions, and current working directory); the vfork() call differs only in the treatment of the virtual address space, as described above.
so it seems Linux does define the behavior of vfork, but if you rely on it, your code won't be portable to other POSIX systems
> The problem with fork isn't really that it's slow. The problem is that if you want it to be not-slow, it locks you into a bunch of OS design decisions: you more or less need a memory subsystem where all writable pages are refcounted and copy-on-write when the refcount is bigger than 1
It may not be slow, but for the common case where fork is almost immediately followed by exec in the process where fork returns zero fork increases those refcounts and exec almost immediately decreases them again hand does typically unnecessary checks whether refcounts became zero). A combined fork/exec syscall can avoid that work.
On the other hand, a sufficiently powerful combined fork/exec call has to have a lot of parameters that it has to check (whether to inherit open pipes, open files, setting the working directory, etc), and that slows it down.
That can be avoided by having multiple variants of combined fork/exec calls, but you would need lots of them to cover all combinations of flags.
I expect either approach should be faster then having fork, then exec as separate calls, especially when the process calling fork has many resources allocated.
Another possible design is instead of forking the current process, you create a new empty process, then the parent calls syscalls to set up the new process, and eventually call exec on the child process. That does mean you either need new syscalls for that, or adapt existing syscalls to take a pidfd as an argument. That also solves some other problems with fork/exec where the default is to inherit a lot of things you probably don't want. With this, you can opt in to inheritance instead of having to opt out.
Or you could create a hybrid between a thread and a process, where it still uses the parent's memory space (unlike fok), but has it's own stack (unlike vfork), and is in its own process (unlike a thread). I think this is technically possible on linux, but there isn't a readily available interface for it. Although it seems like posix_spawn could be implemented that way...
> you create a new empty process, then the parent calls syscalls to set up the new process ...
That does seem like a much better design to me. But I wonder if that was considered way back at the dawn of computing and rejected for good reason?
> I think this is technically possible on linux, but there isn't a readily available interface for it.
Yes there is, see `man clone`. POSIX and glibc are quite different from the kernel in this regard. AFAIK under linux there are just threads of execution that might or might not share various namespaces and memory mappings. That said, the kernel does place a few artificial restrictions on what combinations are allowed in order to (as I understand it) guard against the unintended exercise of entirely untested combinations that serve no known practical purpose.
The practical problem is that if you start doing as you please with the various namespaces and mappings you quickly become incompatible with glibc and by extension most likely the majority of the dynamic libraries available on your system.
I remember reading about an OS where processes weren’t basic building blocks. Instead it had a syscall to create an address space and to create a thread in an address space.
Create a thread in your own address space, and your process becomes multi-threaded. Create an address space, load some code in it, and create a thread there, and you fork/exec-ed.
In my memory, that OS was MACH, but Google doesn’t confirm that for me.
io_uring taught us that if syscalls are expensive, queue them up in a buffer with one syscall to transfer the thread to the os to process it. So, queue up the new process mutations in a buffer with a single syscall to process all of them in a batch. This model should have replaced repetitive syscalls across the kernel years ago.
In addition to what you said: forking from a process running on multiple cores is slow once you have mark all pages as read-only and shoot this out to all cores. TLB synchronization is super expensive. Unix originally didn't support threads (want concurrency? just fork!) but with modern multicore that's clearly unsustainable.
With large enough processes, like say a server JVM process that uses 10s of GBs of RAM, even just copying the page tables for CoW can be slow. And unless you have aggressive overcommit settings you can get an OOM on fork, even if you're just going to exec something small.
vfork helps a little, but it has a lot of restrictions on what you can do before the exec, and on unix that's basically the only place you can do things like close files, change signal masks, drop privileges or set up seccomp, etc.
vfork() helps a LOT. The restrictions on what you can do on the child-side of vfork() are pretty much the same ones as for fork() + you must not do anything to damage the stack frame of the vfork() caller (i.e., you can't return).
> the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork(), or returns from the function in which vfork() was called, or calls any other function before successfully calling _exit(2) or one of the exec(3) family of functions
That's a lot more restrictive. You can't use local variables, or call any functions other than _exit or execve. On linux specifically, I _think_ those restrictions are more relaxed and you can call async-signal-safe functions, however I'm not entirely clear on how relaxed that is, and as far as I understand that isn't portable.
But some of that is nonsense and incorrect. You can very much use local variables, and you'll find tons of vfork()-using code that does that and calls plenty of async-signal-safe functions.
The real restrictions are:
- you can't damage the function call frame
of the caller of vfork(), thus you can't
return from it
- you may only call async-signal-safe
functions on the child side of vfork()
That's basically it. Yes, you'll want to call execve(2) or _exit(2) before long, but there is no time limit as to that, it's just that the whole point of calling vfork() is to make it real cheap to spawn a process, which means ultimately calling execve(2), with _exit(2) being what you do if it execve(2) fails (e.g., because ENOENT).
There is a ton of vfork()-using code that adheres to these real restrictions and has been working fine for decades. That includes several posix_spawn() implementations, the C shell, etc.
I demand evidence that this part: "the behavior is undefined if the process created by vfork() either modifies any data other than a variable of type pid_t used to store the return value from vfork()" is remotely true. That evidence must be of the form of bug reports that were accepted and which stand to scrutiny.
I've never found any such evidence. Have you?
Meanwhile I have a proof by existence that vfork() is safe used much more liberally than you say it may be used.
> You can't use local variables, or call any functions other than _exit or execve.
There are other async-signal-safe functions, and they get used routinely by posix_spawn() and other code to do child-side setup before execve(2), including: I/O redirection, process group setup, signal handling changes, etc.
Didn't he just say that fork turns out to be comparatively faster to the non-fork samples we get? Ie Linux spawns processes faster than Microsoft's kernels?
Didn't I just say that "the problem with fork isn't really that it's slow"? It's all the other OS design choices it forces on you if you want it to be fast.
We don't have any broadly used non-fork samples. Windows, macOS, and Linux all have fork. So the presence of fork can't be the reason for the performance difference.
If you pass null for the section handle, it shares pages with the calling process, thus implementing a forking model. Or at least the parts of a forking model that some people erroneously believe are responsible for performance differences.
anarazel's comment focuses entirely on performance, indicating that they have an impression that the discussion about why fork is bad is about performance. I'm not entirely sure where this impression came from, as it's not mentioned in
rom1v's quote nor a point in the linked paper, "A fork() in the road".
A more accurate way to describe this is that Windows' (NT onward) core execution context model is a bunch of threads that by default share memory, whereas Unixen have a core task context model of a bunch of threads that by default do not share memory.
Both systems are implemented using threads as the execution context, but in Unix, the history means that that you fork+exec most of the time, resulting in a two tasks that do not share memory any more. By contrast, on Windows (NT onward) the common case when creating a new execution context is to create a thread that shares memory with others in its process.
Both systems allow the easy use of the other's core abstraction. On Unix, you can either code like its 1986 and use fork without exec, or use clone(3) or any of its higher level abstractions like pthreads.
You're right that POSIX semantics get tangled when using threads.
That's actually less accurate, not more. It's a post-hoc revision that conflates Unix with Linux.
The Unix model was invented over a decade before the idea of multithreading percolated into mainstream operating systems at all.
The reason that Windows NT started as it did, was that OS/2 had come out in 1987, with kernel threads, and the idea of multithreading had taken root. SunOS 5 gained threading, too.
Windows NT applications development began with threading available as a mechanism from the start, and with a lot of people in the IBM/Microsoft world already knowing about its use in applications development from OS/2.
Whereas with the Unices it came in more gradually, as the applications had often already been designed. The whole libthread versus libpthread thing made things interesting on SunOS for a few years, too. As did the first attempt (LinuxThreads) at providing threads on Linux.
PaulDavisThe1st is saying that the Unix pattern of forking a process (and not calling exec) was an early form of multi-threading (or multi-processing), but unlike threads in NT and later pthreads, they didn't share memory and communication between them required some form of IPC.
Yep, absolutely corrrect. It was true at the lowest level (the semantics of fork) and it was true at the app/platform design level: in Windows you used threads inside a process, on Unix you used multiple communicating processes.
This obviously changed as pthreads came into being, and at this point, I suspect that the typical use for threads-sharing-memory and threads-not-sharing-memory is the same on most platforms.
A reminder that the task_t data structure describes threads and processes not just in Linux, but earlier Unixen also.
Well, Windows before NT isn't the same design as Windows 16 bit, it only shares the name for all practical purposes, and has more influence from OS/2 than Windows 16 bit.
Which is why I took the effort to explicitly refer to Windows NT on my comment, already expecting some traditional answers from UNIX folks.
Also due to historical reasons POSIX threads are the outcome of every UNIX going their own way implementing threads, finally coming to an agreement years later, with all the plus and minus of relying in POSIX for portable code.
whereas Unixen have a core task context model of a bunch of threads that by default do not share memory.
How are those not simply child processes? I don't understand your use of the word 'threads' here.
Does the Unix world not distinguish between threads and processes? In Win32, threads exist within processes, and you can create new threads or child processes.
Second answer: Linux doesn't differentiate between threads and processes. It has a "thread group ID" that serves a small number of purposes, and the rest of the difference is just whether the threads happen to share the same address space.
POSIX threads having problems with signals is, imho, mostly the problem with signals in general. They are pretty poorly designed: https://lwn.net/Articles/414618/
The problem is that threads are not fault boundaries but processes are. So they're not interchangeable when you care about resilience and misbehaving code.
True, but on Windows the approach is then to use COM servers, which have a faster IPC model, and can even serve multiple clients, depending on how the appartement space is configured.
Than UNIX fork/exec model, or calling into Create Process all the time.
Windows has a more rich set of IPC stuff than POSIX, especially since it has a microkernel like design.
If you are going to say it is everything on the same memory space anyway, it isn't.
Optional on Windows 10, and enforced on Windows 11, Hyper-V is always running, and several components including kernel and driver modules are sandboxed into their little worlds.
Several additional sandboxing changes were announced at BUILD.
I would say that pipes and shared memory are the IPC mechanisms? Controlling the state of the exec'd process's file descriptors would counts as a way to set up interprocess communication, but once that's done, it's the pipe or SHM that does the actual communication.
The problem with POSIX IPC is that passing file descriptors between processes (other than parent passing to child via fork) is hard. Yes, SCM_RIGHTS can do it, but it is quite error prone and rarely done.
That's like comparing apples and oranges. When tooling is tied to a platform, you're adding in the entire platform to the comparison.
Mozilla implemented an alternative to COM, called XPCOM. XP here means cross platform. Perhaps you could compare against that to take the platform out of the equation.
Windows was designed with threads-first mentality because on pre-386 machines you don't have viable process memory protection, so your tasks share memory by necessity. This is not a great argument.
Windows NT was never designed with pre-386 machines in mind. That was the territory of the old DOS+Windows. Windows NT from the get-go was for machines with page-based virtual memory.
This is not true. NT never had fork, was always based on the assumption of an MMU and Dave Cutler was a well known fork hater in the 80s long before this paper came out and made it cool to be so. By the time Windows 95 was out, the baseline was 386 with an MMU. CreateThread was initially designed for NT in 1993 though (which didn’t support pre-386 CPUs).
As mentioned elsewhere on this page, Windows NT had fork from the start. Vide NtCreateProcess and what happens if an image file is not explicitly supplied.
You haven't read the doco. I did point to some. The image file is supplied (or not) via the section object.
Think it through. Windows NT supported fork from the start in its POSIX subsystem, that subsystem was layered on top of the Native API, and this is the Native API mechanism that the POSIX subsystem employed. Although it took until Gary Nebbett for someone to publicly show how, even though people knew informally back in 1993.
NT was designed to be platform-agnostic, and its original target was the DEC Alpha. Its process model owes nothing to pre-386 CPUs. The WinAPI CreateProcess function is a layer atop NtCreateProcess, so that is where the pre-386 heritage lives. But even the WinAPI process model changed significantly with 32-bit Windows.
Windows NT was developed on various different CPUs before the Alpha was a thing. When it was released in 1993, it was released for three CPUs: IA-32, MIPS, and Alpha.
I suspect it's a long tail sort of thing; it mostly doesn't matter except when it really matters. It's interesting that the stated motivation for the patch is in the context of agentic tools spawning subcommands. There's some related prior art in this area where the payoffs could be much greater, like fuzzing: https://gts3.org/assets/papers/2017/xu:os-fuzz.pdf is an example. It would be very interesting to see this patch applied to e.g. AFL++
That's not the reason for the performance difference. Windows does have a fork primitive (ZwCreateProcess) and it's still slower than Linux's equivalent.
Again, NtCreateProcess does not implement fork(). The fundamental characteristic of fork is that the child is an exact replica of the parent, down to the instruction pointer. Windows does not have a way to create a process object with such a configuration.
Also, using the Zw prefix doesn’t make you look more knowledgeable, it makes you look like you’re trying way too hard to borrow credibility.
Okay but people don't claim that copying the instruction pointer (a single machine register) is the reason for any speed difference. They claim it's due to the memory sharing. And that's easily disproven since you can share pages, just like on Linux, simply by passing null for the section handle, yet there's still a performance difference.
Why does it matter which prefix I used? They both point to the same routine so my point applies either way.
It's a completely uncontroversial fact that NT does implement fork(). Turn to page 183 of Helen Custer's "Inside Windows NT" and you will read about it.
This paper is great and I also really like one of its references [29] as it goes into some more subtle parts of scalable interfaces, including fork. It's a gem IMO: The Scalable Commutativity Rule: Designing Scalable Software for Multicore Processors https://people.csail.mit.edu/nickolai/papers/clements-sc.pdf
The zygote pattern[1] is a great optimization to deal with the cost of forking, but IMHO, being able to inexpensively spawn a carefully tailored process regardless of the size and scope of the current process would be better.
I would guess it would be a small difference in measurable performance between zygote and a direct clean spawn, but it's one less trick an application needs to do, and it would be very helpful for libraries that spawn things. Spawning inside a library isn't always a great thing to do, but some things would really benefit from process level isolation.
[1] In case one isn't aware, the zygote pattern involves forking a 'zygote' process during application startup, and having that process do any forks that need to happen during application runtime. This reduces the cost of forking in large applications, because the zygote will have few fds open and use little memory. This lets your large application spawn new processes without delaying the application or the startup of the new processes. Some applications will spawn many zygotes to allow parallelism for spawning at runtime.
You're referring to something else, and maybe I'm using the term "zygote" incorrectly.
In all uses of zygotes that I have seen, here's what's really happening:
- `fork` is being used to reduce the cost of starting a process that has a high start-up cost. So, you start one process, run it through the expensive initialization, and then fork it from there to start new processes.
- To make this even faster, you have a pool of pre-forked processes sit around.
- Having pre-forked processes sitting around ready to be used is not expensive because of the CoW property and the fact that a process that forks and then immediately pauses will not have triggered any significant CoW yet.
So, the zygote optimization you speak of is in practice only meaningful on top of systems that are using an optimization uniquely enabled by `fork` (avoiding process initialization costs by cloning a process), and that zygote optimization is further optimized by another property of `fork` (memory sharing of forked processes that haven't done anything else yet).
Oh I see. I guess your zygotes have developed more than mine. I think Google may have coined or at least popularized the term zygote for this in Chrome and Android, Chrome documentation [1] says:
> A zygote process is one that listens for spawn requests from a main process and forks itself in response. Generally they are used because forking a process after some expensive setup has been performed can save time and share extra memory pages.
I think reading the first sentance and stopping covers my zygote, but adding the second sentance covers yours. So I think we're both right!
I think both paths are useful. If your children need time to startup and become ready, spawn one that does start up work, and then it (pre)forks at the ready state to have processes ready to handle requests (your zygote). This does require a traditional fork() to avoid duplication of work.
But if forking is expensive at runtime because you have a million FDs open and a whole lot of memory allocations, spawn spawners before you start doing work (my zygote). This could be unnecessary with a inexpensive way to spawn a new process from an process that has lots of resources in use.
Of course, you can also use my zygotes to spawn your zygotes. Zygoteception.
I quite like the idea. I’m using OpenBSD on an oldish laptop, and fork-exec is expensive enough that it conflicts with the usb subsystem. Isochronous transfers have a 1ms realtime requirement and it seem that the fork-exec system calls hold the giant lock long enough to mess with it (audio stutters).
While I’ve not bothered to profile it, but it seems that process that have lot of mapped pages is the issue (firefox, emacs,…). In the emacs case, the issue is when the main process trying to fork-exec, if I start a shell session (with shell-mode or term-mode), it works fine.
> Oh I see. I guess your zygotes have developed more than mine. I think Google may have coined or at least popularized the term zygote for this in Chrome and Android, Chrome documentation [1] says:
Google may have popularized the term, but this approach was already in use by KDE developers in the KDE 2.x timeframe, where it was used as part of a system called kdeinit.
In this scheme, launching KDE apps from a KDE desktop could bypass much of the startup cost of dynamic linking by forking from a long-running kdeinit process (with kdeinit itself deliberately linked to all large dependency libs like Qt and kdelibs), dynamically loading the application logic (stored as a .so) and then launching the app.
This was more to save startup time due to how long it took to dynamically resolve a multitude of C++-based symbols back then, all the common logic came before the app's own main() would ever be called. But it did also save a bit of memory as well.
adding on the the sibling, what argument to clone allows me to set the fds of the child? AFAIK, you either share the FD table with the parent, or get a copy of it. If the parent has 1 million FDs open and the child doesn't want most of those, dealing with that has real costs. Many applications that tend to have large numbers of FDs and also fork/exec will mitigate the cost by spawning a process during startup that they can then use to spawn processes during runtime without doing it from the main process; this is a nice mitigation, but it shows a missing interface.
The paper explicitly covers it that various memory COW/snapshot mechanisms are probably faster and safer than the zygote pattern. As it stands getting the zygote pattern correct and safe is something you have to plan for upfront. You can’t retrofit it which is why the paper mentions it has poor composability. Also the advantages of the zygote pattern can be overstated since the memory sharing benefit is minimal since it has to happen so early and modern OSes already transparently CoW duplicate pages in the background.
I recommend at least skimming the paper as it covers this. But essentially you can’t just inject a call at a random point in code to start being a zygote. It’s something you have to plan up front as to the exact point you’re going to fork and that you’re going to do it at the start of program before any threads have started or any files are open and before any locks have been acquired. It’s basically all the challenges of invoking fork at arbitrary points in time.
The reason to do a zygote in the first place could be solved with alternative special APIs that are safer and harder to misuse. But we have fork so there’s not as big of a demand despite the warts.
Yes, zygote pattern makes it easy to make fork() into bottleneck - it requires a lot more discipline and low level tricks (linker scripts, compiler-specific extensions, custom sections, low level dependencies on pagesize that get "fun" on ARM servers).
If you don't, you might wake up with fork() causing latency issues.
You can create threads in the zygote. It doesn't "break down", but sure, there's a bit more work.
My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable, preferably lazily (they resume when they are actually needed). So, the zygotes are sitting with those threads suspended. When they become active, they can do work immediately. They might lazily resume those threads as needed.
There are other idioms for this too.
> Raw fork() is terrible. Instead we need a proper primitive to stop and make a snapshot of a process.
Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives
> My trick for that is that the set of threads that I create pre fork have to be suspendable and resumable
Well, yes. You need to wait for all the threads to park themselves at safepoints. This can work if you control the whole runtime, and you don't use something that creates threads behind your back.
This is actually why I've always been interested in a better fork(), it has a lot of parallels with stop-the-world needed for GCs.
> Folks have been saying that it's terrible for as long as I can remember. But it's still there, because it's better than the alternatives
I don't think we have alternatives? Except maybe ptrace()?
I want to be able to install apps from alternative app stores like F-Droid and receive automatic updates, without requiring Google's authorization for app publication.
Manually installing an app via adb must, of course, be permitted. But that is not sufficient.
> Keeping users safe on Android is our top priority.
Google's mandatory verification is not about security, but about control (they want to forbid apps like ReVanced that could reduce their advertising revenue).
When SimpleMobileTools was sold to a shady company (https://news.ycombinator.com/item?id=38505229), the new owner was able to push any user-hostile changes they wanted to all users who had installed the original app through Google Play (that's the very reason why the initial app could be sold in the first place, to exploit a large, preexisting user base that had the initial version installed).
That was not the case on F-Droid, which blocked the new user-hostile version and recommended the open source fork (Fossify Apps). (see also this comment: https://news.ycombinator.com/item?id=45410805)
Yes, it's all about control. Control the platform. Control the access to the platform, and the world is your oyster. And the political and legislation system are their friends. It is the establishment.
The only way to fight is to indoctrinate the next generation, at home, and in school, to use FOSS. People tend to stick to whatever they used in childhood. We the software engineers should volunteer in giving speeches to students about this. It is much easier to sell ideologies to younger people when they are rebellious to the institutions.
I agree with you. But you do realize that it's been like that since about 20 years now. It started because of Microsoft (proprietary software), then Google (propriteary platform), now ChatGPT (proprietary knowledge).
And I tried to tell my kids. And it failed mostly.
But in the long run (a decade), what is exceptional and proprietary will become common FOSS. And everybody will benefit.
I envision this as an ideology. We don't need every kid to follow it, and I don't expect the majority to follow. A 1-2% is good enough. That's why giving speeches to teenages might be the best bang for the buck. There are always kids who need to escape into some cool ideas and it could be the idea of FOSS.
Really its probably the dumbass judge that told Google "The apple app store isn't anti-competitive because they don't allow any competitors on their platform" when google asked why the play store was ruled a monopoly and the app store wasn't.
I cannot think of a more detached and idiotic ruling than that.
The US anti trust legislation punishes the abuse of monopoly power, not being monopoly in itself. Google was found guilty in leveraging their dominating position on the platform to do just that.
On the other hand in the US Apple's App Store was not found to be a monopoly in the first place. Different cases about abusing dominating position also didn't go far.
Hmm, having read that, I am starting to sympathize with Google if they are going to be punished for being open.
No one seems to care that Apple has never allowed freedom on their devices. Even the comments here don't seem to mention it. Google was at least open for a while.
Or maybe no one mentions it just because the closed iPhone is a fait accompli at this point.
Perhaps because Apple never “promised” to be open, Google instead built itself by playing the good guy and started to switch when money called so those who chose them for that reason feel betrayed.
Because that's the law, like it or not. Apple doesn't have a problem because the rules were the rules from day 1. Google did a bait and switch, legally.
What does antitrust law have to do with "day 1"? So if Ford and GM are both already in all 50 states and then they try to divide up territory between them, that's illegal, but at the point when there were still areas one of them wasn't in, they could publicly announce a contractual agreement to not enter into the other's territory? That seems not just questionable but actively bad policy with an enormous perverse incentive.
And if you're going to say this:
> Because that's the law, like it or not.
I would ask you to point me to the text in the statute requiring the courts to do that.
But the ruling is correct. You can't have it both ways, if you invite competition you're not allowed to be anti-competitive. You can be Nintendo, offer a single store, only allow first party hardware, and exercise total control over your product. Then your anticompetitive behavior can only be evaluated externally. But if you open yourself up to internal competition with other phone vendors, other stores, and then you flex your other business units (gapps) to force those other vendors to favor you then you're in big trouble.
> But the ruling is correct. You can't have it both ways, if you invite competition you're not allowed to be anti-competitive
That's just stupid, because being anti-competitive is an emergent outcome, rather than anything specific.
Apple is definitely anti-competitive, but they exploited such a ruling so that they can skirt it. Owning a platform that no other entrants are allowed is anti-competitive - whether you're small or large. It's only when you're large that you should become a target to purge via anti-competitive laws. This allows small players to grow, but always face the threat of purging - this makes them wary of trying to take advantage too much, which results in better consumer outcomes.
That's like Karcher opening a megamall to sell all their offering, vacuums, pressure washers, floor washers, you name it .. and then you, Bosch, complaining you can't sell your vacuum in Karcher's megamall where all the people go.
What are you even saying?
Whereas google was letting Bosch sell vacuums in their megamall, but only if it uses Google dust filters and people buy only Google made dust filters and Bosch isn't allowed to sell their own dust filters in the megamall.
It's like a company buying all the land within a 100 mile radius and then nominally "selling" plots to people but with terms of service attached that restrict what you can do with the land you bought and that allow the company to change the terms at any time. And then, after people have moved in, most of them having not even read the terms or realized it wasn't an ordinary sale, they start enforcing the terms against competitors. Which most people don't notice because they aren't competitors, and because the terms also prohibited anyone in the city from telling people what's going on[1]. Then people eventually notice and start to ask whether terms locking out competitors like that are an antitrust violation, and someone says that they're not because the people there agreed to them.
But how is an agreement prohibiting people from patronizing competitors not an antitrust violation? It's not a matter of who agreed to it, it's matter of what they're requiring you to agree to.
> nominally "selling" plots to people but with terms of service attached that restrict what you can do with the land you bought and that allow the company to change the terms at any time.
> Karcher opening a megamall to sell all their offering
And their mall is monopolistic if it is only for Karcher products. However, because a competitor can easily open a mall next door, it means this Karcher mall is small, and so the enforcers should leave it be. Until the day Karcher buys up all the mall space, in which case, they (regulators) start purging their mall monopoly.
The threat of being purged because you've acquired a large enough monopoly should _always_ be there. It's part of doing business in a fair environment.
> You can be Nintendo, offer a single store, only allow first party hardware, and exercise total control over your product.
How is this not even more anti-competitive?
It's fine to be mad at Google for being duplicitous, but treachery is in the nature of false advertising or breach of contract. Antitrust is something else.
"You can monopolize the market as long as you commit to it from the start" seems like the text of the law a supervillain would be trying pass in order to destroy the world.
You can't monopolize a market where there is no market. Nintendo can be anticompetitive in the wider games industry, but there is no market for software that runs on a Switch.
I didn't say I liked the ruling, just that it's correct. The opposite conclusion would be absurd, that you can invent a market where there isn't one and claim a company has a monopoly over it. You would be asking the court to declare that every computing device is a de facto marketplace for software that could run on it and that you can't privilege any specific software vendor. I would love if that were true but you can hopefully agree that such a thing would be a huge stretch legally.
> You can't monopolize a market where there is no market. The opposite conclusion would be absurd, that you can invent a market where there isn't one and claim a company has a monopoly over it.
There is no such thing as "there is no market". There is always a market. The question is, what's in the market? The typical strategy is to do the opposite -- have Nintendo claim that they're competing with Sony and Microsoft in the same market to try to claim that it isn't a monopoly.
But then the question is, are they the same market? So to take some traditional examples, third party software that could run on MS-DOS could also run on non-Microsoft flavors of DOS. OS/2 could run software for Windows. The various POSIX-compliant versions of Unix and Linux could run the same software as one another. Samsung phones can run the same apps as Pixel phones. Which puts these things in the same market as each other, because they're actually substitutes, even though they're made by different companies.
Conversely, you can't run iOS apps on Android or get iOS apps from Google Play or vice versa. It's not because they're different companies -- both of them could support both if they wanted to -- it's that they choose not to and choices have consequences.
If you intentionally avoid competing in the same market as another company then you're not competing in the same market as that company and the absurdity is trying to have it both ways by doing that and then still wanting to claim them as a competitor.
You avoided the important part, there is no market for hardware that can play Nintendo Switch games and there is no market for software providers on Nintendo Switch. And they are legally allowed to do that. You can sell appliances that are bound to a single vendor and you are allowed to not license your hardware or software to 3rd parties.
Since that is a legally permissible action it would be an odd thing for a court to declare that doing such a thing is anticompetitive. If they did they would be declaring all locked down hardware effectively illegal. And while that might be nice it's a bit of a pipedream. Where Google fucked up is that they did license their software to 3rd parties—good for them. But then Google had some regrets and didn't like the fact that they didn't have control over those 3rd parties. But they did have
some leverage in the form of Google Play and GSM because users expect it to be there on every Android phone. And then they used that leverage. That's the fuckup. They used Google Play and GSM access to make 3rd parties preinstall Chrome and kill 3rd party Android forks. They used anticompetitive practices on their competitors—other Android device manufacturers.
This situation can't occur for Apple or Nintendo because there aren't other iOS/Switch device manufacturers and they don't have to allow them to exist. They can be anticompetitive for other reasons but not this.
> You avoided the important part, there is no market for hardware that can play Nintendo Switch games and there is no market for software providers on Nintendo Switch.
There is a market for these things. Nintendo sells hardware that can play Nintendo Switch games and people buy it. That's a market.
It seems like you're trying to claim that a monopoly isn't a market, but how can that possibly be how antitrust laws work? Your argument is that they don't apply to something if it is a monopoly?
> And they are legally allowed to do that.
That's just assuming the conclusion. Why should it be legal for them to exclude competitors from selling software to their customers? The obviously anti-competitive thing should obviously be a violation of any sane laws prohibiting anti-competitive practices. The insanity is the number of people trying to defend the practice.
Consider what it implies. 20th century GE could have gone around buying houses, installing a GE electrical panel and then selling the houses with a covenant that no one could use a non-GE appliance in that house ever again, or plug in any device that runs on electricity without their permission. They could buy and sell half of all the housing stock in the country and Westinghouse the other half and each add that covenant and you're claiming it wouldn't be an antitrust violation.
Apple wouldn't have been able to get their start because they'd have needed permission from GE or Westinghouse for customers to plug in an Apple II or charge an iPhone and they wouldn't get it because those companies were selling mainframes or flip phones and wouldn't want the competition. If that's not an antitrust violation then we don't have antitrust laws.
> If they did they would be declaring all locked down hardware effectively illegal.
It's fine for hardware to be locked down by and with the specific permission of the person who owns it. But how is it even controversial for the manufacturer locking down hardware for the purpose of excluding competitors to be a violation of the laws against inhibiting competition? It's exactly the thing those laws are supposed to be prohibiting.
Yeah we are fucked, but as long as a small percentage of us, like 1% of the population knows, understands and agrees with the idea I think we are fine.
I don't really see how you can both allow developers to update their apps automatically (which is widely promoted as being good security practice) and also defend against good developers turning bad.
How does Google know if someone has sold off their app? In most cases, F-Droid couldn't know either. A developer transferring their accounts and private keys to someone else is not easily detected.
F-Droid is quite restrictive about what kinds of app they accept, they build the app from source code themselves, and the source code must be published under a FLOSS license. They have some checks that have to pass for each new version of an app.
Although it's possible for a developer to transfer their accounts and private keys to someone shady, F-Droid's checks and open source requirements limit the damage the new developer can do.
One thing worth noting, these checks and restrictions only apply if you're using the original F-Droid repository.
Many times I've seen the IzzyOnDroid repository recommended, but that repo explicitly gives you the APKs from the original developers, so you don't get these benefits.
That's true. The whole point of an open ecosystem is that you get to decide who you get your software from. You can decide on the official F-Droid repository and get the benefits and drawbacks of a strict open source rule with the F-Droid organization's curation if that's your preference. You can add other repositories with different curation if you prefer that.
Anybody slightly competent can put horrendous back doors into any code, in such a way that they will pass F-Droid's "checks", Apple's "checks", and Google's "checks". Source code is barely a speed bump. Behavioral tests are a joke.
Anyone determined enough can break into any house. If not through ingenuitiy, then by a brick to your window. Doesn't mean we shouldn't lock our doors, turn off our lights, and close our curtains anyway.
The fortunate thing is that 99% of people won't bother trying to break your app if it's not dead simple. Advanved security mechanisms to check for backdoors is probably something only billionaire tech companies need to worry about.
You totally misunderstand the threat model. It's not about anybody breaking your app. It's about people making their own apps do things they're not supposed to do.
... and there's always a tradeoff in terms of how much of a deterrent anything is. The app store checks are barely measurable.
The app store checks are barely measureable, yes. Hence why being open source is the best check for any undocumented changes. Even if it's not discovered on FDoid, reports will come out for those who dig. Much easier to view source code than decompiling an APK to analyze.
But at some point there needs to be some level of trust in anything you install. You can't rely on institutions to make sure everything is squeaky clean. They can't even do that on content platforms (or at least, they choose not to afford it).
> In most cases, F-Droid couldn't know either. A developer transferring their accounts and private keys to someone else is not easily detected.
1. The Android OS does not allow installing app updates if the new APK uses a different signing key than the existing one. It will outright refuse, and this works locally on device. There's no need to ask some third party server to verify anything. It's a fundamental part of how Android security works, and it has been like this since the first Android phone ever release.
2. F-Droid compiles all APKs on its store, and signs them with its own keys. Apps on F-Droid are not signed by the developers of those apps. They're signed by F-Droid, and thus can only be updated through and by F-Droid. F-Droid does not just distribute APKs uploaded by random people, it distributes APKs that F-Droid compiled themselves.
So to answer your question, a developer transferring their accounts/keys to someone else doesn't matter. It won't affect the security of F-Droid users, because those keys/accounts aren't used by F-Droid. The worst that can happen is that the new owner tries injecting malware into the source code, but F-Droid builds apps from source and is thus positioned to catch those types of things (which is more than can be said about Google's ability to police Google Play)
And finally,
> How does Google know if someone has sold off their app?
Google should not know anything about the business dealings of potential competitors. Google is a monopoly[1], so there is real risk for developers and their businesses if Google is given access to this kind of information.
Android also has the feature of warning the user if an update is coming from a different source than what is installed. This will happen even if they have the same key. This reply isn't trying to argue against anything you've said. I am just adding to the list of how Android handles updates.
> F-Droid compiles all APKs on its store, and signs them with its own keys. Apps on F-Droid are not signed by the developers of those apps. They're signed by F-Droid, and thus can only be updated through and by F-Droid. F-Droid does not just distribute APKs uploaded by random people, it distributes APKs that F-Droid compiled themselves.
For most programs I use, they just publishing the developer's built (and signed) APK. They do their own build in parallel and ensure that the result is the same as the developer's build (thanks to reproducible builds), but they still end up distributing the developer's APK.
Can you give some examples? I've heard that's a thing, but I'm not familiar with any apps that actually pull it off (reproducible builds are difficult to achieve)
Reproducible builds may be hard to achieve, but that doesn't mean you don't have a list of such builds long enough to crash your browser: https://verification.f-droid.org/verified.html
How do I know they aren’t infiltrated by TLAs? (Three Letter Agencies), or outright bad-actors.
Didn’t F-Droid have 20 or so apps that contained known vulnerabilities back in 2022?
Who are all these people? Why should I trust them, and why do most of them have no link to a bio or repository, or otherwise no way to verify they are who they say they are and are doing what they claim to be doing in my best interests?
I trust them, at least a lot more than I do Google, which is a known bad actor, and collaborator with "TLAs". F-Droid has been around for a very long time, if you didn't know. They've built and earned the trust people have in them today.
> Didn’t F-Droid have 20 or so apps that contained known vulnerabilities back in 2022?
Idk what specific incident you're referring to, but since they build apks themselves in an automated way, if a security patch to an app breaks the build, that needs to be fixed before the update can go out (by F-Droid volunteers, usually). In that case, F-Droid will warn about the app having known unpatched vulnerabilities.
Again, this is above and beyond what Google does in their store. Google Play probably has more malware apps than F-Droid has lines of code in its entire catalog.
Right, that's literally the team marking 12 apps as having known vulnerabilities (seems like it was because of a WebRTC vulnerability that was discovered). It's the F-Droid system working as intended to inform users about what they're installing.
You're calling it an incident like it was an attack or something, but it just seems like everyday software development. Google Play and the App Store don't let me know when apps have known vulnerabilities. I think F-Droid is coming out way ahead here.
So Google and Apple are already known to work with US government agencies. This was revealed in the Snowden leaks in 2013, and confirmed on multiple occasions since. Neither Google nor Apple tell you when apps you're downloading from the store contain known vulnerabilities. We know for a fact that both Google Play and the App Store are filled with scams and malware: it's widely documented.
So to my reading F-Droid comes out ahead on every metric you've listed: It has no known associations with US government agencies. They do inform you when your apps have known vulnerabilities. I'm not aware of any cases of scams or malware being distributed through F-Droid.
I highly recommend it. It's the main store I've been using on my phone for probably more than a decade now.
I understand your concern, though your suspicion is a little shortsighted. It can be personally dangerous to volunteer for projects that directly circumvent the control of the establishment.
For the same reason you trust many things. They have a long track record of doing the right thing. As gaining reputation for doing the wrong thing would more or less destroy them, it's a fair incentive to continue doing the right thing. It's a much better incentive that many random developers of small apps in Google's play store have.
However, that's not the only reason to trust them. They also follow a set of processes, starting with a long list of criteria saying what app's they will accept https://f-droid.org/docs/Inclusion_Policy/ That doesn't mean malware won't slip past them on occasion, but if you look at the amount of malware that slips past F-Droid and projects with similar policies like Debian and compare them to other app stores like Google's, Apple and Microsoft there is no comparison. Some malware slips past Debian's defences once every few years. I would not be surprised if new malware isn't uploaded to Google app store every few minutes. The others aren't much better.
The net outcome of all that is the open source distribution platforms like F-Droid and Debian, that have procedures in place like tight acceptance policies and reproducible builds are by a huge margin the most reliable and trustworthy on the planet right now. That isn't saying they are perfect, but rather if Google's goal is to keep their users safe they should be doing everything in their power to protect and promote F-Droid.
> How do I know they aren’t infiltrated by TLAs? (Three Letter Agencies), or outright bad-actors.
You don't know for sure, but F-Droid policies make it possible to detect if the TLA did something nefarious. The combination of reproducible builds, open source and open source's tendency to use source code management systems that provide to audit trail showing who changed every line shine a lot of sunlight into the area. Sunlight those TLA's your so paranoid about hate.
This is the one thing that puzzles me about F-Droid opposition in particular. Google is taking a small step here towards increasing accountability of app developers. But a single person signing an app is in reality a very small step. There are likely tens if not hundreds of libraries underpinning it, developed by thousands of people. That single developer can't monitor them all, and consequently libraries with malware inserted from upstream repositories like NPM or PyPi regularly slips through. Transparency the open source movement mostly enforces is far greater. You can't even modify the amount of whitespace in a line without it being picked up by some version control system that records who did it, why they did it, and when. So F-Droid is complaining about a small increase in enforced transparency from Google, when they demand far, far more from their contributors.
I get that Google's change probably creates some paper-cuts for F-Droid, but I doubt it's something that can't be worked around if both sides collaborate. This blog post sounds like Google is moving in that direction. Hear, hear!
You can run any software you like on Android, if it's open source. You just compile it yourself, and sign it with the limited distribution signature the blog post mentions. Hell, I've never done it, but re-signing any APK with your own signature sounds like it should be feasible. If it is, you can run any APK you want on your own hardware.
Get a grip. Yes it might be possible the world is out to get you. But it's also possible Google is trying to do exactly what they say on the tin - make the world a safer place for people who don't know shit from clay. In this particular case, if they are trying to restrict what an person with a modicum of skillz can do on their own phone it's a piss poor effort, so I'm inclined to think it's the latter. They aren't even removing the adb app upload hole.
>> In most cases, F-Droid couldn't know either. A developer transferring their accounts and private keys to someone else is not easily detected.
> 1. The Android OS does not allow installing app updates if the new APK uses a different signing key than the existing one. It will outright refuse, and this works locally on device
You missed the and private keys part of the original claim.
If an app updates to require new permissions, or to suddenly require network access, or the owner contact details change, Google Play should ideally stop that during the update review process and let the users know. But that wouldn't be good for business.
An update can become malicious even without change in permissions.
E.g. my now perfectly fine QR reader already has access to camera (obvious), media (to read QR in an image file or photo) and network (enhanced security by on-demand checking the URL for me and showing OG etc so I can more informed choose to open the URL)
But it could now start sending all my photo's to train an LLM or secretly make pictures of the inside of my home, or start mining crypto or whatnot. Without me noticing.
See that's what the intent system was originally designed to prevent.
Your QR reader requires no media permission if it uses the standard file dialogs. Then it can only access files you select, during that session.
Similarly for the camera.
And in fact, it should have no network access whatsoever (and network should be a user controllable permission, as it used to be — the only reason that was removed is that people would block network access to block ads)
> And in fact, it should have no network access whatsoever (and network should be a user controllable permission, as it used to be — the only reason that was removed is that people would block network access to block ads)
Sure, a QR code scanner can work fine without network.
E.g. it could use the network to check a scanned URL against the "safe browsing API" or to pre-fetch the URL and show me a nice OG preview. You are correct to say you may not need nor want this. But I and others may like such features.
Point is not to discuss wether a QR scanner should have network-access, but to say that once a permission is there for obvious or correct reasons, it can in future easily get abused for other reasons. Without changing the permissions.
My mail-app needs network. Nothing prohibits it from abusing this after an update to pull in ads, or send telemetry to third parties. My sound record app needs microphone permissions. Nothing prohibits it from "secretly" recording my conversations after an update (detectable since a LED and icon will light up).
If you want to solve "app becoming malicious after an update", permissions aren't the tool. They are a tiny piece of that puzzle, but "better permissions" aren't the solution either. Nor is "better awareness of permissions by users".
> See that's what the intent system was originally designed to prevent.
> Your QR reader requires no media permission if it uses the standard file dialogs. Then it can only access files you select, during that session.
On the one hand, yes, good point, but it runs into the usual problem with strict sandboxing – it works for the simple default use case, but as soon as you want to do more advanced stuff, offer a nicer UI, etc. etc. it breaks down.
E.g. barcode scanners – yes, technically you could send a media capture intent to ask the camera app to capture a single photo without needing the camera permission yourself, but then you run into the problem that maybe the photo isn't suitable enough for successful barcode detection, so you have to ask the user to take another picture, and perhaps another, and another, and…
So much nicer to request the camera permission after all and then capture a live image stream and automatically re-run the detection algorithm until a code has been found.
The network permission was displayed in the first versions of Android, then removed. I heard (hearsay alert) at the time that it was because so many apps needed it, and they wanted to get rid of always-yes questions. IIRC this happened before the rise of in-app advertising.
If people always answer yes, they grow tired and eventually don't notice the question. I've seen it happen with "do you want to overwrite the previous version of the document you're editing, which you saved two minutes ago?" At that point your question is just poisoning the well. Makes sense, but still, hearsay alert.
As far as I'm concerned they can grant this permission by default. I just want the power to disable it.
A while ago I wanted to scan the NFC chip in my passport. Obviously, I didn't want this information to leave my device.
There are many small utility apps and games that have no reason to require network access. So "need" is not quite the right word here. They _want_ network access and they _want_ to be able to bully users into granting it.
That's a weird justification for granting it by default. But I wouldn't care if I could disable it.
Android doesn't grant this by default, strictly speaking. Rather, an application can enable it by listing it in the application manifest. Most permissions require a question to to the user.
Well, the original intent was to ask the user for permission at installation time, which turned out to be a poor idea after a while. Perhaps you mean that it would have been simple to change the API in some particular way, while retaining compatibility with existing apps? If I remember the timeline correctly, which is far from certain, this happened around the same time as Android passed 100k apps, so a fairly strong compatibility requirement.
I mean, just make it "Granted" by default and give user ability to control it. Permissions API was already broken few times(i.e. Location for bluetooth and granular Files permissions)
> Does Google refuse to veriy their firmware if they offer this feature?
If a manufacturer doesn't follow the Android CDD (https://source.android.com/docs/compatibility/cdd), Google will not allow them to bundle Google's closed source apps (which include the Google Play store). It was originally a measure to prevent fragmentation. I don't know whether this particular detail (not exposing this particular permission) is part of the CDD.
It's not explicitly part of the CDD, but implicitly. The device must support the Android permissions model and is only allowed to extend this implementation using OWN permissions (in a different namespace than 'android'), but not allowed to deviate from it.
INTERNET is a "normal permission", automatically granted at install time if declared in the manifest.
OEMs cannot change the grant behavior without breaking compatibility because:
The CDD explicitly states that the Android security model must remain intact.
Any deviation would fail CTS (Compatibility Test Suite) and prevent Play certification.
Well, apart from the OEM violating the Android Compatibility Definition Document (CDD), failing the Compatibility Test Suite (CTS) and thus not getting their device Play-certified (so not being able to preload all the Google services, there is an economical impact as well:
As OEM you want Carriers to sell your device above everything else, because they are able to sell large volumes.
Carriers make money using network traffic, Google is paying Revenue-Share for ads to Carriers (and OEMs of certain size). Carriers measure this as part of the average revenue per user (ARPU).
--> The device would be designed to create less ARPU for the Carrier and Google and thus be less attractive for the entire ecosystem.
I've been using a similar VPN solution. It works great for apps that absolutely should not be connected, like my keyboard. But it has an obvious downside: you can't use a VPN on your phone while you're using that.
Some apps would use this for loopback addresses, which as far as I know will then need network permission. The problem here is the permission system itself because ironically Google Play is full of malicious software.
And neither Android nor iOS a safer than modern Desktop systems. On the contrary because leaking data is its own security issue.
Yes. Facebook/Meta was using a locally hosted proxy to get info smuggled back without using routes that are increasingly obstructed by things like ad blockers if I recall correctly.
This is a huge problem in the Chrome Web Store and Google is doing very little about it. If you ever made an extension that is even just a little popular, expect to get acquisition offers by people who want to add malicious features somewhere between click fraud, residential IP services or even password stealers.
Same for Play Store. I have 2 games and I keep getting offers all the time. The last one offered $2000 for the developer account or a $100 monthly rent.
From their email pitch:
> We’re now offering from $500 to $2000 for a one-time purchase of a developer account that includes apps, or a rental deal starting from $100.
> No hidden conditions — quick process, secure agreement, and immediate payment upon verification.
> We’re simply looking for reliable accounts to publish our client apps quickly, and yours could be a perfect match.
Indeed, an update can't be more malicious than the permissions allow it to be. You have a calculator app with limited permissions, it is "safe" to set to allow the developer to update it. No danger in that.
But I don't think it is enough, or it is the right model. In other cases, when the app has dangerous permissions already, auto-update should be a no-go.
F-Droid is not just a repository and an organization providing the relevant services, but a community of like-minded *users* that report on and talk about such issues.
> which is widely promoted as being good security practice
Maybe that's the mistake right there?
It is a good practice only as long as you can trust the remote source for apps. Illustration: it is a good security practice for a Debian distro, not so much for a closed source phone app store.
The point here is that app developers have to identify themselves. Google has no intention to verify the content of sideloaded apps, just that it is signed by a real person, for accountability.
They don't know if the person who signed the app is the developer, but should the app happen to be a scam and there is a police investigation, that is the person who will have to answer questions, like "who did you transfer these private keys to?".
This, according to Google and possibly regulators in countries where this will be implemented, will help combat a certain type of scam.
It shouldn't be a problem for YouTube Vanced, at least in the proposed form. The authors, who are already idendified just need to sign their APK. AFAIK, what they are doing is not illegal or they would have been shut down long ago. It may be a problem for others though, and particularly F-Droid, because F-Droid recompiles apps, they can't reasonably be signed by the original author.
The F-Droid situation can resolve itself if F-Droid is allowed to sign the apps it publishes, and in fact, doing that is an improvement in security as it can be a guarantee that the APK you got is indeed the one compiled by F-Droid from publicly available source code.
APKs are already signed. Now Google requries that they be signed by a key which is verified by their own signatures. Which means they can selectively refused to verify whichever keys are inconvenient to them.
Still believe that signing binaries this way is always bullshit.
I stopped developing for mobile systems ages ago because it just isn't fun anymore and the devices are vastly more useless. As a user, I don't use apps anymore either.
But you can bet I won't ever id myself to Google as a dev.
> I don't really see how you can both allow developers to update their apps automatically (which is widely promoted as being good security practice) and also defend against good developers turning bad.
These are not compatible, but only because the first half is simply false. Allowing a developer to send updates is not "good" but "bad" security practice.
This exactly. Transferring ownership is a business transaction. Track that. If the new owner is trying to hide it, this is fraud, and should be dealt with in court.
This is a big problem with Chrome extensions and Google hasn't done anything about it there, so I don't think they actually care about it. I'm not actually sure how you would solve that problem even theoretically.
> without requiring Google's authorization for app publication.
funnily enough, I am installing google drive for computers right now (macOS), I had to download a .pkg and basically sideload the app, which is not published on the Apple Store
>I had to download a .pkg and basically sideload the app, which is not published on the Apple Store
You mean install the app? The fact that Apple and Google wish to suggest that software from outside their gardens is somehow subnormal doesn't mean other people need to adopt their verbiage.
Probably because they require APIs which cannot be used when publishing to the AppStore. The whole Microsoft Office Suite is available in the macOS App Store - but Microsoft Teams must be downloaded from their website and cannot be installed via the AppStore...
Bad example because that .pkg was probably signed with a developer certificate with approval from Apple - just as would be the case on Android in the future.
> Your personal data will be kept safe on our servers, citizen, whether you like it or not.
... and our business partners. And app developers that grab your clipboard. And their business partners. and a few more levels of data brokers. The spi^H^H^H data-vacuum must flow
And of course, code signing can't protect you from such a thing. When software publishing rights get bought, so (usually) do the signing keys.
Curation (and even patching) by independent, third-party volunteers with strong value commitments does protect users from this (and many other things). Code signing is still helpful for F/OSS distributions of software, but the truth is that most of the security measures related to app installation serve primarily to solve problems with proprietary app markets like Google's Play Store and Apple's App Store. Same thing with app sandboxing.
It's unfortunate but predictable when powerful corporations taint genuine security features (like anti-tampering measures, built-in encryption devices, code signing, sandboxing, malware scanning, etc.) by using them as instruments of control to subdue their competitors and their own users.
The entire SimpleMobileTools situation left such a bad taste in my mouth. No upfront communication, it had to be discovered in a GitHub issue thread after people started asking questions.
It was shady as fuck on Kaputa's part, especially given ZipoApps is an Israeli adware company, a.k.a. surveillance company, and given Israel's track record with things like using Pegasus against journalists/activists or blowing up civilian-owned beepers, this should automatically be a major security incident and at least treated as seriously as the TikTok debacle.
Kaputa should be extremely ashamed of himself and outted from the industry. I and many others would have gladly paid a yearly subscription for continued updates of the suite instead of a one-time fee, but instead of openly discussing such a model with his userbase, he went for the dirtiest money he could find.
> If "automatic updates" were optional and off-by-default then users would not be vulnerable to something like SimpleMobileTools
The problem is the vast majority of users want this on by default; they don't want to be bothered with looking at every update and deciding if they should update or not.
> I want to be able to install apps from alternative app stores like F-Droid and receive automatic updates
That's actually possible, though app stores need to implement the modern API which F-Droid doesn't seem to do quite well (the basic version of F-Droid (https://f-droid.org/eu/packages/org.fdroid.basic/) seems to do better). Updating from different sources (i.e. downloading Signal from GPlay and then updating it from F-Droid or vice versa) also causes issues. But plain old alternative app stores can auto-update in the background. Could be something added in a relatively recent version of Android, though.
If this Verified bullshit makes it through, I expect open source Android development to slowly die off. Especially for smaller hobbyist-made apps.
Simplifying `git rebase -i` is a great idea, but unfortunately, that does not match my workflow.
I always keep an history of my branches. For example, when I develop a feature, the first version of the branch is `myfeature.1`, then `myfeature.2`, and so on. This is useful because I can retrieve an old version when something behaves differently in a newer one. (And no, `git reflog` will not help)
This has saved me multiple times. For example, I can determine that a problem was introduced between `myfeature.58` and `myfeature.59`. If the "feature" contains 15 commits, I do:
This lets me see the changes between the 2 versions, commit by commit (even if they don't have the same base).I don't want all the branches containing a commit to be rebased automatically (I already try using tags for old versions instead, but this does not quite fit).