For those scenarios, structured storage fits the bill. The app requests folder permission and the user, using system UIs, grants permissions on the folders they want to enable.
>it makes sense to let the users give it access to all the files on the phone
It doesn't even pretend to be a backup app, and further the permission we're talking about is limited to external storage (though that is a nebulous term on many Android devices where internal storage is split-brained on being internal and partly external).
Further saying "let the user decide" works great in theory and with a considered, rational userbase. In reality it means that everyone just says sure to everything, and soon all of the user's data is exfiltrated and everyone is whining that Google/Apple/et al should have forseen this.
The problem lies that in newer Android versions (11 or higher) you cannot give permissions to arbitrary folders, it's only "shared folders" or "all the files" [0][1] (It's also explained in TFA)
I'm sure they would happily allow you to select the folders you want to give permissions to, but I think that's not possible anymore.
So if you want to sync a local data folder that stores, for example, the tracks you record with your GPS, you cannot do it without full access unless your GPS app allows you to select a shared folder to store its data.
KDE Connect allows me to browse the folder I picked and upload/download files just fine, using SAF. You point the app at a directory, click "grant access", and you have permanent access. The only problem I can think of is that some directories are restricted (https://developer.android.com/about/versions/11/privacy/stor...) because they're a dumping ground for files already.
SAF provides both provider and consumer APIs, and from the language NextCloud uses, I get the feeling they might've missed the consumer API here. Had NextCloud said "we want the user to be able to select the Download folder and certain external media directories" then I would've thought the restrictions are the main problem, but that doesn't seem to be the case here.
This is all guessing by me, but I think that, traditionally, there's been a certain quantity of users that synced folders out of the /Android/data or similar locations and/or needed some extra flexibility. When Android 11 arrived, it was difficult to break that workflow for all those users (and I guess there was also some re-architecture needed for the app, but I don't know how much would it be).
I went through the same with Syncthing, and now I install it from out of Google Play, where it doesn't need Google approval to do the All Files Permission API call.
I understand why the API changes and the hardening of the Android system, but it's also true you're locked out from certain folders even when on developer mode and it feels weird.
It is correct that you cannot access private app folders, meaning stuff like /data/data, /Android/data, /Android/obb. But this is nothing new and you basically need a rooted device to access them. You can however access anything on external storage, which is typically where apps would store data you would want to sync and which would cover by far most use cases from users. At the moment, users are mostly complaining that only media files are synced, but not documents. SAF would solve this.
> I'm sure they would happily allow you to select the folders you want to give permissions to, but I think that's not possible anymore.
Well, it used to be possible before Android 11, that's the thing.
So, following a bit the example I was giving, if you want to sync data from another app to backup it (or whatever), and that App stores its data in the default data folder assigned by Android, since Android 11 it's going to be very difficult to do it.
I used to sync some of those folders with Syncthing in Android, and since Android 11 things have gotten complicated (meaning, you need 'all files' access, and even then, some things might not be accessible). [0]
Hello, we and TFA are talking about the new API where the user picks what directory an app has access to vs. the old API where the app has access to almost everything as you pointed out.
The Google Play store (not Android the OS) is now limiting access to the old API, it has nothing to do with Android 11.
The new API does not work with Linux open(). Some projects (like Syncthing) don't want to put in the work to switch to the new API.
Yeah, I'm not saying that you're in the wrong, I'm saying that they might want to access further than the new `ACTION_OPEN_DOCUMENT_TREE` API call allows. For example, the Downloads folder (you can only open individual files with ACTION_OPEN_DOCUMENT, not the whole folder). And that can be reached with the `MANAGE_EXTERNAL_STORAGE` (all files) API. Is that OK? Well, it's not intrinsically bad, but it clashes with Google's Android hardening directives.
Not very familiar with the NextCloud client, as I haven't used it much, but in the case of Syncthing (and this is totally guessing from what I remember from reading the forums back when the Android 11 upgrade happened): I don't think they don't want to put the work to switch to the new API, I think they want to access places they won't be able to access with the new API, so that's why they won't change the calls.
Yes, you'll need a little bridge to get a ParcelFileDescriptor. Lots of stuff in Android does not work directly with native code (Permissions, Camera/Media recording, HttpURLConnection, sensors/location services, ...), and storage access is now sandboxed in a similar way and needs a few lines of Java/Kotlin. I fully see that this is annoying, and Android is tiring developers with constant changes like these, but on the other hand, I guess most people would agree that securing storage access is a good thing.
It would not be much effort at all for google to make native hooks into these APIs. Certainly less effort than dealing with constant complaints from developers. It feels like they're trying to force lock-in.
With file access, the sensible solution would have been of course to just overlay the permissions onto the regular files API (like I think every other sandboxing solution does), so everything that uses files could keep working as usual and you'd just have to add some limited code to request the appropriate permissions [1], but I guess that would have been too sensible, and when all you've got is a SAF, everything looks like a nail or something…
From what I've gathered, getting access to a single user-picked file via SAF isn't too painful, but for browsing a whole directory tree it's more annoying having to use a completely new API for that, doubly so when you're using a library that expects regular files, plus performance is somewhat lacking, too (judging from complaints and bug reports I've seen, both in that it is easy to run into performance pitfalls, but also that even best-case performance is still worse than classic file access).
> From what I've gathered, getting access to a single user-picked file via SAF isn't too painful, but for browsing a whole directory tree it's more annoying having to use a completely new API for that
That pretty much matches my experience. Single file picker flows can often be implemented with a simple callback.
But for folders it falls apart.
Thanks for mentioning scoped storage. Maybe that provides some sort of solution. I vaguely remember when I was first dealing with this circa 2021 that they had some sort of a low-performance solution that was supposed to work for NDK code.
You can select any file you could before on shared external storage. You can't select private app directories which were always off limits from other apps just like on iOS.
From what I see (and what I've experienced on different sync apps when upgrading Android versions, although I'm not a developer, just a user), access to /Android/data and /Android/obb changed on Android 11:
---
[0] On Android 11 (API level 30) and higher, you cannot use the ACTION_OPEN_DOCUMENT_TREE intent action to request access to the following directories:
- The root directory of the internal storage volume.
- The root directory of each SD card volume that the device manufacturer considers to be reliable, regardless of whether the card is emulated or removable. A reliable volume is one that an app can successfully access most of the time.
- The Download directory.
Furthermore, on Android 11 (API level 30) and higher, you cannot use the ACTION_OPEN_DOCUMENT_TREE intent action to request that the user select individual files from the following directories:
- The Android/data/ directory and all subdirectories.
- The Android/obb/ directory and all subdirectories.
---
[1] On Android 11 (API level 30) and higher, you cannot use the ACTION_OPEN_DOCUMENT intent action to request that the user select individual files from the following directories:
- The Android/data/ directory and all subdirectories.
- The Android/obb/ directory and all subdirectories.
Yeah, yeah, I know. I'm not saying that it's good or bad, I just say that some people like to sync/backup/copy their apps data (that sometimes it's located on private directories, or in the Downloads folder...), and they were able to do it before. And that the new API has less permissions than the one that Google wants them to use.
Google Play, or at least that's what they say in their docs, might allow the use of the MANAGE_EXTERNAL_STORAGE "all files" API call "...if your app includes a use case similar to any of the following: File managers, Backup and restore apps, [...] Document management apps ..." [0][1]. And the Nextcloud app sounds very similar to that.
Of course, it's all "Subject to Google Play review and approval.", but why they used to grant that permission to Nextcloud with no problem and revoked it suddenly... I have no idea.
Note that `MANAGE_EXTERNAL_STORAGE` permission doesn't allow access to /Android anymore either (it's blocked by OS permisisons) so it won't help Nextcloud to retain it either.
Thing is, Nextcloud doesn't need that permission since they can do what they do now with SAF which is how Google determines eligibility for the exception. That is - if you can do it with SAF, you don't need complete access to all private data via MANAGE permission.
> For those scenarios, structured storage fits the bill. The app requests folder permission and the user, using system UIs, grants permissions on the folders they want to enable.
I see your point, but why isn't this the case for document management apps? Also, Nextcloud can be extended with plugins, some of them falling in the document management category.
> Further saying "let the user decide" works great in theory and with a considered, rational userbase.
Nextcloud is mostly used by people that like to self-host their services, so in this case we fall into the rational userbase category.
The overlap is precisely that part which matters: The directory access portions. Both need to ask the user to choose which directories to access, and access all the files in those directories.
What they do after opening the files is very different, and irrelevant. FWIW I use both Nextcloud and that gallery app.
>it makes sense to let the users give it access to all the files on the phone
It doesn't even pretend to be a backup app, and further the permission we're talking about is limited to external storage (though that is a nebulous term on many Android devices where internal storage is split-brained on being internal and partly external).
Further saying "let the user decide" works great in theory and with a considered, rational userbase. In reality it means that everyone just says sure to everything, and soon all of the user's data is exfiltrated and everyone is whining that Google/Apple/et al should have forseen this.