I would _kill_ for a shell that returns bash-like data to stdout, but a JSON blob to pipes, or whatever the not-1970 concept of a pipe would be.

The shell isn’t in charge of what data commands print out.

> The shell isn’t in charge of what data commands print out.

It could be if it captured stdout of processes it launches.

Of course, the program would have to output a structured format that the shell recognized to provide the shell the option to do what is described upthread, so legacy programs wouldn't fit in well with it unless the shell was programmed with how to extract structured data from their unstructured output.

I think I'd rather keep STDOUT et al as byte streams but include an additional channel with details about how to read that data.

eg it could describe the data as a JSON array and the next command would grep through the elements rather than lines.

True, but it's driven the method for output for a long time. If sh/bash exposed functions for objective output I'm sure a lot more utilities commonly in use would output in a similar methodology.

No it hasn’t. The shell isn’t involved at all in program output. When a program writes to stdout it’s talking directly to the kernel, and the shell is suspended and not running.

I don't really care what outputs the data I see. I care about how it's formatted and whether that format is easily readable + usable by consuming utilities.

I don't understand why your being standoffish over a technicality.

Okay, that’s fine. I think it would be nice for an OS to have a set of utilities that communicate with each other with objects instead of text. So I agree with you.

I’m just pointing out that this has literally zero whatsoever to do with the shell. You posted it as a comment in an article involving a choice of shells.

How am I being standoffish?

That's true for POSIX shells but if you're breaking POSIX comparability then you can start doing all kinds of clever/stupid hacks to turn pipes into something more than dumb byte streams.

Coincidentally, this is what I'm working on with my shell.

I still don't understand what this has to do with the shell. Pipes are a kernel concept, not a shell concept.

Shells are just user interfaces so they don't have to have pipes nor even be CLI based. For example on Windows, explorer.exe is technically a shell. On Linux you can run git as a shell. Obviously these don't solve the problems you guys are raising for optimum command line usage but it illustrates that interesting things can be done if you shift your mindset from POSIX. For example some REPL environments might make a higher level language it's first class citizen (eg Powershell).

The standard byte streams are just 3 file descriptors and on Linux are generally symlinks inside /proc but all that is handled by the pipe syscall (as you stated). However there is still a lot of code between the shell parsing your entered command line and pipe() being called and how the shell decides to implement fork() and pipe() is largely down to the shell designers. Going back to my earlier point about POSIX shells, there is a general expectation that these kinds of shells should just fork() and pipe(); which makes sense when you think about their heritage but it does allow developers to get creative if you're intending to break POSIX.

For example they could check the receiver processes for a signature (magic byte or something) to see if it's a supporting tool and if it is, the shell could then "man in the middle" the pipe; where it sets itself as the destination pipe for the first process and the source for the second process (eg `ls | grep foo` would be called as `ls | shell | grep`). That way the shell could then wrap STDOUT from `ls` up inside a more complex object and pass that to `grep`...and only do all this if `grep` already appears to support this new shell's object system.

Other ideas is that type information could be passed via a fourth file descriptor, UNIX socket or over a localhost TCP/IP listener. Or maybe you don't support smart processes and instead write your shell to do some of the heavy lifting instead via shell builtins (like how Bash has a few builtin functions) where you could write smarter data wrangling tools so it doesn't matter what crap `ls` or `grep` throw at you because you can pipeline that into a shell builtin.

This last approach of smarter builtins is the direction I took with my own hobby shell but I'm now looking for ways to pass that data type information along to other processes without breaking non-supporting processes.

I'm half asleep at the moment so probably haven't don't a good job explaining my thought process but there are a few of use playing around writing "alternative shells" (for want a better description).

PowerShell is the modern equivalent basically. You can also use it in Linux.

But I think it has a bit of a learning curve because not thinking in string outputs is quite different.