Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Does this support out-of-tree format decoders? From an initial glance it looks like all decoders are in-tree and written in golang. We have a lot of internal binary formats at $WORK that I would like to use this on...


Hi! yes it's kind of support but in a very go:ish at the moment. You can use fq as submodule, import/register your own format decoders and then run cli.Main. More or less what https://github.com/wader/fq/blob/master/fq.go does. I have private version of fq for work with some proprietary formats that does this and it works great. One issue is that the decoder and format API might change, not sure i can give any stability guarantees atm and i want to evolve a bit more. Also it would be great to be able to hook into existing formats more in some way.

In the future i hope to support writing decoders in jq and or support some declarative format like kaitai.


Seems unlikely since the decoders are defined not just in-tree but in "host" code, the definitions are neither data-driven nor a DSL.

So it would require some sort of native (Go) plugins system, which I understand is about as bad as in Rust owing to there being no standard ABI (or plugins system for that matter).

therefore the way to have bespoke / internal formats would be to maintain an internal fork of the tool.


"(...) some sort of native (Go) plugins system (...)"

See: https://pkg.go.dev/plugin




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: