You can absolutely serve with Keras if your inference server is in Python. For instance, if you're looking for a basic solution, you can just set up a Flask app that calls `predict()` on a Keras model.
If you're looking for a high-performance solution that is entirely Python-free, then you can simply export your Keras model as a TF SavedModel and serve it via TFServing. TFServing is C++ based and works on both CPU and GPU.
Otherwise, if you want to reuse an existing LLM (or just see how a large one would be implemented in practice) you can check out the models from KerasNLP. For instance, this is BERT, basically just a stack of TransformerEncoders. https://github.com/keras-team/keras-nlp/blob/master/keras_nl...
So far the export story focuses on SavedModel and the services that consume that format, e.g. TFLite, TFjs and TFServing. You can just do `model.export(path)`, and you also have access to the `ExportArchive` class for fine-grained configuration.
We have not tried CoreML export yet.
PyTorch `.compile()` works with Keras models. It may not necessarily result in a speedup however.
According to PyPI downloads and user surveys (like the yearly StackOverflow survey) the two main frameworks are TensorFlow and PyTorch for Deep Learning, and Scikit-Learn for classical ML.
Every time some survey pops up, TF somehow looks still relevant while I know noone who uses it or even considers using it unless they are new to DL field. How much of TF here is v2.0?:)
This means that the API, the abstractions, the workflows are battle-tested.
The codebase itself went through 2 months of private beta and 5 months of public beta. It is already used in production by several companies. It's not as battle tested as older frameworks, but it's fairly reliable.
To clarify, I have never attacked PyTorch, on Twitter or otherwise.
What happened is that I was a target of online harassment campaign from 2017 to January 2021 (when it stopped abruptly), which originated from a PyTorch developer and took the form of frequent anonymous emails (or messages on social networks) with insults and threats. The thread you link to below is me complaining about the harassment campaign and calling for civility.
Some frameworks fans can be extremely toxic, e.g. creating anonymous accounts on HN for the purpose of bashing one framework or one person.
We don't have a separate `ops.linalg` package, but we do include `numpy.linalg` ops as part of `keras.ops`. For now only 2 ops are supported: `qr` and `solve`. We're open to adding any `numpy.linalg` op that turns out to be useful (or you could open a PR for any op you need).
Hey Francois, congrats to you and the team on the launch! I've generally chosen Pytorch over Tensorflow for my day to day, but now that Keras is framework agnostic I'm excited to revisit it.
One thing I'm wondering about is if it's possible (or necessary?) to use Keras in concert with Pytorch Lightning. In some ways, Lightning evolved to be "Keras for Pytorch," so what is the path forward in a world where both exist as options for Pytorch users—do they interoperate or are they competitors/alternatives to each other?
Both Keras models/layers (with the PyTorch backend) and Lightning Modules are PyTorch Modules, so they should be able to interoperate with each other in a PyTorch workflow. We have not tried this with Lightning, but we've had a good experience with custom PyTorch Modules.
More broadly, it's feasible to use Keras components with any framework built on PyTorch or JAX in the sense that it's always possible to write "adapter layers" that wrap a Keras layer and make it usable by another framework, or the other way around. We have folks doing this to use Flax components (from JAX) as Keras layers, and inversely, to use Keras layers as Flax Modules.
We made sure that TFLite workflows would run smoothly with Keras 3 models. We did not come up with any TFLite related improvements. The focus was on the multi-backend architecture, distribution, and training performance.
Super cool, does that mean if someone trains something using the PyTorch backend, I can still use it with the coral if I load the weights using the tensorflow backend?
That's right, if the model is backend-agnostic you can train it with a PyTorch training loop and then reload it and use it with TF ecosystem tools, like serve it with TF-Serving or export it to Coral TPU.
Hi, first off thank you for your contributions, and this goes to the entire team. Keras is a wonderful tool and this was definitely the right move to do. No other package nails the “progressive disclosure” philosophy like Keras.
This caught my eye:
> “Right now, we use tf.nest (a Python data structure processing utility) extensively across the codebase, which requires the TensorFlow package. In the near future, we intend to turn tf.nest into a standalone package, so that you could use Keras Core without installing TensorFlow.”
I recently migrated a TF project to PyTorch (would have been great to have keras_core at the time) and used torch.nested. Could this not be an option?
A second question. For “customizing what happens in fit()”. Must this be written in either TF/PyTorch/Jax only, or can this be done with keras_core.ops, similar to the example shown for custom components? The idea would be you can reuse the same training loop logic across frameworks, like for custom components.
At this time, there are no backend-agnostic APIs to implement training steps/training loops, because each backend handles training very differently so no shared abstraction can exist (expecially for JAX). So when customizing fit() you have to use backend-native APIs.
If you want to make a model with a custom train_step that is cross-backend, you can do something like:
BTW it looks the previous account is being rate-limited to less than 1 post / hour (maybe even locked for the day) so I will be very slow to answer questions.
This looks awesome; I was a big fan of Keras back when it had pluggable backends and a much cleaner API than Tensorflow.
Fast forward to now, and my biggest pain point is that all the new models are released on PyTorch, but the PyTorch serving story is still far behind TF Serving. Can this help convert a PyTorch model into a servable SavedModel?
For a Keras Core model to be usable with the TF Serving ecosystem, it must be implemented either via Keras APIs (Keras layers and Keras ops) or via TF APIs.
To use pretrained models, you can take a look at KerasCV and KerasNLP, they have all the classics, like BERT, T5, OPT, Whisper, StableDiffusion, EfficientNet, YOLOv8, etc. They're adding new models regularly.
Congrats on the launch! I learned Keras back when I first got in to ML, so really happy to see it making a comeback. Are there some example architectures available/planned that are somewhat complex, and not just a couple layers (BERT, ResNet, etc.)?
Yes, you can check out KerasCV and KerasNLP which host pretrained models like ResNet, BERT, and many more. They run on all backends as of the latest releases (today), and converting them to be backend-agnostic was pretty smooth! It took a couple of weeks to convert the whole packages.
Firstly thanks to all the team for everything you have done and congrats on this. It must have been a ton of work and I am excited to get my hands on it.
If you're looking for a high-performance solution that is entirely Python-free, then you can simply export your Keras model as a TF SavedModel and serve it via TFServing. TFServing is C++ based and works on both CPU and GPU.