Skip to content

Releases: keras-team/keras

v3.13.0

18 Dec 00:06
@sachinprasadhs sachinprasadhs
v3.13.0
986ff97
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
v3.13.0 Latest
Latest

BREAKING changes

Starting with version 3.13.0, Keras now requires Python 3.11 or higher. Please ensure your environment is updated to Python 3.11+ to install the latest version.

Highlights

LiteRT Export

You can now export Keras models directly to the LiteRT format (formerly TensorFlow Lite) for on-device inference.
This changes comes with improvements to input signature handling and export utility documentation. The changes ensure that LiteRT export is only available when TensorFlow is installed, update the export API and documentation, and enhance input signature inference for various model types.

Example:

import keras
import numpy as np

# 1. Define a simple model
model = keras.Sequential([
    keras.layers.Input(shape=(10,)),
    keras.layers.Dense(10, activation="relu"),
    keras.layers.Dense(1, activation="sigmoid")
])

# 2. Compile and train (optional, but recommended before export)
model.compile(optimizer="adam", loss="binary_crossentropy")
model.fit(np.random.rand(100, 10), np.random.randint(0, 2, 100), epochs=1)

# 3. Export the model to LiteRT format
model.export("my_model.tflite", format="litert")

print("Model exported successfully to 'my_model.tflite' using LiteRT format.")

GPTQ Quantization

  • Introduced keras.quantizers.QuantizationConfig API that allows for customizable weight and activation quantizers, providing greater flexibility in defining quantization schemes.

  • Introduced a new filters argument to the Model.quantize method, allowing users to specify which layers should be quantized using regex strings, lists of regex strings, or a callable function. This provides fine-grained control over the quantization process.

  • Refactored the GPTQ quantization process to remove heuristic-based model structure detection. Instead, the model's quantization structure can now be explicitly provided via GPTQConfig or by overriding a new Model.get_quantization_layer_structure method, enhancing flexibility and robustness for diverse model architectures.

  • Core layers such as Dense, EinsumDense, Embedding, and ReversibleEmbedding have been updated to accept and utilize the new QuantizationConfig object, enabling fine-grained control over their quantization behavior.

  • Added a new method get_quantization_layer_structure to the Model class, intended for model authors to define the topology required for structure-aware quantization modes like GPTQ.

  • Introduced a new utility function should_quantize_layer to centralize the logic for determining if a layer should be quantized based on the provided filters.

  • Enabled the serialization and deserialization of QuantizationConfig objects within Keras layers, allowing quantized models to be saved and loaded correctly.

  • Modified the AbsMaxQuantizer to allow specifying the quantization axis dynamically during the __call__ method, rather than strictly defining it at initialization.

Example:

  1. Default Quantization (Int8)
    Applies the default AbsMaxQuantizer to both weights and activations.
model.quantize("int8")
  1. Weight-Only Quantization (Int8)
    Disable activation quantization by setting the activation quantizer to None.
from keras.quantizers import Int8QuantizationConfig, AbsMaxQuantizer

config = Int8QuantizationConfig(
    weight_quantizer=AbsMaxQuantizer(axis=0),
    activation_quantizer=None 
)

model.quantize(config=config)
  1. Custom Quantization Parameters
    Customize the value range or other parameters for specific quantizers.
config = Int8QuantizationConfig(
    # Restrict range for symmetric quantization
    weight_quantizer=AbsMaxQuantizer(axis=0, value_range=(-127, 127)),
    activation_quantizer=AbsMaxQuantizer(axis=-1, value_range=(-127, 127))
)

model.quantize(config=config)

Adaptive Pooling layers

Added adaptive pooling operations keras.ops.nn.adaptive_average_pool and keras.ops.nn.adaptive_max_pool for 1D, 2D, and 3D inputs. These operations transform inputs of varying spatial dimensions into a fixed target shape defined by output_size by dynamically inferring the required kernel size and stride. Added corresponding layers:

  • keras.layers.AdaptiveAveragePooling1D
  • keras.layers.AdaptiveAveragePooling2D
  • keras.layers.AdaptiveAveragePooling3D
  • keras.layers.AdaptiveMaxPooling1D
  • keras.layers.AdaptiveMaxPooling2D
  • keras.layers.AdaptiveMaxPooling3D

New features

  • Add keras.ops.numpy.array_splitop a fundamental building block for tensor parallelism.
  • Add keras.ops.numpy.empty_like op.
  • Add keras.ops.numpy.ldexp op.
  • Add keras.ops.numpy.vander op which constructs a Vandermonde matrix from a 1-D input tensor.
  • Add keras.distribution.get_device_count utility function for distribution API.
  • keras.layers.JaxLayer and keras.layers.FlaxLayer now support the TensorFlow backend in addition to the JAX backed. This allows you to embed flax.linen.Module instances or JAX functions in your model. The TensorFlow support is based on jax2tf.

OpenVINO Backend Support:

  • Added numpy.digitize support.
  • Added numpy.diag support.
  • Added numpy.isin support.
  • Added numpy.vdot support.
  • Added numpy.floor_divide support.
  • Added numpy.roll support.
  • Added numpy.multi_hot support.
  • Added numpy.psnr support.
  • Added numpy.empty_like support.

Bug fixes and Improvements

  • NNX Support: Improved compatibility and fixed tests for the NNX library (JAX), ensuring better stability for NNX-based Keras models.
  • MultiHeadAttention: Fixed negative index handling in attention_axes for MultiHeadAttention layers.
  • Softmax: The update on Softmax mask handling, aimed at improving numerical robustness, was based on a deep investigation led by Jaswanth Sreeram, who prototyped the solution with contributions from others.
  • PyDataset Support: The Normalization layer's adapt method now supports PyDataset objects, allowing for proper adaptation when using this data type.

TPU Test setup

Configured the TPU testing infrastructure to enforce unit test coverage across the entire codebase. This ensures that both existing logic and all future contributions are validated for functionality and correctness within the TPU environment.

New Contributors

Full Changelog: v3.12.0...v3.13.0

Contributors

mattjj, erahulkulkarni, and 13 other contributors
Assets 2
Loading

Keras 3.12.0

27 Oct 20:22
@fchollet fchollet
v3.12.0
adbfd13
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.12.0

Highlights

Keras has a new model distillation API!

You now have access to an easy-to-use API for distilling large models into small models while minimizing performance drop on a reference dataset -- compatible with all existing Keras models. You can specify a range of different distillation losses, or create your own losses. The API supports multiple concurrent distillation losses at the same time.

Example:

# Load a model to distill
teacher = ...
# This is the model we want to distill it into
student = ...

# Configure the process
distiller = Distiller(
    teacher=teacher,
    student=student,
    distillation_losses=LogitsDistillation(temperature=3.0),
)
distiller.compile(
    optimizer='adam',
    loss='sparse_categorical_crossentropy',
    metrics=['accuracy']
)

# Train the distilled model
distiller.fit(x_train, y_train, epochs=10)

Keras supports GPTQ quantization!

GPTQ is now built into the Keras API. GPTQ is a post-training, weights-only quantization method that compresses a model to int4 layer by layer. For each layer, it uses a second-order method to update weights while minimizing the error on a calibration dataset.

Learn how to use it in this guide.

Example:

model = keras_hub.models.Gemma3CausalLM.from_preset("gemma3_1b")
gptq_config = keras.quantizers.GPTQConfig(
    dataset=calibration_dataset,
    tokenizer=model.preprocessor.tokenizer,
    weight_bits=4,
    group_size=128,
    num_samples=256,
    sequence_length=256,
    hessian_damping=0.01,
    symmetric=False,
    activation_order=False,
)
model.quantize("gptq", config=gptq_config)
outputs = model.generate(prompt, max_length=30)

Better support for Grain datasets!

  • Add Grain support to keras.utils.image_dataset_from_directory and keras.utils.text_dataset_from_directory. Specify format="grain" to return a Grain dataset instead of a TF dataset.
  • Make almost all Keras preprocessing layers compatible with Grain datasets.

New features

  • Add keras.layers.ReversibleEmbedding layer: an embedding layer that can also also project backwards to the input space. Use it with the reverse argument in call().
  • Add argument opset_version in model.export(). Argument specific to format="onnx"; specifies the ONNX opset version.
  • Add keras.ops.isin op.
  • Add keras.ops.isneginf, keras.ops.isposinf ops.
  • Add keras.ops.isreal op.
  • Add keras.ops.cholesky_inverse op and add upper argument in keras.ops.cholesky.
  • Add keras.ops.image.scale_and_translate op.
  • Add keras.ops.hypot op.
  • Add keras.ops.gcd op.
  • Add keras.ops.kron op.
  • Add keras.ops.logaddexp2 op.
  • Add keras.ops.view op.
  • Add keras.ops.unfold op.
  • Add keras.ops.jvp op.
  • Add keras.ops.trapezoid op.
  • Add support for over 20 news ops with the OpenVINO backend.

Breaking changes

  • Layers StringLookup & IntegerLookup now save vocabulary loaded from file. Previously, when instantiating these layers from a vocabulary filepath, only the filepath would be saved when saving the layer. Now, the entire vocabulary is materialized and saved as part of the .keras archive.

Security fixes

New Contributors

Full Changelog: v3.11.0...v3.12.0

Contributors

WIgor, SamKnightGit, and 14 other contributors
Loading

Keras 3.11.3

22 Aug 17:48
@rtg0795 rtg0795
v3.11.3
b491c86
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.11.3

What's Changed

Full Changelog: v3.11.2...v3.11.3

Contributors

rtg0795
Loading

Keras 3.11.2

11 Aug 21:12
@laxmareddyp laxmareddyp
v3.11.2
251ac34
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.11.2

What's Changed

New Contributors

Full Changelog: v3.11.1...v3.11.2

Contributors

laxmareddyp
Loading

Keras 3.11.1

31 Jul 22:02
@rtg0795 rtg0795
v3.11.1
0e11071
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.11.1

What's Changed

Full Changelog: v3.11.0...v3.11.1

Contributors

rtg0795
Loading

Keras 3.11.0

29 Jul 23:59
@fchollet fchollet
v3.11.0
7bf852c
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.11.0

What's Changed

  • Add int4 quantization support.
  • Support Grain data loaders in fit()/evaluate()/predict().
  • Add keras.ops.kaiser function.
  • Add keras.ops.hanning function.
  • Add keras.ops.cbrt function.
  • Add keras.ops.deg2rad function.
  • Add keras.ops.layer_normalization function to leverage backend-specific performance optimizations.
  • Various bug fixes and performance optimizations.

Backend-specific changes

JAX backend

  • Support NNX library. It is now possible to use Keras layers and models as NNX modules.
  • Support shape -1 for slice op.

TensorFlow backend

  • Add support for multiple dynamic dimensions in Flatten layer.

OpenVINO backend

  • Add support for over 30 new backend ops.

New Contributors

Full Changelog: v3.10.0...v3.11.0

Contributors

iazzi, timovdk, and 6 other contributors
Loading

Keras 3.10.0

19 May 22:57
@fchollet fchollet
v3.10.0
3bedb9a
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.10.0

New features

  • Add support for weight sharding for saving very large models with model.save(). It is controlled via the max_shard_size argument. Specifying this argument will split your Keras model weight file into chunks of this size at most. Use load_model() to reload the sharded files.
  • Add optimizer keras.optimizers.Muon
  • Add image preprocessing layer keras.layers.RandomElasticTransform
  • Add loss function keras.losses.CategoricalGeneralizedCrossEntropy (with functional version keras.losses.categorical_generalized_cross_entropy)
  • Add axis argument to SparseCategoricalCrossentropy
  • Add lora_alpha to all LoRA-enabled layers. If set, this parameter scales the low-rank adaptation delta during the forward pass.
  • Add activation function keras.activations.sparse_sigmoid
  • Add op keras.ops.image.elastic_transform
  • Add op keras.ops.angle
  • Add op keras.ops.bartlett
  • Add op keras.ops.blackman
  • Add op keras.ops.hamming
  • Add ops keras.ops.view_as_complex, keras.ops.view_as_real

PyTorch backend

  • Add cuDNN support for LSTM with the PyTorch backend

TensorFlow backend

  • Add tf.RaggedTensor support to Embedding layer
  • Add variable-level support for synchronization argument

OpenVINO backend

  • Add support for over 50 additional Keras ops in the OpenVINO inference backend!

New Contributors

Full Changelog: v3.9.0...v3.10.0

Contributors

superbobry, pschuh, and 29 other contributors
Loading

Keras 3.9.2

02 Apr 20:22
@rtg0795 rtg0795
v3.9.2
9067d80

Choose a tag to compare

View all tags
Keras 3.9.2

What's Changed

  • Fix Remat error when called with a model.

Full Changelog: v3.9.1...v3.9.2

Loading

Keras 3.9.1

27 Mar 17:17
@rtg0795 rtg0795
v3.9.1
54bdd19

Choose a tag to compare

View all tags
Keras 3.9.1

What's Changed

  • Fix flash attention TPU error
  • Fix incorrect argument in JAX flash attention.

Full Changelog: v3.9.0...v3.9.1

Loading

Keras 3.9.0

04 Mar 23:24
@fchollet fchollet
v3.9.0
eb1f844
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Choose a tag to compare

View all tags
Keras 3.9.0

New features

  • Add new Keras rematerialization API: keras.RematScope and keras.remat. It can be used to turn on rematerizaliation for certain layers in fine-grained manner, e.g. only for layers larger than a certain size, or for a specific set of layers, or only for activations.
  • Increase op coverage for OpenVINO backend.
  • New operations:
    • keras.ops.rot90
    • keras.ops.rearrange (Einops-style)
    • keras.ops.signbit
    • keras.ops.polar
    • keras.ops.image.perspective_transform
    • keras.ops.image.gaussian_blur
  • New layers:
    • keras.layers.RMSNormalization
    • keras.layers.AugMix
    • keras.layers.CutMix
    • keras.layers.RandomInvert
    • keras.layers.RandomErasing
    • keras.layers.RandomGaussianBlur
    • keras.layers.RandomPerspective
  • Minor additions:
    • Add support for dtype argument to JaxLayer and FlaxLayer layers
    • Add boolean input support to BinaryAccuracy metric
    • Add antialias argument to keras.layers.Resizing layer.
  • Security fix: disallow object pickling in saved npz model files (numpy format). Thanks to Peng Zhou for reporting the vulnerability.

New Contributors

Full Changelog: v3.8.0...v3.9.0

Contributors

jurca, ibraaaa, and 11 other contributors
Loading