Failed tests in the last release (v3.8.0) test suite

[gdrosa]

Summary:
While attempting to install Keras in editable mode and running the test suite, several tests failed. I now detail the steps I followed to reproduce the errors.

---

Steps to Reproduce:

1. Created a conda environment:

   conda create -n keras python=3.10
   conda activate keras

2. Cloned the Keras repository:

   git clone https://github.com/keras-team/keras.git
   cd keras

3. Moved to the latest release:

   git fetch --tags
   git checkout tags/v3.8.0

4. Installed the requirements:

   python3.10 -m pip install -r requirements.txt
   ./shell/api_gen.sh

5. Installed Keras in editable mode:

   python3.10 -m pip install -e .

6. Configured the backend:

   export KERAS_BACKEND="tensorflow"

7. Ran the test suite:

   python3.10 -u -m pytest --cov=keras --cov-report=html:htmlcov --cov-report=xml:coverage.xml > keras_golden_run

---

Observed Behavior:

• The test suite reported the following short summary:

  FAILED integration_tests/jax_custom_fit_test.py::test_custom_fit - TypeError: test_custom_fit.<locals>.CustomModel.train_step() missing 1 required positional argument: 'data'
  FAILED integration_tests/tf_distribute_training_test.py::test_model_fit - RuntimeError: Virtual devices cannot be modified after being initialized
  FAILED integration_tests/torch_custom_fit_test.py::test_custom_fit - AttributeError: 'CustomModel' object has no attribute 'zero_grad'
  FAILED integration_tests/torch_workflow_test.py::TorchWorkflowTest::test_keras_layer_in_nn_module - AssertionError: 0 != 2
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_notop_custom_input_shape_EfficientNetV2B0_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_notop_custom_input_shape_EfficientNetV2B1_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_notop_custom_input_shape_EfficientNetV2B2_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_notop_custom_input_shape_EfficientNetV2B3_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_pooling_EfficientNetV2B0_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_pooling_EfficientNetV2B1_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_pooling_EfficientNetV2B2_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  FAILED keras/src/applications/applications_test.py::ApplicationsTest::test_application_pooling_EfficientNetV2B3_channels_first - tensorflow.python.framework.errors_impl.InvalidArgumentError: {{function_node __wrapped__BroadcastTo_device_/job:localhost/replica:0/task:0/device:CPU:0}} Unable to broadcast tensor of shape [3] to tensor of shape [1,3,1,1] [Op:BroadcastTo]
  = 12 failed, 12870 passed, 346 skipped, 8 xfailed, 2 xpassed in 1283.05s (0:21:23) =
  

• Errors include:

  • TypeError in integration_tests/jax_custom_fit_test.py.
  • RuntimeError in integration_tests/tf_distribute_training_test.py.
  • InvalidArgumentError in several applications_test.py cases involving EfficientNetV2.

---

Environment Details:

• Python Version: 3.10
• Anaconda Version: 4.10.1
• Operating System: Ubuntu 22.04.5 LTS

---

Additional Notes:

1. I avoided using pip_build.py as I wanted to install Keras in editable mode.
2. The error logs seem to point towards compatibility or configuration issues:
   • The integration_tests errors suggest issues with the test setup for jax and torch.
   • The applications_test.py errors may involve TensorFlow device configurations or input shapes for EfficientNetV2.

---

Request for Help:
I expect the test suite to pass without any errors, or at least there should be no critical errors, so:

• Should these failures be ignored, or are they indicative of underlying issues?
• Should I reconfigure the backend to run these tests?
• Are there additional dependencies or configurations required to run the full test suite?

I guess that I should use both Jax and PyTorch as backends to run these tests, but maybe I can just ignore them if I am using TensorFlow. Or am I missing something? However, any assistance in resolving this issue would be greatly appreciated.

Thank you.