When you decorate a function with `tf.function` decorator, sometimes you need to keep state between invocations of this function. The problem here is that the changes to the state will not be visible in the decorated function if the state is saved in the Python variables. To illustrate the problem, Tensorflow 2.2 is used: import tensorflow as tf print(tf.__version__) 2.2.0 To see the problem, let’s consider the following code. Let’s assume that we need to scale a given Tensor `x` and we do it using `tf.

Tensorflow 2 uses so called Eager mode by default. In this mode, it is easy to define tensors interactively, for example, in ipython and see their values. However, in Eager mode the execution is slow, which becomes noticable during model training. Tensorflow 2 offers another mode of execution called Graph mode. In this mode, first the computational graph is created and then used to compute loss function and its gradient. This mode is more performance efficient.

This is an introduction to using Hyperopt library. I will use mostly terminology of machine learning (ML) as this library appeared in the ML community. Hyperopt library is used to choose the hyperparameters, that is, parameters that must be set before the learning process. The learning process is the process of fitting a given model to some dataset, which is done by minimization of some function. For example, when you fit model \(\hat f (x)\) by optimizing function \[ \frac{1}{N} \sum_{i=1}^N \left( y_i - \hat f (x_i) \right)^2 + \lambda R(f), \]