## Issue

I want to train a recurrent neural network using `Tensorflow`

. My model outputs a 1 by 100 vector for each training sample. Assume that `y = [y_1, y_2, ..., y_100]`

is my output for training sample `x`

and the expected output is `y'= [y'_1, y'_2, ..., y'_100]`

.

I wish to write a custom loss function that calculates the loss of this specific sample as follows:

```
Loss = 1/sum(weights) * sqrt(w_1*(y_1-y'_1)^2 + ... + w_100*(y_100-y'_100)^2)
```

which `weights = [w_1,...,w_100]`

is a given weight array.

Could someone help me with implementing such a custom loss function? (I also use mini-batches while training)

## Solution

I want to underline that you have 2 possibilities according to your problem:

**[1] If the weights are equal for all your samples:**

You can build a loss wrapper. Here a dummy example:

```
n_sample = 200
X = np.random.uniform(0,1, (n_sample,10))
y = np.random.uniform(0,1, (n_sample,100))
W = np.random.uniform(0,1, (100,)).astype('float32')
def custom_loss_wrapper(weights):
def loss(true, pred):
sum_weights = tf.reduce_sum(weights) * tf.cast(tf.shape(pred)[0], tf.float32)
resid = tf.sqrt(tf.reduce_sum(weights * tf.square(true - pred)))
return resid/sum_weights
return loss
inp = Input((10,))
x = Dense(256)(inp)
pred = Dense(100)(x)
model = Model(inp, pred)
model.compile('adam', loss=custom_loss_wrapper(W))
model.fit(X, y, epochs=3)
```

**[2] If the weights are different between samples:**

You should build your model usind `add_loss`

in order to dinamically take into account the weights for each sample. Here a dummy example:

```
n_sample = 200
X = np.random.uniform(0,1, (n_sample,10))
y = np.random.uniform(0,1, (n_sample,100))
W = np.random.uniform(0,1, (n_sample,100))
def custom_loss(true, pred, weights):
sum_weights = tf.reduce_sum(weights)
resid = tf.sqrt(tf.reduce_sum(weights * tf.square(true - pred)))
return resid/sum_weights
inp = Input((10,))
true = Input((100,))
weights = Input((100,))
x = Dense(256)(inp)
pred = Dense(100)(x)
model = Model([inp,true,weights], pred)
model.add_loss(custom_loss(true, pred, weights))
model.compile('adam', loss=None)
model.fit([X,y,W], y=None, epochs=3)
```

When using `add_loss`

you should pass all the tensors involved in the loss as input layer and pass them inside the loss for the computation.

At inference time you can compute predictions as always, simply removing the true and weights as input:

```
final_model = Model(model.input[0], model.output)
final_model.predict(X)
```

Answered By – Marco Cerliani

Answer Checked By – Mildred Charles (Easybugfix Admin)