Using context length of 60, trying to predict next 7 days Close price. Error is : Lags cannot go further than history length

[Mohan16071996]

System Info

Error info:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-39-7551ed0c5de0>](https://localhost:8080/#) in <cell line: 12>()
     20 
     21         optimizer.zero_grad()
---> 22         outputs = model(
     23             past_values=past_time_features[:, :, 0],
     24             past_time_features=past_time_features[:, :, 1:],

7 frames
[/usr/local/lib/python3.10/dist-packages/transformers/models/autoformer/modeling_autoformer.py](https://localhost:8080/#) in get_lagged_subsequences(self, sequence, subsequences_length, shift)
   1481         sequence_length = sequence.shape[1]
   1482         if max(indices) + subsequences_length > sequence_length:
-> 1483             raise ValueError(
   1484                 f"lags cannot go further than history length, found lag {max(indices)} "
   1485                 f"while history length is only {sequence_length}"

ValueError: lags cannot go further than history length, found lag 7 while history length is only 60

Who can help?

@kashif

from torch.utils.data import Dataset, DataLoader
from sklearn.preprocessing import StandardScaler
import numpy as np


from torch.utils.data import Dataset, DataLoader

class TimeSeriesDataset(Dataset):
    def __init__(self, data, input_size, pred_size, target_col):
        self.data = data
        self.input_size = input_size
        self.pred_size = pred_size
        self.target_col = target_col
        self.scaler = StandardScaler()
        self.data_scaled = self.data.copy()
        self.data_scaled.iloc[:, :] = self.scaler.fit_transform(self.data)

    def __len__(self):
        return len(self.data) - self.input_size - self.pred_size + 1

    def __getitem__(self, idx):
        end = idx + self.input_size
        pred_end = end + self.pred_size
        x = self.data_scaled.iloc[idx:end, :]
        y = self.data_scaled.iloc[end:pred_end][self.target_col]
        past_time_features = x.values
        future_time_features = self.data_scaled.iloc[end:pred_end, :].values
        past_observed_mask = ~np.isnan(past_time_features[:, 0])
        return past_time_features, future_time_features, past_observed_mask, y.values

input_size = 60
pred_size = 7
target_col = 'Close'

dataset = TimeSeriesDataset(combined_data, input_size, pred_size, target_col)
data_loader = DataLoader(dataset, batch_size=32, shuffle=False)

from transformers import AutoformerConfig, AutoformerForPrediction

lags = [1, 2, 3, 4, 5, 6, 7]


config = AutoformerConfig(
    context_length=input_size,
    prediction_length=pred_size,
    lags_sequence=lags,
    encoder_layers=3,
    decoder_layers=3,
    d_model=64,
    n_heads=4,
    e_layers=2,
    d_layers=1,
    dropout=0.1,
    activation="gelu",
    learning_rate=1e-4,
    normalization="layer"
)
model = AutoformerForPrediction(config)

import torch
import torch.optim as optim

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

criterion = torch.nn.MSELoss()
optimizer = optim.Adam(model.parameters(), lr=config.learning_rate)

num_epochs = 10

for epoch in range(num_epochs):
    model.train()
    for batch in data_loader:
        past_time_features, future_time_features, past_observed_mask, y_batch = batch
        past_time_features = past_time_features.to(device).float()
        future_time_features = future_time_features.to(device).float()
        past_observed_mask = past_observed_mask.to(device).bool()
        y_batch = y_batch.to(device).float()

        optimizer.zero_grad()
        **outputs = model(**    --- Error in this line
            past_values=past_time_features[:, :, 0], 
            past_time_features=past_time_features[:, :, 1:], 
            future_time_features=future_time_features[:, :, 1:], 
            past_observed_mask=past_observed_mask
        ).predictions

        loss = criterion(outputs, y_batch)
        loss.backward()
        optimizer.step()

    print(f"Epoch {epoch+1}/{num_epochs}, Loss: {loss.item()}")

Expected behavior

Model to be trained for all epochs given.

[Mohan16071996]

Can someone please respond to this as soon as possible?

[RUFFY-369]

Hi @Mohan16071996, you are getting the described error as the condition if max(indices) + subsequences_length > sequence_length: is amounting to this:7+60>60 as you have set the context length size to the context_length=input_size in config and input_size = 60,and the length of lags is already amounting to 7. As lags are extra context provided so obviously it can't take more context than what already exist or basically you would be predicting future values using information that doesn't exist yet as Autoformer uses a self supervised learning approach,

So, please adapt accordingly then the error would be gone. Just use context length as default or avoid using lags if you are using context_length= input_size.

Cheers!

[kashif]

right @Mohan16071996 so yeah increase the amount of time series by the max of the lag indices as that is how many time steps back the helpers will look to pull in the lagged values...

[Mohan16071996]

Hi @kashif and @RUFFY-369
So, if I understand correctly, context length cannot increase lags in autoformer and also in Informer I believe?
So, to forecast for next 7 days, I should use context length as 7 or lesser values?
Can you please elaborate?

[RUFFY-369]

@Mohan16071996 No, basically context length is the length of input sequence or number of timesteps from the past data which you want to take in the context for making a prediction and as Autoformer is a self supervised model so it cant take a context length beyond its data which is the sequence length. So, give whatever value you want to experiment with but the sum of context length and lags should be less than or equal to length of the sequence or input size.
And prediction length is the length of horizon which you want to predict(in your case it's 7).

If you want further more details then I think there is a blog about Autoformer

[Mohan16071996]

I have used input size as 60, context length to be 50 and prediction length to be 7,
I get different error now:
input length 50 and time feature lengths 53 does not match

Can you please suggest how to approach this issue?

[Mohan16071996]

I am still confused between those two errors, I guess I am making a mistake in shapes of past and future features and in lags, input and context length configuration. Is there a blog or site that clearly mentions about these configuration?
I am stuck on this issue for the past two days.

[RUFFY-369]

Hi @Mohan16071996 , yes, there is a fully detailed blog on Autoformer by Hugging face which previously mentioned: https://huggingface.co/blog/autoformer. I think going through that blog will give you more clarity on the model.

And regarding your error it is due to this block of code where lagged seq derived from context length and time features shape are compared before passing them as inputs.

So, yeah if you could go through that blog then it will help you a lot.

Cheers,
Prakarsh