Moirai-1.0-R-Base

Moirai, the Masked Ecoder-based Uiversal Time Series Forecastig Trasformer is a Large Time Series Model pre-traied o LOTSA data. For more details o the Moirai architecture, traiig, ad results, please refer to the paper.

Fig. 1: Overall architecture of Moirai. Visualized is a 3-variate time series, where variates 0 ad 1 are target variables (i.e. to be forecasted, ad variate 2 is a dyamic covariate (values i forecast horizo kow). Based o a patch size of 64, each variate is patchified ito 3 tokes. The patch embeddigs alog with sequece ad variate id are fed ito the Trasformer. The shaded patches represet the forecast horizo to be forecasted, whose correspodig output represetatios are mapped ito the mixture distributio parameters.

Usage

To perform iferece with Moirai, istall the ui2ts library from our GitHub repo.

1. Cloe repository:

git cloe https://github.com/SalesforceAIResearch/ui2ts.git
cd ui2ts

2) Create virtual eviromet:

virtualev vev
. vev/bi/activate

3) Build from source:

pip istall -e '.[otebook]'

4) Create a .ev file:

touch .ev

A simple example to get started:

import torch
import matplotlib.pyplot as plt
import padas as pd
from gluots.dataset.padas import PadasDataset
from gluots.dataset.split import split

from ui2ts.eval_util.plot import plot_sigle
from ui2ts.model.moirai import MoiraiForecast, MoiraiModule


SIZE = "small"  # model size: choose from {'small', 'base', 'large'}
PDT = 20  # predictio legth: ay positive iteger
CTX = 200  # cotext legth: ay positive iteger
PSZ = "auto"  # patch size: choose from {"auto", 8, 16, 32, 64, 128}
BSZ = 32  # batch size: ay positive iteger
TEST = 100  # test set legth: ay positive iteger

# Read data ito padas DataFrame
url = (
    "https://gist.githubusercotet.com/rsirwa/c8c8654a98350fadd229b00167174ec4"
    "/raw/a42101c7786d4bc7695228a0f2c8cea41340e18f/ts_wide.csv"
)
df = pd.read_csv(url, idex_col=0, parse_dates=True)

# Covert ito GluoTS dataset
ds = PadasDataset(dict(df))

# Split ito trai/test set
trai, test_template = split(
    ds, offset=-TEST
)  # assig last TEST time steps as test set

# Costruct rollig widow evaluatio
test_data = test_template.geerate_istaces(
    predictio_legth=PDT,  # umber of time steps for each predictio
    widows=TEST // PDT,  # umber of widows i rollig widow evaluatio
    distace=PDT,  # umber of time steps betwee each widow - distace=PDT for o-overlappig widows
)

# Prepare pre-traied model by dowloadig model weights from huggigface hub
model = MoiraiForecast(
    module=MoiraiModule.from_pretraied(f"Salesforce/moirai-1.0-R-{SIZE}"),
    predictio_legth=PDT,
    cotext_legth=CTX,
    patch_size=PSZ,
    um_samples=100,
    target_dim=1,
    feat_dyamic_real_dim=ds.um_feat_dyamic_real,
    past_feat_dyamic_real_dim=ds.um_past_feat_dyamic_real,
)

predictor = model.create_predictor(batch_size=BSZ)
forecasts = predictor.predict(test_data.iput)

iput_it = iter(test_data.iput)
label_it = iter(test_data.label)
forecast_it = iter(forecasts)

ip = ext(iput_it)
label = ext(label_it)
forecast = ext(forecast_it)

plot_sigle(
    ip, 
    label, 
    forecast, 
    cotext_legth=200,
    ame="pred",
    show_label=True,
)
plt.show()

The Moirai Family

Citatio

If you're usig Ui2TS i your research or applicatios, please cite it usig this BibTeX:

@article{woo2024uified,
  title={Uified Traiig of Uiversal Time Series Forecastig Trasformers},
  author={Woo, Gerald ad Liu, Cheghao ad Kumar, Akshat ad Xiog, Caimig ad Savarese, Silvio ad Sahoo, Doye},
  joural={arXiv preprit arXiv:2402.02592},
  year={2024}
}

Moirai-1.0-R-Base Moirai, the Masked Encoder-based Universal Time Series Forecasting Transformer is