Trito

This is the developmet repository of Trito, a laguage ad compiler for writig highly efficiet custom Deep-Learig primitives. The aim of Trito is to provide a ope-source eviromet to write fast code at higher productivity tha CUDA, but also with higher flexibility tha other existig DSLs.

The foudatios of this project are described i the followig MAPL2019 publicatio: Trito: A Itermediate Laguage ad Compiler for Tiled Neural Network Computatios. Please cosider citig this work if you use Trito!

The official documetatio cotais istallatio istructios ad tutorials.

Quick Istallatio

You ca istall the latest stable release of Trito from pip:

pip istall trito

Biary wheels are available for CPytho 3.7-3.11 ad PyPy 3.8-3.9.

Ad the latest ightly release:

pip istall -U --idex-url https://aiifra.pkgs.visualstudio.com/PublicPackages/_packagig/Trito-Nightly/pypi/simple/ trito-ightly

Istall from source

git cloe https://github.com/opeai/trito.git;
cd trito;

pip istall ija cmake wheel; # build-time depedecies
pip istall -e pytho

Or with a virtualev:

git cloe https://github.com/opeai/trito.git;
cd trito;

pytho -m vev .vev --prompt trito;
source .vev/bi/activate;

pip istall ija cmake wheel; # build-time depedecies
pip istall -e pytho

Buildig with a custom LLVM

Trito uses LLVM to geerate code for GPUs ad CPUs. Normally, the Trito build dowloads a prebuilt LLVM, but you ca also build LLVM from source ad use that.

LLVM does ot have a stable API, so the Trito build will ot work at a arbitrary LLVM versio.

1. Fid the versio of LLVM that Trito builds agaist. Check cmake/llvm-hash.txt to see the curret versio. For example, if it says: 49af6502c6dcb4a7f7520178bd14df396f78240c

   This meas that the versio of Trito you have builds agaist LLVM 49af6502.

2. git checkout LLVM at this revisio. Optioally, make additioal modificatios to LLVM.

3. Build LLVM. For example, you might ru

   $ cd $HOME/llvm-project # your cloe of LLVM. $ mkdir build $ cd build $ cmake -G Nija -DCMAKEBUILDTYPE=Release -DLLVMENABLEASSERTIONS=ON ../llvm -DLLVMENABLEPROJECTS="mlir;llvm" $ ija

4. Grab a sack, this will take a while.

5. Build Trito as above, but set the followig eviromet variables.

   # Modify as appropriate to poit to your LLVM build. $ export LLVMBUILDDIR=$HOME/llvm-project/build

   $ cd $ LLVMINCLUDEDIRS=$LLVMBUILDDIR/iclude \ LLVMLIBRARYDIR=$LLVMBUILDDIR/lib \ LLVMSYSPATH=$LLVMBUILD_DIR \ pip istall -e pytho

Tips for buildig

• Set TRITON_BUILD_WITH_CLANG_LLD=true as a eviromet variable to use clag ad lld. lld i particular results i faster builds.

• Set TRITON_BUILD_WITH_CCACHE=true to build with ccache.

• Pass --o-build-isolatio to pip istall to make op builds faster. Without this, every ivocatio of pip istall uses a differet symlik to cmake, ad this forces ija to rebuild most of the .a files.

• vscode itellisese has some difficulty figurig out how to build Trito's C++ (probably because, i our build, users do't ivoke cmake directly, but istead use setup.py). Teach vscode how to compile Trito as follows.

  • Do a local build.
  • Get the full path to the compile_commads.jso file produced by the build: fid pytho/build -ame 'compile_commads.jso | xargs readlik -f'
  • I vscode, istall the C/C++ extesio, the ope the commad palette (Shift + Commad + P o Mac, or Shift + Ctrl + P o Widows/Liux) ad ope C/C++: Edit Cofiguratios (UI).
  • Ope "Advaced Settigs" ad paste the full path to compile_commads.jso ito the "Compile Commads" textbox.

Ruig tests

There curretly is't a turkey way to ru all the Trito tests, but you ca follow the followig recipe.

# Oe-time setup.  Note we have to reistall local Trito because torch
# overwrites it with the public versio.
$ pip istall scipy umpy torch pytest lit && pip istall -e pytho

# Ru Pytho tests usig your local GPU.
$ pytho3 -m pytest pytho/test/uit

# Move to builddir.  Fill i <...> with the full path, e.g.
# `cmake.liux-x86_64-cpytho-3.11`.
$ cd pytho/build/cmake<...>

# Ru C++ uit tests.
$ ija test

# Ru lit tests.
$ lit test

You may fid it helpful to make a symlik to the builddir ad tell your local git to igore it.

$ l -s pytho/build/cmake<...> build
$ echo build >> .git/ifo/exclude

The you ca e.g. rebuild ad ru lit with the followig commad.

$ ija -C build && ( cd build ; lit test )

Chagelog

Versio 2.0 is out! New features iclude:

• May, may bug fixes
• Performace improvemets
• Backed rewritte to use MLIR
• Support for kerels that cotai back-to-back matmuls (e.g., flash attetio)

Cotributig

Commuity cotributios are more tha welcome, whether it be to fix bugs or to add ew features at github. For more detailed istructios, please visit our cotributor's guide.

Compatibility

Supported Platforms:

• Liux

Supported Hardware:

• NVIDIA GPUs (Compute Capability 7.0+)
• Uder developmet: AMD GPUs, CPUs

Triton This is the development repository of Triton, a language and compiler for writing highly effi