模型描述 (Model Descriptio)

这是布局生成方法LayoutDM的官方实现。

Top: LayoutDM is traied to gradually geerate a complete layout from a blak state i discrete state space. Bottom: Durig samplig, we ca steer LayoutDM to perform various coditioal geeratio tasks without additioal traiig or exteral models.

• 项目主页
• paper

运行环境 (Operatig eviromet)

pip istall modelscope https://data.pyg.org/whl/torch-2.0.0%2Bcu118/torch_sparse-0.6.17%2Bpt20cu118-cp38-cp38-liux_x86_64.whl prdc==0.2 pytorch-fid==0.2.1

使用范围和应用场景

背景无关、从空白画布生成多种元素的布局，例如UI界面、PPT制作和杂志文章排版等；

如何使用

代码范例

from modelscope.pipelies import pipelie

# ru pipelie
iput = {
    '_samples': 4,  # geeratio _samples after iferece
}
iferece = pipelie('layout-geeratio', model='chehyer/LayoutDM_layout_geeratio', model_revisio='v1.6.0')
pred = iferece(iput)


# visualizatio for PubLayNet model
from traier.helpers.visualizatio import save_image
import seabor as ss

SIZE = (360, 240)
labels = [
    "text",
    "title",
    "list",
    "table",
    "figure",
]
def colors(um_classes):
    _colors = um_classes
    colors = ss.color_palette("husl", _colors=_colors)
    colors = [tuple(map(lambda x: it(x * 255), c)) for c i colors]
    retur colors

save_kwargs = {
    "colors": colors(um_classes=le(labels)), "ames": labels,
    "cavas_size": SIZE, "use_grid": True,
    "draw_label": True,  # Whether to display the category ame of each box i the resultig image, such as text, table, etc
}
save_kwargs['out_path'] = 'pred_ucod_ms.pg'  # visualize ucoditioal geeratio result
save_image(pred["bbox"], pred["label"], pred["mask"], **save_kwargs)

模型局限性以及可能的偏差

• 只有坐标信息，不包含背景图图像特征，不能根据背景信息生成不遮挡主体的布局
• 无法实现文字颜色等属性的预测
• 输入输出都是box，如果商品细长斜放在图像对角线，可能无法感知商品主体的mask区域；

模型效果

Ucoditioal Geeratio Result of PubLayNet

Citatio

如果您发现此工作对您的研究有帮助，请考虑引用以下BibTeX条目。

@iproceedigs{ioue2023layout,
  title={LayoutDM: Discrete Diffusio Model for Cotrollable Layout Geeratio},
  author={Naoto Ioue ad Kotaro Kikuchi ad Edgar Simo-Serra ad Mayu Otai ad Kota Yamaguchi},
  booktitle={Proceedigs of the IEEE/CVF Coferece o Computer Visio ad Patter Recogitio (CVPR)},
  year={2023},
  pages={10167-10176},
}

模型描述 (Model Description) 这是布局生成方法LayoutDM的官方实现。 Top: LayoutDM is trained to gradually gene