当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。

您可以通过如下git cloe命令，或者ModelScope SDK来下载模型

Mega-NeRF

This repository cotais the code eeded to trai Mega-NeRF models ad geerate the sparse voxel octrees used by the Mega-NeRF-Dyamic viewer.

The codebase for the Mega-NeRF-Dyamic viewer ca be foud here.

Note: This is a prelimiary release ad there may still be outstadig bugs.

Citatio

@IProceedigs{Turki_2022_CVPR,
    author    = {Turki, Haithem ad Ramaa, Deva ad Satyaarayaa, Mahadev},
    title     = {Mega-NERF: Scalable Costructio of Large-Scale NeRFs for Virtual Fly-Throughs},
    booktitle = {Proceedigs of the IEEE/CVF Coferece o Computer Visio ad Patter Recogitio (CVPR)},
    moth     = {Jue},
    year      = {2022},
    pages     = {12922-12931}
}

Demo

Setup

coda ev create -f eviromet.yml
coda activate mega-erf

The codebase has bee maily tested agaist CUDA >= 11.1 ad V100/2080 Ti/3090 Ti GPUs. 1080 Ti GPUs should work as well although traiig will be much slower.

Pretraied Models

Traied with 8 submodules (to compare with mai paper)

• Rubble: model cluster masks
• Buildig: model cluster masks
• Quad: model cluster masks
• Residece: model cluster masks
• Sci-Art: model cluster masks
• Campus: model cluster masks

Larger models (traied with 25 submodules with 512 chaels each)

• Rubble: model cluster masks
• Buildig: model cluster masks
• Residece: model cluster masks
• Sci-Art: model cluster masks

Data

Mill 19

• The Buildig scee ca be dowloaded here.
• The Rubble scee ca be dowloaded here.

UrbaScee 3D

1. Dowload the raw photo collectios from the UrbaScee3D dataset
2. Dowload the refied camera poses for oe of the scees below:

• Residece
• Sci-Art
• Campus

1. Ru pytho scripts/copy_images.py --image_path $RAW_PHOTO_PATH --dataset_path $CAMERA_POSE_PATH

Quad 6k Dataset

1. Dowload the raw photo collectios from here.
2. Dowload the refied camera poses
3. Ru pytho scripts/copy_images.py --image_path $RAW_PHOTO_PATH --dataset_path $CAMERA_POSE_PATH

Custom Data

We strogly recommed usig PixSFM to refie camera poses for your ow datasets. Mega-NeRF also assumes that the dataset is properly geo-refereced/aliged such that the secod value of its ray_altitude_rage parameter properly correspods to groud level. If usig PixSFM/COLMAP the model_aliger utility might be helpful, with Mahatta world aligmet beig a possible fallback optio if GPS aligmet is ot possible. We provide a script to covert from PixSFM/COLMAP output to the format Mega-NeRF expects.

If creatig a custom dataset maually, the expected directory structure is:

• /coordiates.pt: Torch file that should cotai the followig keys:
• 'origi_drb': Origi of scee i real-world uits
• 'posescalefactor': Scale factor mappig from real-world uit (ie: meters) to [-1, 1] rage
• '/{val|trai}/rgbs/': JPEG or PNG images
• '/{val|trai}/metadata/': Image-specific image metadata saved as a torch file. Each image should have a correspodig metadata file with the followig file format: {rgb_stem}.pt. Each metadata file should cotai the followig keys:
• 'W': Image width
• 'H': Image height
• 'itrisics': Image itrisics i the followig form: [fx, fy, cx, cy]
• 'c2w': Camera pose. 3x3 camera matrix with the covetio used i the origial NeRF repo, ie: x: dow, y: right, z: backwards, followed by the followig trasformatio: torch.cat([camera_i_drb[:, 1:2], -camera_i_drb[:, :1], camera_i_drb[:, 2:4]], -1)

Traiig

1. Geerate the traiig partitios for each submodule: pytho scripts/create_cluster_masks.py --cofig cofigs/mega-erf/${DATASET_NAME}.yml --dataset_path $DATASET_PATH --output $MASK_PATH --grid_dim $GRID_X $GRID_Y
   • Note: this ca be ru across multiple GPUs by istead ruig pytho -m torch.distributed.ru --stadaloe --odes=1 --proc_per_ode $NUM_GPUS --max_restarts 0 scripts/create_cluster_masks.py <args>
2. Trai each submodule: pytho mega_erf/trai.py --cofig_file cofigs/mega-erf/${DATASET_NAME}.yml --exp_ame $EXP_PATH --dataset_path $DATASET_PATH --chuk_paths $SCRATCH_PATH --cluster_mask_path ${MASK_PATH}/${SUBMODULE_INDEX}
   • Note: traiig with agaist full scale data will write hudreds of GBs / several TBs of shuffled data to disk. You ca dowsample the traiig data usig trai_scale_factor optio.
   • Note: we provide a utility script based o parscript to start multiple traiig jobs i parallel. It ca ru through the followig commad: CONFIG_FILE=cofigs/mega-erf/${DATASET_NAME}.yaml EXP_PREFIX=$EXP_PATH DATASET_PATH=$DATASET_PATH CHUNK_PREFIX=$SCRATCH_PATH MASK_PATH=$MASK_PATH pytho -m parscript.dispatcher parscripts/ru_8.txt -g $NUM_GPUS
3. Merge the traied submodules ito a uified Mega-NeRF model: pytho scripts/merge_submodules.py --cofig_file cofigs/mega-erf/${DATASET_NAME}.yaml --ckpt_prefix ${EXP_PREFIX}- --cetroid_path ${MASK_PATH}/params.pt --output $MERGED_OUTPUT

Evaluatio

Sigle-GPU evaluatio: pytho mega_erf/eval.py --cofig_file cofigs/erf/${DATASET_NAME}.yaml --exp_ame $EXP_NAME --dataset_path $DATASET_PATH --cotaier_path $MERGED_OUTPUT

Multi-GPU evaluatio: pytho -m torch.distributed.ru --stadaloe --odes=1 --proc_per_ode $NUM_GPUS mega_erf/eval.py --cofig_file cofigs/erf/${DATASET_NAME}.yaml --exp_ame $EXP_NAME --dataset_path $DATASET_PATH --cotaier_path $MERGED_OUTPUT

Octree Extractio (for use by Mega-NeRF-Dyamic viewer)

pytho scripts/create_octree.py --cofig cofigs/mega-erf/${DATASET_NAME}.yaml --dataset_path $DATASET_PATH --cotaier_path $MERGED_OUTPUT --output $OCTREE_PATH

Ackowledgemets

Large parts of this codebase are based o existig work i the erf_pl, NeRF++, ad Pleoctree repositories. We use svox to serialize our sparse voxel octrees ad the geerated structures should be largely compatible with that codebase.

当前模型的贡献者未提供更加详细的模型介绍。模型文件和权重，可浏览“模型文件”页面获取。 您可以通过如下git clone命令，或者ModelScope SDK来下载模型 Mega-NeRF This