Extend3D: Town-Scale 3D Generation

Seungwoo Yoon Jinmo Kim Jaesik Park*

Seoul National University
{dotori000, jmkim1012, [Link]}@[Link]
arXiv:2603.29387v1 [[Link]] 31 Mar 2026

Landmark Occluded Area

Single 2D Image

3D Vatican City
Figure 1. The result of Extend3D. We generated a large-scale 3D scene from an image of Vatican City captured from Google Earth [9].

Abstract discovered that treating the incompleteness of 3D structure

as noise during 3D refinement enables 3D completion via
In this paper, we propose Extend3D, a training-free a concept, which we term under-noising. Furthermore, to
pipeline for 3D scene generation from a single image, built address the sub-optimality of object-centric models for sub-
upon an object-centric 3D generative model. To overcome scene generation, we optimize the extended latent during
the limitations of fixed-size latent spaces in object-centric denoising, ensuring that the denoising trajectories remain
models for representing wide scenes, we extend the latent consistent with the sub-scene dynamics. To this end, we in-
space in the x and y directions. Then, by dividing the ex- troduce 3D-aware optimization objectives for improved ge-
tended latent space into overlapping patches, we apply the ometric structure and texture fidelity. We demonstrate that
object-centric 3D generative model to each patch and cou- our method yields better results than prior methods, as evi-
ple them at each time step. Since patch-wise 3D generation denced by human preference and quantitative experiments.
with image conditioning requires strict spatial alignment project page
between image and latent patches, we initialize the scene
using a point cloud prior from a monocular depth estimator
and iteratively refine occluded regions through SDEdit. We

* Corresponding author.

1
1. Introduction the latents at each time step using 3D-aware optimization
objectives to align the image and point cloud, ensuring that
In the modern era, 3D scene assets are essential across fields the denoising paths remain consistent with the sub-scene
such as game development, filmmaking, animation, simu- dynamics.
lation, and other areas of content production. Creating de- The qualitative results show that our method is scalable
tailed 3D scenes requires substantial human effort and re- and generalizable. Through human preference and quanti-
sources, even with the provided 3D assets. Therefore, a tai- tative experiments, we demonstrate that our method outper-
lored generative model for 3D scenes would help reduce forms state-of-the-art models in terms of geometry, appear-
such costs and enhance productivity in industries. ance, and completeness, and is more faithful to the given
Despite recent advances in 3D generative models, which image. Through an ablation study, we also demonstrate that
have enabled the creation of production-ready high-quality overlapping patch-wise flow, initialization, and optimiza-
3D objects, generating large-scale 3D scenes remains chal- tion are crucial for training-free 3D scene generation.
lenging. One of the main challenges is that most current The main contributions of this paper are:
3D datasets [3, 4, 8] consist of object-centric data, and • We extend the latent space to integrate object-centric
lack cases with complex arrangements of multiple objects models into 3D scene generation, enabling a more gen-
and a background. Consequently, previous data-centric ap- eralizable and scalable generation pipeline.
proaches were unable to generate large general scenes. • We introduce an overlapping patch-wise flow with image
Moreover, existing latent generative models [44, 47] rep- conditioning that captures local information and mitigates
resent 3D data with a fixed latent size, thereby limiting the errors arising from object-centric models.
level of detail of generated results. As the 3D scene grows • We incorporate an iterative under-noised SDEdit process
in size, the output becomes blurry due to the limited latent and 3D-aware optimization to complete occluded regions
dimensionality, resembling a low-resolution image. To ade- in the monocular depth point cloud and to overcome the
quately represent the scene’s details, the latent size should deviation of object-centric models from scene dynamics.
be adapted to the scale of the result.
Therefore, research has been conducted to develop
2. Related Work
training-free pipelines for generating 3D scenes using
object-centric models. Previous work has explored gener- 3D generative models. There have been numerous recent
ating 3D scene blocks through an outpainting process [7, studies on generative models that can generate 3D objects
49]. However, results from these approaches indicate that conditioned on text or images. Currently, their main ap-
outpainting can degrade block consistency, particularly in proach is the latent flow model [22, 33] applied to voxel-
large-scale scenes, making seams visible. Moreover, they based or set-based latents.
rely entirely on the sub-scene generation capabilities of Trellis [44] generates 3D Gaussians [13], radiance
object-centric models, which are not sufficient. field [29], and mesh, using two steps of latent flow mod-
In this paper, we introduce Extend3D, a novel training- els where each generates a voxelized sparse structure and
free pipeline for generating 3D scenes from a single im- structured latents. Hunyuan3D [47] utilizes the latent flow
age. To achieve greater detail and scalability in large-scale model to generate shapes with set-based latents, as proposed
3D scene generation, we have expanded the latent space in [45]. TripoSG [18] also uses the set-based latent repre-
of a pre-trained 3D object generation model. Inspired by sentation of [45] to generate a mesh. These models have the
training-free, high-resolution image generation methods, limitation that they are trained with object-centric datasets.
such as those presented in recent works [1, 6, 10, 17, 20, Moreover, structurally, current flow-based approaches suf-
21, 42], we divide the extended latent space into overlap- fer from the limitation that their latent size is predefined, so
ping patches and generate them simultaneously. Unlike pre- the output 3D can only have a confined range of details. We
vious outpainting methods, our approach automatically re- solve these problems by extending the latents to represent a
fines fine object details within the scene. This is possible be- large-scale scene.
cause neighboring overlapping patches can influence each To overcome the issues of object-centric models, some
other, increasing the likelihood of accurately reconstructing attempts have been made to train models using 3D scene
their 3D representations. datasets. BlockFusion [43] trains a diffusion model to gen-
However, there are challenges in 2D-3D spatial align- erate cropped sub-scenes and generate the scene by extrap-
ment and the object-centrality of pretrained models. To olation. PDD [23] trains a multi-scale diffusion model for
overcome this, we used the input image and the point cloud coarse-to-fine scene generation. LT3SD [28] generates a
extracted from the monocular depth estimator [39] as priors 3D scene hierarchically with a latent tree representation.
to initialize and optimize the extended latents. We initialize NuiScene [16] trains an autoregressive model with chunk
the structure from the point cloud and refine the occluded VAE and vector sets. Nevertheless, since all of these meth-
regions using SDEdit [27] with under-noising. We optimize ods are trained on limited datasets, they can generate 3D

2
scenes with fewer categories than object-centric models. lead to repeated objects. We therefore provide structure pri-
They also do not consider detailed model conditioning, such ors to generate a high-fidelity 3D scene.
as image conditions, when designing hierarchical frame-
Generation with priors. Several studies are trying to pro-
works. Unlike them, our method can generate general 3D
vide priors for pre-trained generative models for various
scenes with detailed image conditioning.
purposes. SDEdit [27] is a representative method of image
Training-free 3D scene generation. Recent advances in editing that can be applied to [11, 22, 33, 34]. SDEdit par-
object-centric 3D generative models and the shortage of 3D tially noise the original image, producing an edited image
scene datasets have led researchers to develop training-free whose perturbed distributions retain the original image’s
3D scene generation pipelines using these object-centric style, meeting the intended image style. Readout Guid-
models. ance [25] trains a small neural network to extract properties
SynCity [7] generates tiles of 3D sub-scenes sequentially (e.g., pose, depth, or edges) from the intermediate latent rep-
with Trellis from a text using Flux inpainting [15]. Because resentation. Then, it computes the loss with respect to the
SynCity attaches separate 3D sub-scenes, there are incon- property and provides a loss gradient as guidance, similar
sistencies between tiles, and seams are visible. An image- to the classifier guidance [5].
to-3D scene generation pipeline, 3DTown [49], initializes We apply SDEdit in our Extend3D to refine the initial-
a scene with the point cloud from VGGT [37] and then ized structure. Unlike image editing, we propose an under-
completes it patch by patch using RePaint [24] and Trel- noising technique designed for the 3D completion task.
lis. Although 3DTown can generate 3D towns from images Also, instead of guidance, we optimize the intermediate la-
with high fidelity, it can only be used with restricted input tent with a loss explicitly designed for 3D scene generation,
due to the limitations of object-centric models (e.g., van- assuming that the priors have ground-truth knowledge of 3D
ishing floors). Also, regardless of initialization, some ob- structure and texture.
jects in the scene ignore certain input information, such as
rotation. EvoScene [48] further leverages a video diffusion 3. Preliminaries
model [36] on 3DTown, but suffers from similar problems.
To address the problems of separate and sequential 3D
3.1. Latent Flow Model for 3D Generation
sub-scene generation, we simultaneously generate 3D sub- A modern approach for high-quality 3D generative models
scenes with interacting denoising paths. With small transi- is the latent flow model. They use voxelized latents of fixed
tions between overlapping patches, the generation process size or set-based latents (e.g., point clouds) within a con-
can effectively capture local information and prevent geo- fined region to represent 3D space. While our approach is
metrical errors through simultaneous generation. Also, un- not restricted to a specific generative model, it can be ap-
like previous works that rely solely on sub-scene generation plied to general voxel-based latents or set-based latent flow
using an object-centric model, we optimize the latent rep- models. We illustrate our idea using Trellis [44], which is
resentation at each step to prevent paths from transitioning one of the leading 3D generative models.
from sub-scene to object dynamics. Trellis generates 3D representations with two steps of
latent flow models given a condition CI encoded from an
Training-free high-resolution image generation. In the
image I by DINOv2 [31], and both steps are generaliz-
field of image generation, training-free high-resolution im-
able to flow models for voxelized or set-based latents. The
age generation has been widely researched and has led to
first step of the model generates a sparse structure (SS)
massive discoveries on the dynamics of the scaled-up la-
{pi } ⊂ [M ]3 (where [M ] := {0, 1, ..., M − 1}), which
tent denoising process. The primary purpose of this area is
represents a set of occupied coordinates in a voxel grid. In
to generate high-resolution images from pre-trained models
sparse structure generation, low-resolution voxelized noise
trained on relatively low-resolution data. N ×N ×N
ZSS
1 ∈ R is denoised to ZSS
0 with vector field vSS ,
MultiDiffusion [1] generates a high-resolution image decoded with decoder D, and activated voxel coordinates
from text with an extended 2D latent with overlapping are collected as:
patches. DemoFusion [6] solves the object repetition prob-
lem of Multidiffusion with two ideas: progressive upsam-
\latent _1^\mathrm {SS} \sim \mathcal {N}(\bm 0, \mI ), \quad \frac {d}{dt}\latent _t^\mathrm {SS}=\fvec _\mathrm {SS}(\latent _t^\mathrm {SS}, C_\image , t), \\ \{\pos _i\}=\{\pos : \mathcal {D}(\latent _0^{SS})_\pos > 0\}.
pling and dilated sampling. Later research [20, 21], addi-
tionally refines dilated sampling. (2)
When these methods are naively applied to extended 3D
latent generation, however, we found that they fail to gener- As the decoder is trained as a VAE, there is a trained en-
ate 3D scenes with high fidelity due to the unique dynamics coder E that encodes the occupancy grid O ∈ RM ×M ×M
of the model’s image-condition, 3D, and object centrality. into a low-resolution latent representation. The second step
For instance, the floor vanishes, or poorly correlated patches of the model conducts denoising on a structured latent

3
 × step Optimize with (Sec. 4.3)

Overlapping
Patch-wise
Flow
Monocular
Extended Latent (Fig. 3) Extended Vector Extended Latent Depth
Point Cloud Estimator Image

n Output
tio Priors
Sparse Structure Denoising aliza
iti
(Sec. 4.1) In Optimize with (Sec 4.3)
× × step
Overlapping
denoise decode Patch-wise
noise to 𝑡!"#$% encode Flow

Extended Latent (Fig. 3) Extended Vector Extended Latent

Extended Latent Extended Latent Occupancy
Voxel

1. Sparse Structure Generation (Iterative SDEdit) 2. Structured Latent Generation

(Sec. 4.2) (Sec. 4.1)
Figure 2. An overall pipeline of our Extend3D. Extend3D consists of two parts: sparse structure generation and structured latent gener-
ation. In the denoising part of both steps, an overlapping patch-wise flow was used (Sec. 4.1 and Fig. 3). In sparse structure generation,
iterative SDEdit is used to initialize the structure (Sec. 4.2). Vector fields in both steps are optimized with priors (Sec. 4.3).

(SL AT), where a set-based latent feature is matched to a generation, we extended the 3D latents of a pre-trained
coordinate of sparse structure as: object-centric 3D generative model [44] to represent more
detailed, larger 3D scenes. We extend the latents in the x and
\latent _t^\mathrm {\SLat }=\{(\pos _i, \lat _{i, t})\}\subset [M]^3\times \mathbb {R}^l, \\ \lat _{i, 1} \overset {\mathrm {iid}}{\sim } \mathcal {N}(\bm 0, \mI ), \quad \frac {d}{dt}\latent _t^\mathrm {\SLat }=\fvec _\mathrm {\SLat }(\latent _t^\mathrm {\SLat }, C_\image , t),
y coordinates, and a portion of the extended latent serves as
(4) a conventional latent for the pre-trained object-centric 3D
generative model.
with invariant pi and vector field vSL AT . SL AT is then de- To handle extended latents, we divide them into over-
coded to 3D representations such as 3D Gaussians, radi- lapping patches, generated simultaneously via separate but
ance field, or mesh by sparse decoders (DGS , DNeRF , and coupled denoising paths conditioned on image patches
Dmesh ), and usually. In this paper, we will use the notations (Sec. 4.1). Additionally, to address the underlying issues of
Zt that can refer to both ZSS
t and Zt
SL AT
, and v for vSS and the object-centric model (e.g., vanishing floor, inability to
vSL AT for simplicity. generate sub-scenes, and randomly rotated objects) and to
3.2. SDEdit mitigate the problems associated with patch-wise genera-
tion (e.g., repeated objects and seams between patches), we
We introduce SDEdit to refine the initialized structure, treat- incorporate priors into the generation process. We first ini-
ing scene generation as a 3D sub-scene editing task. SDEdit tialize the scene with a point cloud from a depth estimator
(g)
noises latent of a “guide” (e.g., image to be edited) Z0 to and perform iterative under-noised SDEdit. This completes
Ztstart and denoises it to Z0 to get the edited result. With the the occluded area and refines the scene while generating the
added noise, the perturbed distribution meets the intended structure (Sec. 4.2). We then optimize the scene at every
distribution while preserving information in the guidance. time step using the point cloud and an image of the entire
Although SDEdit was designed for diffusion models [35], scene. We also propose a loss function that treats the point
we can integrate it into flow models, with the following cloud as a prior for the voxel-based latent (Sec. 4.3). The
equations: overall pipeline is illustrated in Fig. 2 and Sec. A.1.
\label {SDEdit_noise} \latent _{t_\mathrm {start}}=(1-t_\mathrm {start})\cdot \latent _0^{(g)}+t_\mathrm {start}\cdot \noise ; \hspace {0.3em} \noise \sim \mathcal {N}(\bm 0, \mI ), \\ \frac {d}{dt}\latent _t=\fvec (\latent _t, C, t), 4.1. Overlapping Patch-wise Flow
(6) In order to generate a detailed 3D structure and texture, we
introduce an extended latent for sparse structure ZSS t ∈
where C refers to the editing condition. When tstart in- RaN ×bN ×N and an extended SL AT ZtSL AT ⊂ [aM ] ×
creases, the denoising path gets longer, causing the effect [bM ] × [M ] × Rl where Zt can refer to both. Here, a and
of conditioning and generative models to be enlarged. b are extension factors. (From here, we will use ↓ to notate
non-extended latents or vectors.)
4. Method
We divide these latents into overlapping patches with a
Extend3D is a training-free pipeline that generates a 3D division factor d. We refer to the (i, j)-th latent patches as
scene from a single scene image. To implement 3D scene ϕSS SS SL AT
i,j (Zt ) and ϕi,j (Zt
SL AT
). This process can be described

4
 Overlapping Patch-wise Flow Overlapping Patch Sampling (𝝓)
Sliding Window 𝕎
Patchify

𝐾
Image
Condition 𝐾⁄𝑑
Averaged
Flow (𝒗↓𝒕 ) Extended Latent Patch (𝑖, 𝑗)
𝜙 Flow (𝒗↓𝒕 ) Σ𝜙 !&
Overlapping Patch Coupling (𝚺𝝓!𝟏 )
Flow (𝒗↓𝒕 ) !&
𝜙#,% Zero Padding
Extended Latent 𝒁) Extended Vector 𝒗𝒕
Flow (𝒗↓𝒕 )

Overlapping Patches Overlapping Patches Extended Latent

(Latent) (Vector) Patch (𝑖, 𝑗)

Figure 3. Overlapping patch-wise flow. The extended latent is divided into latent patches with the sliding window. We then obtain the
patch vector for each latent patch and merge them into a single extended latent vector, thereby coupling the patches.

as a N 3 or M 3 -sized sliding window W moving with stride consistent global structure. We apply dilated sampling dur-
N/d or M/d to sample patches, illustrated as sampling ing the sparse structure generation phase and leave the de-
in Fig. 3. The patches can be mapped back to their origi- tails to Sec. A.3.
nal positions by setting the values at the other positions to
zero (zero padding), thereby coupling them, as illustrated in 4.2. Initialize with Prior
−1
Fig. 3. We represent these inverse mappings as (ϕSS
i,j ) and When directly denoising sparse structure from pure Gaus-
SL AT −1 sian noise using Eq. (9), all patches fail to initialize each
(ϕi,j ) . We leave the rigorous definitions of the map-
pings in Sec. A.5. sub-scene due to the inherent limitation of the object-centric
We also patchify the image condition with ψi,j , which models. Moreover, the coarse structure is determined dur-
crops the image region to exactly match the (i, j)-th 3D ing the early denoising stage [42], before the patches are
patch (see details in Sec. A.2). Similar to MultiDiffusion, sufficiently coupled, so that the image condition and the 3D
we get the vector field of the extended latents by merging latent are not well spatially aligned. Consequently, the out-
the vector fields for each patch, where the overlapping re- put becomes noisy, fragmented, and unstable as in Fig. 7
gions are averaged across the patches, as illustrated in the (B). This motivates the need for a robust structural prior at
left side of Fig. 3. The entire overlapping patch sampling, initialization.
merging, and denoising process can be formulated as: Inspired by 3DTown [49], we initialize the scene struc-
ture with a point cloud P extracted from a monocular depth
\label {indep vector} \fvec _{i, j}(\elatent _t,\, \image ,\, t)=\original {\fvec }\bigl ( \patch _{i,j}(\elatent _t),\, C_{\psi _{i,j}(\image )},\, t \bigr ), \\ \label {extended vector} \efvec (\elatent _t,\, \image ,\, t) = \sum _{i,j} \patch _{i,j}^{-1}\!\bigl ( \fvec _{i, j} \bigr ) ~\oslash ~ \sum _{i,j} \mathbf {1}_{\sliding _{i,j}}, \\ \label {extended_dynamics} \frac {d}{dt}\elatent _t=\efvec (\elatent _t, \image ,t), estimator. Specifically, we adopt MoGe-2 [38, 39] for our
Extend3D. The predicted point cloud is voxelized into an
occupancy grid O0 ∈ RaM ×bM ×M . Because the monoc-
ular depth estimator cannot infer the occluded regions, the
(9) resulting occupancy voxel grid contains empty areas that
should be rectified using the pre-trained generative model.
(g)
where ⊘ is an element-wise division. Equation (7) can be To address this, with an encoded voxel grid Z0 = E(O0 ),
calculated independently from the other patches and in par- our Extend3D performs SDEdit. Unlike standard SDEdit,
allel, but the dynamics of different patches, even far away, ① ② ③
can be coupled by overlaps.
The advantage of divided but coupled dynamics is the
ability to refine errors in other patches. By detecting slight
① noise ②
movement of the sliding window, our method can iden-
tify local information from changes in the image and in la- 𝑡#$%&' 𝑡&()*(
tent features between patches. Additionally, because some 𝒁! 𝒁"
objects are at the centers of patches, we can leverage the ③ denoise

object-centric model more effectively. The beneficial effect

of overlapping patch-wise flow can be found in Fig. 7 (A). Figure 4. Motivation of under-noising. The blue arrows represent
Noted in DemoFusion [6], AccDiffusion [21], and Cut- actual noising or denoising, while the purple arrow illustrates how
Diffusion [20], dilated sampling is crucial for generating a the model is presumed to perceive.

5
which applied Eq. (5), we introduce under-noising: connect the point cloud priors and generated voxels, not just
by attaching two distinct voxel grids. With the LSS , we op-
\label {extend3d_noise} \elatent _{t_\mathrm {start}}=(1-t_\mathrm {noise})\cdot \elatent _0^{(g)}+t_\mathrm {noise}\cdot \noise , \quad \noise \sim \mathcal {N}(\bm 0, \mI ), (10) timize v̂t with Adam optimizer [14].
In the structured latent generation step, we apply the
where tstart > tnoise , ensuring that the latent is denoised extended rendering loss [40, 46] as follows:
more aggressively than it was originally noised. By under-
noising the guide structure, the pre-trained model may treat \hat {\image }=\mathrm {Render}(\mathcal {D}_{\mathrm {GS}}(\elatent _t^{\mathrm {\SLat }}-t\cdot \oefvec ), \camera ), \\ \mathcal {L}_{\mathrm {\SLat }}=\mathrm {LPIPS}(\hat {\image }, \image )-\mathrm {SSIM}(\hat {\image }, \image ),
missing or occluded parts as additional noise, illustrated as
the arrow 2 in Fig. 4. Finally, the denoising process, rep- (13)
resented as arrow 3 , allows such areas to be filled. This
where Render is a differentiable renderer (such as Gaussian
is similar to adding high-frequency noise to enhance image
splatting) and P is a camera parameter of an image view-
detail in image super-resolution [12]. We empirically vali-
point provided by the depth estimator. This optimizes the
date this choice in Sec. 5.4.
entire scene with an image in the original camera view. Be-
SDEdit can fill the unwanted empty areas. However, it
cause an object-centric model often loses details in scene
often fails to fully complete the scene, leaving some holes.
textures, this optimization helps refine them. Also, it makes
To mitigate this, as a single SDEdit process partially re-
the seams invisible because the boundary is optimized at ev-
fines the structure, we apply SDEdit iteratively as: On =
ery time step, ensuring paths are more consistent with each
SDEdit(On−1 ), represented in Fig. 2. This process itera-
other. With LSL AT , we also optimize v̂t with Adam.
tively fills the occluded region of O0 and eventually com-
Please refer to Sec. A.1 for the algorithm details.
pletes the scene.
4.3. Optimize with Prior 5. Experiments
During denoising, sub-scenes deviate from a scene-like 5.1. Human Preference
structure toward an object-like structure due to the object-
centric model’s properties, leading to distortion or a vanish- Table 1. Human preference win rate (%) of our method.
ing floor, even with proper initialization. To prevent devi- versus. Geometry Faithfulness Appearance Completeness
ation and to align the denoising paths with the condition- Trellis [44] 50.0 66.4 67.1 62.1
ing, we optimize the extended latents over time steps us- Hunyuan3D [47] 73.6 75.7 75.0 75.0
ing the point cloud and the image. When solving Eq. (9) EvoScene [48] 87.1 87.9 87.1 87.1

with the discrete ODE solver, instead of moving directly

along v(Zt , I, t), we use v̂t , an optimized vector starting To score the visual aestheticity of 3D scenes, we con-
from v(Zt , I, t). By optimization, we can leverage the pre- ducted a human preference study. We compared our method
trained model for the occluded region while optimizing on with Trellis [44] and Hunyuan3D-2.1 [47], the current best
the seen region, as in Readout Guidance [25]. In addition, open-sourced 3D generation models, and EvoScene, which
optimizing the vector field can improve consistency across is specifically designed for large-scene generation. Human
patches by simultaneously optimizing the entire scene, as in annotators (10 participants) ranked the methods on four cri-
[17]. We introduce two optimization losses, one for sparse teria: geometry, faithfulness, appearance, and completeness,
structure generation and one for structured latent genera- given 14 images and 3D scenes. As a result (Tab. 1), our Ex-
tion, as explained in Sec. 3.1 and illustrated in Fig. 2. tend3D outperformed previous methods in four criteria.
In the sparse structure generation step, we define:
5.2. Quantitative Results
\label {ss_optim_loss} \mathcal {L}_{\mathrm {SS}}=-\frac {1}{|\pc |}\sum _{\pos \in \pc }\log {\sigma ((\mathcal {D}(\elatent _t^{\mathrm {SS}}-t\cdot \oefvec _t)})_\pos ), (11) Table 2. Quantitative results.
LPIPS ↓ SSIM ↑ PSNR ↑ CD ↓ F-score (0.05) ↑
where σ is a sigmoid function. The loss function is designed Trellis [44] 0.650 0.239 10.0 0.0315 0.442
to enforce that the initialized voxels do not disappear during Hunyuan3D [47] 0.683 0.255 10.4 0.0192 0.567
EvoScene [48] 0.482 0.310 13.2 0.0188 0.498
the denoising process, motivated by binary cross-entropy
Ours w/o LSL AT 0.400 0.333 13.8 0.0078 0.708
loss. It gives a positive signal on predicted voxels where Ours 0.240 0.611 20.4 0.0086 0.694
points exist. Voxels with dense point clouds will have more
weight in the loss. While this loss can be minimized by We render the 3D scene into the camera view of the in-
increasing the number of voxels, combined with the pre- put image using the camera parameter estimator [39], and
trained model every time step, it merely prevents the de- obtain LPIPS, SSIM, and PSNR scores on 100 input im-
sired voxels from disappearing, rather than creating unde- ages [9, 15, 19, 30, 41] spanning diverse wide scenes. As
sired voxels. Moreover, for the same reason, it can smoothly shown in Tab. 2, our method achieved the best scores across

6
 Ours EvoScene Trellis Hunyuan3D-2.1
Figure 5. Qualitative result of our Extend3D. Our 3D scene generation result (with a = b = 2) is compared to the results of state-of-
the-art 3D generative models. While previous methods may not accurately represent the image or lose scene details, our method effectively
expresses the image condition in 3D. The input image is generated using Flux.1 [dev] [15]. We provide additional results in Sec. A.7.

Ours SynCity
Figure 6. Qualitative comparison with SynCity. The results are generated from the text prompt, medieval market.

three metrics, indicating that it is most faithful to the in- a 36× larger result than the original latent space. We present
put image in terms of structure and texture. Using 45 im- more examples of wide scenes in Fig. 17 and Fig. 18. Also,
ages and ground-truth mesh pairs from the UrbanScene3D our Extend3D can represent general scenes. It can gener-
dataset [19], we evaluated the geometric results using the ate a town, a table of foods, a study scene, and an indoor
Chamfer Distance (CD) and F-score with a threshold of room, illustrated in Fig. 5 and Sec. A.7. In diverse cases, our
0.05. Table 2 shows that our method surpasses the results method outperformed previous 3D generative models. Also,
of the previous methods. compared with SynCity, our method can generate scenes
Table 3. Comparison between 3D scene generation methods. without patch boundaries, as shown in Fig. 6 and Fig. 15.
Moreover, because SynCity generates a scene in an out-
CLIP ↑ HPSv3 ↑ Intra-LPIPS ↓
painting approach, it cannot refine the unnatural edge of the
SynCity [7] 0.251 3.254 0.631
water. Compared to EvoScene in Fig. 16, our results had
Ours 0.276 3.519 0.571
less distorted geometry and more detailed textures.
We also compared our method with the state-of-the-
art training-free 3D scene generation pipeline, SynCity. 5.4. Ablation Study
Since SynCity is text-conditioned, whereas ours is image- We conducted an ablation study on three proposed methods
conditioned, we first generated scene images from key- in our Extend3D and presented the results in Fig. 7. When
words using ChatGPT [30] and then used Extend3D. We we obtained the results for Fig. 7 (A) and (B), we did not
render the results of SynCity and Extend3D to get CLIP optimize the latent to emphasize the structural difference.
score [32], HPSv3 [26], and Intra-LPIPS [17]. Here, Intra-
Overlapping Patch-wise Flow. We claim that the coupled
LPIPS refers to LPIPS between patches within a single
paths of patches can mutually rectify and effectively cap-
scene, measuring the patch consistency. Table 3 shows that
ture local information. To validate this argument, we com-
our Extend3D is superior to SynCity in text compatibility,
pare results for varying division factors. As illustrated in
quality, and patch-wise consistency.
Fig. 7 (A), d = 2 distorted local structure, while d = 4 did
5.3. Qualitative Results not. These results demonstrate that patch interactions cor-
rect each other, and the sliding window’s stride enables the
Figure 1 shows the scalability of our method. Given a town-
extended latent to capture finer details.
scale scene image, the extended latent can fully capture de-
tails, including landmarks and small buildings, and produce Initialize with Prior. In the first part of Fig. 7 (B), the re-

7
 (A) Overlapping Patch-wise Flow (B) Initialization with Priors

𝑑=2 𝑑=4 w/o Initialization w/ Initialization

(C) Optimization with Priors

𝒕𝐧𝐨𝐢𝐬𝐞 = 𝟎. 𝟔
𝒕𝐬𝐭𝐚𝐫𝐭 = 𝟎. 𝟖

𝑡)*+,- = 0.6
𝑡,./0. = 0.6

𝑡)*+,- = 0.8
𝑡,./0. = 0.8
+SS Optimization +SLat Optimization

Figure 7. Ablation study. All the images, except for the ablation of under-noising, are taken from the input image camera viewpoint. We
set a = b = 2 to generate the 3D scenes in this figure.
Table 5. Ablation study on prior initialization and optimization
Table 4. Ablation study for varying division factor d.
LPIPS ↓ SSIM ↑ PSNR ↑ CD ↓ F-score (0.05) ↑
LPIPS ↓ SSIM ↑ PSNR ↑ CD ↓ F-score (0.05) ↑
p.w. flow only 0.606 0.209 9.63 0.0348 0.261
d=2 0.251 0.598 19.8 0.0088 0.692 + initialize 0.425 0.312 13.0 0.0083 0.693
d=4 0.240 0.611 20.4 0.0086 0.694 + SS optim. 0.400 0.333 13.8 0.0078 0.708
d=8 0.237 0.615 20.5 0.0079 0.699 + SL AT optim. 0.240 0.611 20.4 0.0086 0.694
Table 6. Ablation study on under-noising.
tnoise / tstart LPIPS ↓ SSIM ↑ PSNR ↑ CD ↓ F-score (0.05) ↑
sults with and without initialization are compared. Without 0.6 / 0.6 0.388 0.324 13.4 0.0081 0.657
initialization (i.e., tstart = 1), the structure is totally broken 0.8 / 0.8 0.550 0.216 9.91 0.0292 0.378
with important objects disappeared, buildings not in proper 0.8 / 0.6
0.622 0.219 9.79 0.0518 0.249
(over-noise)
position, etc. We therefore conclude that proper initializa-
0.6 / 0.8
tion is essential for extended latents. In the second part, we (under-noise)
0.387 0.327 13.5 0.0078 0.680
compared three results with different tnoise and tstart . When
tnoise = tstart (usual SDEdit), the structure maintained the hances the quality of the scenes. From the geometric results
holes in the initial point cloud or was destroyed due to the in Tab. 5, we find that initialization and SS optimization re-
tstart trade-off in SDEdit. However, with tnoise < tstart fine the geometry, whereas SL AT optimization sometimes
(under-noising), the occluded region of the initial structure degrades it. Since SL AT optimization usually enhances a
is completed naturally. 3D scene’s texture, there is a trade-off between geometry
Optimize with Prior. Figure 7 (C) shows the ablation study and texture when optimizing the SL AT. Table 6, conducted
for optimization with priors. Starting from the base model, without optimization and with niter = 1 as in the qualita-
we sequentially added sparse structure and SL AT optimiza- tive experiment, demonstrates that under-noising is the best
tion. Without sparse structure optimization, the floor and choice of tstart and tnoise in 3D completion.
parts of the objects vanished, as in Sec. A.7. Structured la-
tent optimization could refine seams and distortion between 6. Conclusion
patches compared to those without optimization. Further-
We propose a training-free 3D scene generation pipeline,
more, the overall quality of the scene’s structure and texture
Extend3D. By the extended latent space of the pre-trained
improved (e.g., the fork and chips in the figure).
object-centric model, we enabled scalable 3D scene gener-
In addition, as in Sec. 5.2, we quantitatively evaluated the ation. We demonstrate that our method (overlapping patch-
generated scenes. Table 4 and Tab. 5 show the effective- wise flow, initialization, and optimization) and its schemes
ness of our proposed methods, consistent with the qualita- (iterative SDEdit, under-noising, and 3D-aware optimiza-
tive observation that increasing d and providing priors en- tion objectives) achieve notable improvements in image-

8
guided 3D scene generation. Chen, and Ying Shan. Scalecrafter: Tuning-free higher-
resolution visual generation with diffusion models. In ICLR,
Limitations. We found three limitations in our method. 2024. 2
Firstly, occluded region completion is sometimes incom- [11] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffu-
plete, for example, the one representing a room in Sec. A.7. sion probabilistic models. In NeurIPS, 2020. 3
Secondly, SL AT optimization requires considerable mem- [12] Jinho Jeong, Sangmin Han, Jinwoo Kim, and Seon Joo Kim.
ory, especially for large scenes (computational cost analysis Latent space super-resolution for higher-resolution image
is provided in Sec. A.4). Lastly, our framework shows lim- generation with diffusion models. In CVPR, 2025. 6
ited performance on street-level images. The problem is due [13] Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler,
to a significant mismatch between the scales of the x and y and George Drettakis. 3d gaussian splatting for real-time
coordinates, arising from the vanishing points. It would be a radiance field rendering. TOG, 42(4), 2023. 2
direction for future work to implement 3D generation from [14] Diederik P. Kingma and Jimmy Ba. Adam: A method for
a wider range of image types. stochastic optimization. In ICLR, 2015. 6
[15] Black Forest Labs. Flux. [Link]
Acknowledgements black-forest-labs/flux, 2024. 3, 6, 7, 15, 24
[16] Han-Hung Lee, Qinghong Han, and Angel X. Chang.
This work was supported by the IITP grant funded by Nuiscene: Exploring efficient generation of unbounded out-
MSIT [[Link]-2021-II211343: AI Graduate School (Seoul door scenes. In ICCV, 2025. 2
National University) (5%), [Link]-2025-25442338: AI [17] Yuseung Lee, Kunho Kim, Hyunjin Kim, and Minhyuk
Star Fellowship (45%), and [Link]-2025-02303703: Real- Sung. Syncdiffusion: Coherent montage via synchronized
world multi-space fusion and 6DoF free-viewpoint immer- joint diffusions. In NeurIPS, 2023. 2, 6, 7
sive visualization for extended reality (50%)]. [18] Yangguang Li, Zi-Xin Zou, Zexiang Liu, Dehu Wang, Yuan
Liang, Zhipeng Yu, Xingchao Liu, Yuan-Chen Guo, Ding
References Liang, Wanli Ouyang, et al. Triposg: High-fidelity 3d shape
synthesis using large-scale rectified flow models. arXiv
[1] Omer Bar-Tal, Lior Yariv, Yaron Lipman, and Tali Dekel.
preprint arXiv:2502.06608, 2025. 2
Multidiffusion: Fusing diffusion paths for controlled image
generation. In ICML, 2023. 2, 3 [19] Liqiang Lin, Yilin Liu, Yue Hu, Xingguang Yan, Ke Xie, and
Hui Huang. Capturing, reconstructing, and simulating: the
[2] Hanke Chen, Yuan Liu, and Minchen Li. Trellisworld:
urbanscene3d dataset. In ECCV, 2022. 6, 7, 21, 24, 25
Training-free world generation from object generators. arXiv
preprint arXiv:2510.23880, 2025. 13 [20] Mingbao Lin, Zhihang Lin, Wengyi Zhan, Liujuan Cao,
and Rongrong Ji. Cutdiffusion: A simple, fast, cheap,
[3] Jasmine Collins, Shubham Goel, Kenan Deng, Achleshwar
and strong diffusion extrapolation method. arXiv preprint
Luthra, Leon Xu, Erhan Gundogdu, Xi Zhang, Tomas F Yago
arXiv:2404.15141, 2024. 2, 3, 5
Vicente, Thomas Dideriksen, Himanshu Arora, et al. Abo:
Dataset and benchmarks for real-world 3d object understand- [21] Zhihang Lin, Mingbao Lin, Meng Zhao, and Rongrong Ji.
ing. In CVPR, 2022. 2 Accdiffusion : An accurate method for higher-resolution im-
[4] Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, age generation. In ECCV, 2024. 2, 3, 5, 11
Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana [22] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maxi-
Ehsani, Aniruddha Kembhavi, and Ali Farhadi. Objaverse: milian Nickel, and Matt Le. Flow matching for generative
A universe of annotated 3d objects. In CVPR, 2023. 2 modeling. In ICLR, 2023. 2, 3
[5] Prafulla Dhariwal and Alexander Nichol. Diffusion models [23] Yuheng Liu, Xinke Li, Xueting Li, Lu Qi, Chongshou Li, and
beat gans on image synthesis. In NeurIPS, 2021. 3 Ming-Hsuan Yang. Pyramid diffusion for fine 3d large scene
[6] Ruoyi Du, Dongliang Chang, Timothy Hospedales, Yi-Zhe generation. In ECCV, 2024. 2
Song, and Zhanyu Ma. Demofusion: Democratising high- [24] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher
resolution image generation with no $$$. In CVPR, 2024. 2, Yu, Radu Timofte, and Luc Van Gool. Repaint: Inpainting
3, 5 using denoising diffusion probabilistic models. In CVPR,
[7] Paul Engstler, Aleksandar Shtedritski, Iro Laina, Christian 2022. 3
Rupprecht, and Andrea Vedaldi. Syncity: Training-free gen- [25] Grace Luo, Trevor Darrell, Oliver Wang, Dan B Goldman,
eration of 3d worlds. In ICCV, 2025. 2, 3, 7, 11, 20 and Aleksander Holynski. Readout guidance: Learning con-
[8] Huan Fu, Rongfei Jia, Lin Gao, Mingming Gong, Binqiang trol from diffusion features. In CVPR, 2024. 3, 6
Zhao, Steve Maybank, and Dacheng Tao. 3d-future: 3d fur- [26] Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li.
niture shape with texture. IJCV, 129(12):3313–3337, 2021. Hpsv3: Towards wide-spectrum human preference score. In
2 ICCV, 2025. 7
[9] Google. Google earth. [Link] [27] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jia-
web, 2025. 1, 6, 22, 23, 24 jun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided
[10] Yingqing He, Shaoshu Yang, Haoxin Chen, Xiaodong Cun, image synthesis and editing with stochastic differential equa-
Menghan Xia, Yong Zhang, Xintao Wang, Ran He, Qifeng tions. In ICLR, 2022. 2, 3

9
[28] Quan Meng, Lei Li, Matthias Nießner, and Angela Dai. Yang. Moge-2: Accurate monocular geometry with metric
Lt3sd: Latent trees for 3d scene diffusion. In CVPR, 2025. 2 scale and sharp details. In NeurIPS, 2025. 2, 5, 6
[29] Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, [40] Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli.
Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. Nerf: Image quality assessment: from error visibility to structural
Representing scenes as neural radiance fields for view syn- similarity. IEEE Transactions on Image Processing, 13(4):
thesis. In ECCV, 2020. 2 600–612, 2004. 6
[30] OpenAI. Gpt-4 technical report. arXiv preprint [41] Joey Wilson, Jingyu Song, Yuewei Fu, Arthur Zhang, An-
arXiv:2303.08774, 2023. 6, 7, 14, 15, 16, 17, 18, 19, 20, drew Capodieci, Paramsothy Jayakumar, Kira Barton, and
21, 24 Maani Ghaffari. Motionsc: Data set and network for real-
[31] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. time semantic mapping in dynamic environments. RA-L, 7
Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, (3):8439–8446, 2022. 6, 14, 24
Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, [42] Haoning Wu, Shaocheng Shen, Qiang Hu, Xiaoyun Zhang,
Mido Assran, Nicolas Ballas, Wojciech Galuba, Russell Ya Zhang, and Yanfeng Wang. Megafusion: Extend diffusion
Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael models towards higher-resolution image generation without
Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Herve Je- further tuning. In WACV, 2025. 2, 5
gou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr [43] Zhennan Wu, Yang Li, Han Yan, Taizhang Shang, Weixuan
Bojanowski. DINOv2: Learning robust visual features with- Sun, Senbo Wang, Ruikai Cui, Weizhe Liu, Hiroyuki Sato,
out supervision. TMLR, 2024. 3 Hongdong Li, and Pan Ji. Blockfusion: Expandable 3d scene
[32] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya generation using latent tri-plane extrapolation. TOG, 43(4),
Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, 2024. 2
Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- [44] Jianfeng Xiang, Zelong Lv, Sicheng Xu, Yu Deng, Ruicheng
ing transferable visual models from natural language super- Wang, Bowen Zhang, Dong Chen, Xin Tong, and Jiaolong
vision. In ICML, 2021. 7 Yang. Structured 3d latents for scalable and versatile 3d gen-
[33] Robin Rombach, Andreas Blattmann, Dominik Lorenz, eration. In CVPR, 2025. 2, 3, 4, 6, 11
Patrick Esser, and Björn Ommer. High-resolution image syn- [45] Biao Zhang, Jiapeng Tang, Matthias Niessner, and Peter
thesis with latent diffusion models. In CVPR, 2022. 2, 3 Wonka. 3dshape2vecset: A 3d shape representation for neu-
[34] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denois- ral fields and generative diffusion models. TOG, 42(4):1–16,
ing diffusion implicit models. In ICLR, 2021. 3 2023. 2
[35] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Ab- [46] Richard Zhang, Phillip Isola, Alexei Efros, Eli Shechtman,
hishek Kumar, Stefano Ermon, and Ben Poole. Score-based and Oliver Wang. The unreasonable effectiveness of deep
generative modeling through stochastic differential equa- features as a perceptual metric. In CVPR, 2018. 6
tions. In ICLR, 2021. 4 [47] Zibo Zhao, Zeqiang Lai, Qingxiang Lin, Yunfei Zhao,
[36] Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Haolin Liu, Shuhui Yang, Yifei Feng, Mingxin Yang, Sheng
Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- Zhang, Xianghui Yang, et al. Hunyuan3d 2.0: Scaling diffu-
iao Yang, Jianyuan Zeng, Jiayu Wang, Jingfeng Zhang, Jin- sion models for high resolution textured 3d assets generation.
gren Zhou, Jinkai Wang, Jixuan Chen, Kai Zhu, Kang Zhao, arXiv preprint arXiv:2501.12202, 2025. 2, 6, 11
Keyu Yan, Lianghua Huang, Mengyang Feng, Ningyi Zhang, [48] Kaizhi Zheng, Yue Fan, Jing Gu, Zishuo Xu, Xuehai He, and
Pandeng Li, Pingyu Wu, Ruihang Chu, Ruili Feng, Shiwei Xin Eric Wang. Self-evolving 3d scene generation from a
Zhang, Siyang Sun, Tao Fang, Tianxing Wang, Tianyi Gui, single image. arXiv preprint arXiv:2512.08905, 2025. 3, 6,
Tingyu Weng, Tong Shen, Wei Lin, Wei Wang, Wei Wang, 11, 21
Wenmeng Zhou, Wente Wang, Wenting Shen, Wenyuan Yu, [49] Kaizhi Zheng, Ruijian Zhang, Jing Gu, Jie Yang, and
Xianzhong Shi, Xiaoming Huang, Xin Xu, Yan Kou, Yangyu Xin Eric Wang. Constructing a 3d town from a single im-
Lv, Yifei Li, Yijing Liu, Yiming Wang, Yingya Zhang, Yi- age. arXiv preprint arXiv:2505.15765, 2025. 2, 3, 5
tong Huang, Yong Li, You Wu, Yu Liu, Yulin Pan, Yun
Zheng, Yuntao Hong, Yupeng Shi, Yutong Feng, Zeyinzi
Jiang, Zhen Han, Zhi-Fan Wu, and Ziyu Liu. Wan: Open
and advanced large-scale video generative models. arXiv
preprint arXiv:2503.20314, 2025. 3
[37] Jianyuan Wang, Minghao Chen, Nikita Karaev, Andrea
Vedaldi, Christian Rupprecht, and David Novotny. Vggt: Vi-
sual geometry grounded transformer. In CVPR, 2025. 3
[38] Ruicheng Wang, Sicheng Xu, Cassie Dai, Jianfeng Xiang, Yu
Deng, Xin Tong, and Jiaolong Yang. Moge: Unlocking ac-
curate monocular geometry estimation for open-domain im-
ages with optimal training supervision. In CVPR, 2025. 5
[39] Ruicheng Wang, Sicheng Xu, Yue Dong, Yu Deng, Jianfeng
Xiang, Zelong Lv, Guangzhong Sun, Xin Tong, and Jiaolong

10
 Extend3D: Town-Scale 3D Generation
Supplementary Material
A. Appendix area corresponding to the given pixel is in the patch (i, j)
since the structure was initialized with the depth estimator.
A.1. Algorithms We therefore define the image patch ψi,j (I) by collecting
all Ix,y where corresponding qx,y ∈ Wi,j , setting the other
Algorithm 1 Sparse Structure Generation
pixels to be black, and cropping out black regions so that
1: Input: I, P the image becomes square.
2:
3: O0 ← 1P A.3. Dilated Sampling
4: Define schedule
We follow the recipe for dilated sampling in [21]. In this
5: [tstart = t1 > ... > tk = 0]
section, we assume that the unextended latent shape is
6: for 0 ≤ n < niter do
(g) K × K × K for the sake of generalization. We divide the
7: Z0 ← E(On )
extended latent into K × K non-overlapping patches so that
8: Sample ϵ ∼ N (0, I)
(g)
the size of each patch is a × b × K. We then randomly sam-
9: Zt1 ← (1 − tnoise ) · Z0 + tnoise · ϵ ple a pillar of 3D latent in each patch. The sampled pillars
10: for 1 ≤ m < k do are attached by maintaining their relative positions to be a
11: Initialize v̂ ← v(Ztm , I, tm ) K ×K ×K shaped latent. We sample a×b samples without
12: Optimize v̂ with Adam replacement, and we call them dilated samples. The dilated
13: Ztm+1 ← Ztm + (tm+1 − tm ) · v̂ samples are passed through the pre-trained model with the
14: end for image condition CI without image patchification. When di-
15: On+1 ← (D(Z0 ) > 0) lated sampling is applied, Eq. (8) is altered to be:
16: end for
17: return {p : (Oniter )p > 0} \begin {aligned} \efvec (\elatent _t, \image , t)=~& (1-\gamma _t) \mathrm {PatchWise}(\elatent _t, C_\image ,t) \\ &+ \gamma _t\mathrm {Dilated}(\elatent _t, C_\image ,t) , \end {aligned}
(14)
Algorithm 2 Structured Latent Generation
1: Input: {pi }, I, P
2:
\gamma _t=0.5\cos ^\alpha (\pi -\pi t)+0.5, (15)
3: Initialize Z1 with {pi } where PatchWise is equal to Eq. (8) and α is a hyperpa-
4: Define schedule [1 = t1 > ... > tk = 0] rameter. We set α = 5 in our experiments. We use dilated
5: for 1 ≤ m < k do sampling only for sparse structure generation because we
6: Initialize v̂ ← v(Ztm , I, tm ) empirically observed that it worsens texture when applied
7: Optimize v̂ with Adam to structured latent generation.
8: Ztm+1 ← Ztm + (tm+1 − tm ) · v̂
9: end for A.4. Computational Cost
10: return Z0

Algorithm 3 Extend3D Table 7. Computational costs of 3D generation methods.

(a = b = 2, d = 4, niter = 5)
1: Input: I
2: VRAM (GB) Time (min)
3: P, P ← M oGe2(I) Trellis [44] 17 0.06
4: {pi } ← SS(I, P) Hunyuan3D [47] 56 1.78
5: Z ← SL AT ({pi }, I, P ) SynCity [7] 49 52.0
6: return (DGS (Z), Dmesh (Z)) EvoScene [48] 68 35.3
Ours 28 14.1

A.2. Image Patchification We compared the computational cost of our method with
We precisely patchify an input image using point cloud co- previous methods in Tab. 7, using an NVIDIA RTX Pro
ordinates extracted from the monocular depth estimator. We 6000. Although our pipeline is heavier than object-centric
can map a pixel Ix,y to a coordinate of the point cloud qx,y methods (Trellis or Hunyuan3D), it requires substantially
in the extended latent space. When qx,y ∈ Wi,j , the 3D less memory and time than other scene-level pipelines.

11
Memory cost. For a = b = 2 and d = 4 case, the peak GPU Table 8. Ablation study on varying number of iterations niter .
memory used for our method is 28GB. Most of the mem-
ory is utilized in SL AT optimization process. Without SL AT CD ↓ F-score (0.05) ↑
optimization, it took 14GB VRAM. Also, for a = b = 6 niter = 0 0.0079 0.647
and d = 4, without the SL AT optimization, the peak mem- niter = 1 0.0075 0.681
ory was 61 GB. Table 2 shows that our Extend3D achieves niter = 2 0.0077 0.688
promising results even without SL AT optimization. There- niter = 3 0.0082 0.677
fore, we can ensure reasonable quality with limited GPU
resources.
Inference time. The computational complexity of our
method is O(abd2 niter ). A larger (larger a and b), better
detailed (larger d), and more complete (larger niter ) scene
requires more inference time, so there is a trade-off between
output quality and inference time.
𝑛!"#$ = 1 𝑛!"#$ = 5
A.5. Rigorous Definition
Figure 8. Qualitative effect of iterative SDEdit
Here, we provide a detailed and rigorous definition of ϕSS
i,j ,
SL AT SS −1 SL AT −1
ϕi,j , (ϕi,j ) , and (ϕi,j ) for clarity. Firstly, we define
patchification ϕi,j through sliding windows as: that our method can address incompleteness in depth esti-
mation. This also implies that our method is robust to the
\sliding _{i, j}^K=\left [\frac {iK}{d},\frac {iK}{d}+K\right )\times \left [\frac {jK}{d},\frac {jK}{d}+K\right )\times [K], (16) minor errors of the depth estimator. While a single SDEdit
step improved results, additional SDEdit steps sometimes
degraded output quality, as shown in Tab. 8. Since the qual-
\patch _{i,j}^\mathrm {SS}(\elatent _t^\mathrm {SS})=(\elatent _t^\mathrm {SS})_{\sliding _{i, j}^N} \in \mathbb {R}^{N\times N\times N}, (17) itative results, such as Figure 8, require iterations to com-
plete the geometry, there is a trade-off between scene com-
pletion and geometric detail.

\label {slat_patch_sampling} \begin {aligned} \patch _{i,j}^\mathrm {\SLat }\!\left (\elatent _t^\mathrm {\SLat }\right ) &= \Biggl \{ \Bigl ( \pos - \Bigl (\tfrac {iM}{d},\, \tfrac {jM}{d},\, 0\Bigr ), \lat _t \Bigr ) :\\ &\qquad (\pos , \lat _t) \in \elatent _t^\mathrm {\SLat },\; \pos \in \sliding _{i, j}^M \Biggr \}. \end {aligned} A.7. More Results
(18) We provide additional comparisons and large scene recon-
struction results in Fig. 9, Fig. 10, Fig. 11, Fig. 12, Fig. 13,
Fig. 14, Fig. 15, Fig. 16, Fig. 17, and Fig. 18.

The subtraction in Eq. (18) is for coordinate normalization A.8. Quantitative Evaluation Set
into [M ]3 . Secondly, the inversions of these processes are
The evaluation sets used for appearance evaluation (LPIPS,
defined as:
SSIM, PSNR) and for geometry evaluation (CD, F-score)
(\patch _{i,j}^\mathrm {SS})^{-1}(\mX )_{x,y,z}=\mathbf {1}_{(x,y,z)\in \sliding ^N_{i, j}}\cdot \mX _{x-\frac {iN}{d},y-\frac {jN}{d}, z}, (19) are illustrated in Fig. 19 and Fig. 20, respectively. We used
the keywords from the original SynCity research for a text-
prompted generation comparison. The keyword list is as
follows: autumn, campsite, campus, cyberpunk, mars, me-
dieval market, medieval, oasis, paris, SF, solarpunk, and
\begin {aligned} (\patch _{i,j}^\mathrm {\SLat })^{-1}(\mX ) &= \Biggl ( \mX + \Bigl ( \Bigl (\tfrac {iM}{d},\, \tfrac {jM}{d},\, 0\Bigr ), \bm 0 \Bigr ) \Biggr )\\ &\quad \cup \; \Bigl \{ (\pos , \bm 0) : \pos \in \{\pos _i\},\; \forall \lat ~ (\pos , \lat ) \notin \mX \Bigr \}, \end {aligned}
suburban. Given a keyword, we used a prompt below for
image generation before Extend3D:

(20) Create a realistic, high-resolution aerial (bird’s-

setting the value of other positions to zero. eye) view of {keyword}. The image should be per-
fectly framed so that no buildings, objects, or
A.6. Ablation on Iterative SDEdit landscape features are cropped or cut off. Show
We evaluated geometric results of different values of niter . the entire scene clearly from directly above, with
The result with niter = 0 represents the decoded SL AT natural lighting, realistic depth, and detailed tex-
with geometry from the monocular depth estimator. Com- tures. The result should look like a professional
pared to the niter = 0 case, with a single under-noised drone photograph taken at high altitude, with bal-
SDEdit step, the geometry improved noticeably, indicating anced composition and clean edges.

12
A.9. SL AT Decoding
While our main contribution is on the latent generation and
most of our results in the paper are represented in 3D Gaus-
sian Splatting, we also introduce an overlapping-patch SDF
representation for mesh decoding. Unlike 3D Gaussians, if
mesh patches are decoded disjointly and simply attached,
the seams will be discontinuous, because there will be no
faces. Therefore, similar to the overlapping patch-wise flow
in our paper, we first decode SL AT into SDFs in an overlap-
ping patch-by-patch manner. These SDF patches are then
combined into a single SDF by averaging the values in the
overlapping regions. Inspired by TRELLISWorld [2], we
apply a cosine-based weighting to avoid near-zero SDF val-
ues at the seams. Finally, we extract the mesh from the com-
bined SDF in a single step.
Unlike meshes, we decode SL AT into 3D Gaussians with
disjoint SL AT patches, since it is not necessary to consider
connectivity in 3D Gaussians.

13
 Ours EvoScene

Trellis Hunyuan3D-2.1

Ours EvoScene

Trellis Hunyuan3D-2.1

Figure 9. Example results for diverse images. In this figure, we set a = b = 2. We compared our results with state-of-the-art open-source
3D generative models. The second image is from CarlaSC dataset [41]. The other image is generated by ChatGPT [30].

14
 Ours EvoScene

Trellis Hunyuan3D-2.1

Ours EvoScene

Trellis Hunyuan3D-2.1

Figure 10. Example results for diverse images. In this figure, we set a = b = 2. We compared our results with state-of-the-art open-
source 3D generative models. The images are generated by ChatGPT [30] and Flux.1 [dev] [15].

15
 Ours EvoScene

Trellis Hunyuan3D-2.1

Ours EvoScene

Trellis Hunyuan3D-2.1

Figure 11. Example results for diverse images. In this figure, we set a = b = 2. We compared our results with state-of-the-art open-
source 3D generative models. The images are generated by ChatGPT [30].

16
 Ours EvoScene

Trellis Hunyuan3D-2.1

Ours EvoScene

Trellis Hunyuan3D-2.1

Figure 12. Example results for diverse images. In this figure, we set a = b = 2. We compared our results with state-of-the-art open-
source 3D generative models. The images are generated by ChatGPT [30].
17
 Ours EvoScene

Trellis Hunyuan3D-2.1

Ours EvoScene

Trellis Hunyuan3D-2.1

Figure 13. Example results for diverse images. In this figure, we set a = b = 2. We compared our results with state-of-the-art open-
source 3D generative models. The images are generated by ChatGPT [30].

18
 Ours EvoScene

Trellis Hunyuan3D-2.1

Ours EvoScene

Trellis Hunyuan3D-2.1

Figure 14. Example results for diverse images. In this figure, we set a = b = 2. We compared our results with state-of-the-art open-
source 3D generative models. The images are generated by ChatGPT [30].

19
 Autumn

SF

Oasis

Solarpunk

Ours SynCity

Figure 15. Example results of comparison to SynCity. We compared our results with SynCity [7]. Given a prompt, we first generated an
image with ChatGPT [30] and used Extend3D for our results. The input prompts are on the left.

20
 Ours EvoScene

Figure 16. Example results of comparison to EvoScene [48]. In this figure, we set a = b = 2. The images are from ChatGPT [30] and
UrbanScene3D [19].

21
Figure 17. The large scale result of Extend3D. We generated large scale (a = b = 6) 3D scene from the image of Köln captured from
Google Earth [9]. We didn’t use SL AT optimization in this result due to the memory shortage.

22
Figure 18. The large scale result of Extend3D. We generated large scale (a = b = 6) 3D scene from the image of Athens captured from
Google Earth [9]. We didn’t use SL AT optimization in this result due to the memory shortage.

23
Figure 19. 100 images for appearance evaluation. The images are from ChatGPT [30], Flux.1 [dev] [15], CarlaSC [41], Google Earth [9],
and UrbanScene3D [19].

24
Figure 20. 45 images for geometry evaluation. The images are from UrbanScene3D [19] synthetic scenes, together with ground truth
meshes.

25