Dev/temporal bundle #2030

CedricThebault · 2025-12-05T07:31:09Z

This pull request introduces a temporal smoothness constraint to the bundle adjustment process in the Structure-from-Motion (SfM) pipeline. The main changes add new parameters for temporal constraints, update the bundle adjustment implementation to support these constraints, and provide more granular evaluation and debugging capabilities. The changes also include a minor addition of a random noise variance option per view in track simulation.

Key changes:

Temporal Smoothness Constraint Integration

Added a new TemporalConstraintParams struct and corresponding parameters (e.g., position/ orientation weights, regularization weights) to configure temporal smoothness constraints in bundle adjustment (BundleAdjustment.hpp, BundleAdjustmentCeres.hpp). [1] [2]
Introduced a new refinement option flag REFINE_TEMPORAL_SMOOTHNESS_CONSTRAINT to enable temporal constraints during optimization (BundleAdjustment.hpp).
Implemented the logic to add temporal smoothness constraints as residual blocks in the Ceres problem, including support for regularization and evaluation of these constraints separately (BundleAdjustmentCeres.cpp). [1] [2] [3] [4] [5] [6]

User Interface and Parameterization

Extended the SfMExpanding node with new parameters to control temporal constraints (e.g., enable/disable, weights for position/orientation, regularization, and debug option to exit after pose interpolation). These parameters are only enabled if temporal constraints are active (SfmExpanding.py).

Evaluation and Debugging Enhancements

Modified the bundle adjustment process to separately track and log costs for landmark and temporal constraint residual blocks, aiding in debugging and analysis (BundleAdjustmentCeres.cpp). [1] [2]

Track Simulation

Added an advanced parameter to optionally use random noise variance per view in the TracksSimulating node (TracksSimulating.py).

Infrastructure and Minor Fixes

Updated function signatures and internal logic to pass and manage residual block IDs for more granular control over Ceres problem components (BundleAdjustmentCeres.cpp). [1] [2] [3] [4] [5]
Included necessary headers for temporal constraints and pose filtering (BundleAdjustmentCeres.cpp).

These changes collectively enable temporal regularization in SfM bundle adjustment, providing greater flexibility and control for handling sequential data, such as video frames, and improving the robustness and quality of the resulting reconstructions.

Copilot

Pull request overview

This pull request introduces temporal smoothness constraints to the bundle adjustment process in the Structure-from-Motion (SfM) pipeline, enabling better handling of sequential data such as video frames. The implementation adds new cost functions that constrain camera pose trajectories to be smooth over time, along with configuration parameters and debugging capabilities.

Key changes:

Adds temporal constraint cost functions supporting C0 (continuity), C1 (velocity), and C2 (acceleration) smoothness with configurable weights
Implements pose interpolation to fill missing poses using temporal filtering before bundle adjustment
Extends the SfM expanding node with new UI parameters for temporal constraint configuration
Adds separate cost evaluation and logging for landmark and temporal constraint residuals

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 24 comments.

Show a summary per file

File	Description
`src/aliceVision/sfm/bundle/costfunctions/temporalConstraint.hpp`	New file implementing temporal constraint cost functors and regularization terms
`src/aliceVision/sfm/bundle/BundleAdjustment.hpp`	Adds `TemporalConstraintParams` struct and `REFINE_TEMPORAL_SMOOTHNESS_CONSTRAINT` flag
`src/aliceVision/sfm/bundle/BundleAdjustmentCeres.hpp`	Updates function signatures to support temporal constraints and separate residual block tracking
`src/aliceVision/sfm/bundle/BundleAdjustmentCeres.cpp`	Implements temporal constraint integration, residual block ID tracking, and separate cost evaluation
`src/aliceVision/sfm/pipeline/expanding/SfmBundle.hpp`	Adds temporal constraint parameter storage and configuration methods
`src/aliceVision/sfm/pipeline/expanding/SfmBundle.cpp`	Integrates pose interpolation and temporal constraints into the bundle adjustment workflow
`src/aliceVision/sfm/utils/poseFilter.hpp`	Adds `interpolateMissingPoses` method and updates filter signature
`src/aliceVision/sfm/utils/poseFilter.cpp`	Implements pose interpolation and helper functions for ordered pose retrieval
`src/software/pipeline/main_sfmExpanding.cpp`	Adds CLI parameters for temporal constraint configuration
`src/software/utils/main_tracksSimulating.cpp`	Adds random noise variance per view option for track simulation
`src/aliceVision/sfmDataIO/sceneSample.cpp`	Updates sphere scene generation with observations and modified focal lengths
`meshroom/aliceVision/SfmExpanding.py`	Adds UI parameters for temporal constraint weights and debugging options
`meshroom/aliceVision/TracksSimulating.py`	Adds UI parameter for random noise variance per view

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-05T07:39:27Z

meshroom/aliceVision/SfmExpanding.py

+        desc.FloatParam(
+            name="tscC2positionWeight",
+            label="Temporal Constraint C2 Position Weight",
+            description="",


Empty description field for user-facing parameter. Add a meaningful description explaining what C2 (second derivative/acceleration) position weight controls.

Suggested change

description="",

description="Controls the influence of the second derivative (acceleration) of camera position in temporal constraints. Higher values enforce smoother changes in acceleration between consecutive frames.",

src/aliceVision/sfm/utils/poseFilter.cpp

Copilot · 2025-12-05T07:39:28Z

src/aliceVision/sfmDataIO/sceneSample.cpp

+                continue;
+            }
+
+            std::cout << point3D.transpose() << " -- " << landmark.X.transpose() << std::endl;


Debug std::cout statement should be removed. This appears to be leftover debug code that prints landmark coordinates for every landmark and view combination, which could produce excessive output for large scenes (1000 points × 240 poses = 240,000 lines). Either remove this line or replace it with a conditional debug log.

Suggested change

std::cout << point3D.transpose() << " -- " << landmark.X.transpose() << std::endl;

Copilot · 2025-12-05T07:39:29Z

src/aliceVision/sfm/bundle/BundleAdjustmentCeres.cpp

+        ALICEVISION_LOG_WARNING("Unable to get ordered frameIDs.");
+        throw std::runtime_error("Unable to get ordered frameIDs.");


Inconsistent error messages. Line 834 logs "Unable to get ordered frameIDs" as a warning, and then line 835 throws an exception with the same message. Either the warning is redundant (if we're throwing anyway), or the message should be more specific. Consider:

ALICEVISION_LOG_ERROR("Unable to get ordered frameIDs for temporal constraints."); throw std::runtime_error("Unable to get ordered frameIDs for temporal constraints.");

Suggested change

ALICEVISION_LOG_WARNING("Unable to get ordered frameIDs.");

throw std::runtime_error("Unable to get ordered frameIDs.");

ALICEVISION_LOG_ERROR("Unable to get ordered frameIDs for temporal constraints.");

throw std::runtime_error("Unable to get ordered frameIDs for temporal constraints.");

src/aliceVision/sfm/bundle/BundleAdjustmentCeres.cpp

Copilot · 2025-12-05T07:39:33Z

meshroom/aliceVision/SfmExpanding.py

+        desc.FloatParam(
+            name="tscC0orientationWeight",
+            label="Temporal Constraint C0 Orientation Weight",
+            description="",


Empty description field for user-facing parameter. Add a meaningful description explaining what C0 (continuity) orientation weight controls.

Suggested change

description="",

description="Weight for enforcing C0 (continuity) in camera orientation between consecutive frames. Higher values increase the smoothness of orientation transitions.",

Copilot · 2025-12-05T07:39:34Z

meshroom/aliceVision/SfmExpanding.py

+        desc.FloatParam(
+            name="tscC1orientationWeight",
+            label="Temporal Constraint C1 Orientation Weight",
+            description="",


Empty description field for user-facing parameter. Add a meaningful description explaining what C1 (first derivative/velocity) orientation weight controls.

Suggested change

description="",

description="Weight for enforcing smoothness of camera orientation changes (first derivative/velocity) over time.",

Copilot · 2025-12-05T07:39:34Z

meshroom/aliceVision/SfmExpanding.py

+        desc.FloatParam(
+            name="tscC2orientationWeight",
+            label="Temporal Constraint C2 Orientation Weight",
+            description="",


Empty description field for user-facing parameter. Add a meaningful description explaining what C2 (second derivative/acceleration) orientation weight controls.

Suggested change

description="",

description="Weight for the temporal constraint on the second derivative (acceleration) of camera orientation. Higher values enforce smoother changes in orientation over time, reducing abrupt rotational accelerations between consecutive frames.",

Copilot · 2025-12-05T07:39:34Z

src/aliceVision/sfmDataIO/sceneSample.cpp

+            std::shared_ptr<camera::Pinhole> camPinHole = std::dynamic_pointer_cast<camera::Pinhole>(cam);
+            if (!camPinHole)
+            {
+                std::cout << "Camera is not pinhole in getInputRigScene" << std::endl;


Error message contains incorrect function name. The message says "getInputRigScene" but this is in the function generateSphereScene. The error message should be:

std::cout << "Camera is not pinhole in generateSphereScene" << std::endl;

Also consider using ALICEVISION_LOG_ERROR for consistency with the rest of the codebase instead of std::cout.

Copilot · 2025-12-05T07:39:34Z

meshroom/aliceVision/SfmExpanding.py

+        desc.FloatParam(
+            name="tscPositionWeight",
+            label="Temporal Constraint Position Weight",
+            description="",


Empty description field for user-facing parameter. All the temporal constraint weight parameters (lines 89, 98, 107, 116, 125, 134, 143, 152, 161) have empty descriptions. Users will not understand what these parameters control without documentation. Add meaningful descriptions explaining what each weight controls (C0 = continuity, C1 = velocity, C2 = acceleration constraints).

CedricThebault added 3 commits December 4, 2025 17:40

Temporal Constraint added to bundle adjustment

fd9b7a6

Add observations to sceneSample

fa1e3b1

Add noise variance per view in tracksSimulating

db0dc09

CedricThebault requested review from Copilot and servantftransperfect December 5, 2025 07:31

Copilot started reviewing on behalf of CedricThebault December 5, 2025 07:31 View session

Copilot finished reviewing on behalf of CedricThebault December 5, 2025 07:35

Copilot AI reviewed Dec 5, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

Dev/temporal bundle #2030

Dev/temporal bundle #2030

Uh oh!

CedricThebault commented Dec 5, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Copilot AI Dec 5, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	description="",
	description="Controls the influence of the second derivative (acceleration) of camera position in temporal constraints. Higher values enforce smoother changes in acceleration between consecutive frames.",

		ALICEVISION_LOG_WARNING("Unable to get ordered frameIDs.");
		throw std::runtime_error("Unable to get ordered frameIDs.");

	description="",
	description="Weight for enforcing C0 (continuity) in camera orientation between consecutive frames. Higher values increase the smoothness of orientation transitions.",

	description="",
	description="Weight for enforcing smoothness of camera orientation changes (first derivative/velocity) over time.",

	description="",
	description="Weight for the temporal constraint on the second derivative (acceleration) of camera orientation. Higher values enforce smoother changes in orientation over time, reducing abrupt rotational accelerations between consecutive frames.",

Uh oh!

Dev/temporal bundle #2030

Are you sure you want to change the base?

Dev/temporal bundle #2030

Uh oh!

Conversation

CedricThebault commented Dec 5, 2025

Temporal Smoothness Constraint Integration

User Interface and Parameterization

Evaluation and Debugging Enhancements

Track Simulation

Infrastructure and Minor Fixes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 5, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants