Completely newbie questions from someone outside the field who hasn't been following closely:
Is the spatial memory a fixed size (how big?) or does it grow over time?
And is there a point at which is is saturated and future results decline?
Very interesting stuff. I wonder how this one camera (one viewpoint), flat images models work in completely novel environments (not seen in training data). I am wondering if this model could be used with stereo cameras as is.
Can this result in a colmap dataset that can be used by Gaussian Splatting generation?
There would not be much point. Colmap is already very capable in reconstructing a 3D scene from images from unknown poses if you have the camera intrinsics.
Besides processing speed, this project (and the underlying dust3r model) strength is that it works with very few images. You basically just need 2, and it can infer pseudo instrinsics and matching extrinsics on it's own.
I don't see why it could not be adapted to output gaussian splats instead. As a matter of fact, it's already been done with dust3r: https://github.com/nerlfield/wild-gaussian-splatting.