Beautiful.
What I love about gaussian splats is the way they degrade - instead of a hard cutoff or LoD changing spheres into cubes etc., they get increasingly "dreamy" - the basic idea is still there, just less detailed.
Take for example this scene:
https://superspl.at/scene/e721ea7c
If you navigate closer to the trees, things around you become blurry - as if the very fabric of reality unraveled.
Wow this is a time killer... ended up here: https://superspl.at/scene/ff1d0393 beautiful!
I read [1], but I still don't quite know what I'm looking at. My guess is a 3D model reconstructed from lots of detailed pictures?
Lots of translucent blobs composited to produce the appearance of a strawberry.
There is no mesh or model. The visual surface of the strawberry could be made up of blobs spaced far apart physically and not where the surface appears to be.
This is why they are called radiance fields, they model the light not the geometry.
Practically the blobs positions/rotations can be constrained to better physically match the geometry of a strawberry.
I'm not sure i agree. The blobs are exactly where the surface appear to be because they are constrained by multiple viewing angles.
Otherwise the splat would fall apart as soon as the viewing angle is changed slightly (Which it absolutely does in many examples on supersplat, you cannot really create an out of distribution view with 3GS, it's not magic)
https://youtube.com/watch?v=X8yRlA7jqEQ is how I learned about them. They're really cool!
This video explains how Gaussian splatting works: https://www.youtube.com/watch?v=X8yRlA7jqEQ
My intuition is that in theory focus stacking should not be necessary as preprocessing step for 3dgs (or photogrammetry). Does anyone know if there is any recent developments in this regard?
Focus stacking generally is not perfect process and can lead to artifacts/errors and I'd imagine those can then compound when stacked images are used for 3dgs. Also the image focus actually provides some depth data in itself that could be useful?
If you don't focus stack and try to train on partially unfocused images, the optimizer will try to match the rendered view to be also partially unfocused.
You would have to mask out the blurry areas for each image.
Great work! There are more awesome splats on the author's profile page: https://superspl.at/user?id=danylyon
Someone: Please combine microscopy with gaussian splatting.
Don't know if this would be in your wheelhouse, but for very nice macro splats, check out the work by Dany Bittel: https://danybittel.ch/macro.html
For example this bumblebee: https://superspl.at/scene/cf6ac78e
Edit: I completely missed that this was posted by him (:
This was also my first thought when I zoomed in into the strawberry. I wonder if you can achieve a microscope like effect with a more suitable setup. E.g. better lighting, zoom, lens, etc.
I have done 2x macro (an ant).. and want to try 5x.. but as you get closer, the depth of field becomes really shallow. You can do focus stacking but you risk that the individual areas in focus are less ideal aligned and the tracking can't make any sense out of the geometry anymore.
Dany, this is so cool.
I'm wondering if the splat community has decided this paper is valuable -- https://github.com/fraunhoferhhi/Self-Organizing-Gaussians -- looking at all the detail in the strawberry splat made me wonder how small one can get the download, and what the current state of the art is for compression.
Thanks! We have two compressed formats, the sog by PlayCanvas and spz with sparkjs. Both now support LODs and compress really well.
Imagine if we start designing GPUs around this technology as opposed to vectors. Imagine what voxel engines would look like. Would love a simulated experience or a small scale that theorizes about this.
You might want to throw that one away :)
It is in fact still on it's mount and slowly rotting / molding.. for a second capture :-)
Yup. It's rotten on the other side. Or maybe lead poisoned
Sorry if I fell for Poe's law, but just for clarity, the strawberry's rotten underside is most likely missing splats in the rendering.
Of course it is. That's where the joke comes from...
Edit. TIL Poe's law
not sure if that's a joke but i think that's just the effect of the strawberry being placed on a glass/plastic surface to be filmed from underneath
Can you show the setup?
(Can we do a Gaussian Splat of the setup of the photograph for the Gaussian Splat of the Strawberry?)
I just posted some images.. I have actually done my studio, (quality is.. let's say painterly ;-): https://superspl.at/scene/0a3916cd
What happened to the bottom of that poor strawberry?
It was mounted at the bottom.. and I can't quite reach it with my camera. Might have to try some two pass way.
I wonder would a good, sharp needle and thread make for good mounting for a soft object like this? Thread the needle, pass it right through the strawberry and the secure the thread on something above and below. As long as the strawberry doesn't slide down the thread (hopefully a strawberry is light enough friction would hold it in place!)
Anyway, very cool splat, fair play
How about green screen + rotate the strawberry on a skewer
Mount it on a needle/skewer, it should let you capture it in one pass.
A thread (or two) running through the strawberry may be the solution. This should give it some stability while giving enough space at the bottom for the camera to fit.
Assuming that the person that did this has not tried that. If you look at the setup photos, the grape is resting on a couple of nails. This suggests that many different things have been tried.
Gaussian splat casualty. The bottom looks like partially missing from the reconstruction.
When you cut the splat in half, result is either fuzzy fog or sort-of fibrous crystals. As depicted here.
From the link: "Shot from 90 perspectives, 88 focus stacked images each. Nikon Z8, full frame, f/7.1, exposure 1/160, ISO 100, Laowa 180mm macro lens, with LED light and bluescreen." Insane!
And it only takes 20 Minutes to shoot all 7920 photos, the Z8 is crazy fast.
this is awesome, I wonder what's under there, looks black, maybe thats where they mounted and rotated the strawberry...
Yes mounting.. and I can't quite reach all the way from below.
Gaussian splats look really good from a distance, but as soon as you zoom in, they really fall off a cliff :/
Lovely! How was the mechanical setup to ensure that all those shots are consistent, and how long did it take?
I posted some pictures.. takes only 20 min!
What? KIRI Engine makes splats. I always wondered what 3DGS might mean.
Yes. I knoweth what "splats" are: They are splats of fuzzy blobs on the display surface.
