Interesting! The example shows two images that appear to have a pixel-perfect matching region. Is that a requirement or does it work with images that are only somewhat similar?
seems to be doing some mean-square error to find best matching region.
Related to this, is there a name for the effect when you stitch together video frames into a static background while keeping the moving objects moving? The best example I can think of is this Bigfoot video[0, 1], where the shaky footage has been combined into a bigger canvas with "Bigfoot" moving through it. It's a combination of video stabilization and image panorama, but with some smarts to only keep one version of the moving object in each finished frame.
[0] https://www.youtube.com/watch?v=Q60mSMmhTZU [1] https://x.com/rowancheung/status/1641519493447819268
Interesting to see something like this!
My computer science masters thesis was based on the same goal. I used a 2D convolution which meant you can merge images with inexact overlaps. I had to run a high-pass filter first to limit the image details to their edges only or else the convolution incorrectly matched bright areas.
In reality merging pictures is further complicated because the source images may be slightly rotated relative to each other and also due to the images being slightly curved due to lens distortion.
My supervisor wanted me to do a PHD on the topic!
Thank you for your comments! For sure, the CNN is expressive for learning the characteristics of images. However, in this development, I tried to not use deep-learning because I believe that it is important to provide fast, consistent results without the need for training data. If you are particularly interested in this app, I would be glad if you could create a pull request to extend the algorithm.
Nicely done and keep up the practice. I recall during my Masters needing to translate facial landmark points from a Cartesian coordinate system into points that could would appear on their respective images. It wasn't for anything major, I just wanted a visual representation of my work. Its these little "neat" projects that help build larger breakthroughs.
look at those for loops! should look into fft-based correlation, can even do so with melon transform for scale and circular harmonic transform for rotation
What are the practical applications for this tool? Typically stitching images for something like panoramas requires significantly more advanced image processing algorithms because the pixels do not perfectly overlap.
Even in web browsers that support screenshotting an entire page, websites often unload elements that are off-screen. A solution like this can take a bunch of screen-length images and stitch them into a full view of the document.
Thank you for comments! Certainly, this application may not be able to handle any kinds of images. However, I tried to stitch images without using deep-learning. Therefore, the strength of this app is that when this app receives the same images, it always produces consistent results. In the future, I will try to develop a more effective image merging method in more generalized scenario.