Hyperlapse processed with HUSTL
Making good hyperlapse videos is difficult. It usually requires photographers to take photos between consistent distance intervals and keep the height of the camera and the direction of the lens steady and smooth. Alternatively, they can mount their camera on expensive steadicams or gimbals. To make creating high-quality hyperlapses easier, we present HUSTL (Hyper-Ultra-Super-Time-Lapse), an open source three-stage software pipeline based on the state of the art academic research papers. Our tool allows users to only provide rough hand-held hyperlapse frames to create a smoothed hyperlapse video.
There are 3 major challenges to creating a high-quality hyperlapse video:
- Some frames can be sub-optimal and should be skipped
- Frames may have different white balance, lighting conditions, and some frames may be over-exposed or under-exposed
- Frames need to be aligned to ensure that the camera movement looks smooth
For more details, please refer to the project writeup.
Soldiers Memorial Gate
Areas Needing Improvements
As shown in the demo, even though the camera movements have been smoothed, there are still a few issues degrading the hyperlapse:
- Objects can appear to be warped unnaturally, especially in image corners and especially when the warping needed is significant.
- HUSTL struggles in narrow environments (like hallways) where objects move out of frame relatively quickly compared to an open environment.
- HUSTL may cause unnatural lighting effects and unintended brightness changes when transitioning between well-lit and dimly-lit environments.
We believe a better video stabilization algorithm (functioning like the “Warp Stabilizer” in Adobe Premiere) may be better equipped to resolve the first 2 issues. A potential method to resolve the 3rd issue may look like:
- Project the color space to a logarithmic space to reduce risk of information loss when adjusting brightness
- Adjust the images so that the brightness (average brightness) is consistent across all frames
- Run the images through the HUSTL color consistency pipeline
- Project the logarithmic space back to the normal linear color space
Neel Joshi, Wolf Kienzle, Mike Toelle, Matt Uyttendaele, and Michael F. Cohen. Real-Time Hyperlapse Creation via Optimal Frame Selection ACM Transactions on Graphics, 2015 ↩︎
Jaesik Park, Yu-Wing Tai, Sudipta Sinha, and In So Kweon. Efficient and Robust Color Consistency for Community Photo Collections Computer Vision and Pattern Recognition (CVPR), 2016 ↩︎