Your parents probably owned a globe. A globe takes a large and complex concept, and projects it onto an intuitive artifact that took up space in their home. Similarly, table-top figurines and objects were used to plan battles and grasp spatial issues. By taking up space, these objects took up a part of our lives, and at a glance opened up our thoughts.
The news is a complex and constantly updating stream of information with strong spatial and temporal dimensions. Innovations in AR glasses and phones allow for spatial interfaces which allow us to place information and enterntainment objects in the physical world. These same interfaces can be utilized to display news in a novel and engaging way, which we call Holographic News. Holographic news, as compared to web-pages viewed on a laptop and on a phone, have the advantage of being spatial, time persistent, and highly interactive.
We utilize the spatial aspect to shrink a story onto the table-top, giving the user more perspective.
Contributions
volumetric video algorithm from stereo-panoramic cameras
Editing studio software for table-top news expriencesInteractive AR Widgets:
Interactive AR widgets that track a story that have updates dailyEditing Software:
Volumetric Video Capture:
VR video today is really just panoramic video. If one had a photogrammetric model of the world, they would be able to view a video from different perspectives, digitally manipulate the captured world, and shrink it down to a table-top size. Stereo panoramic cameras are becoming more and more popular, however I have yet to see a good implementation of turning the stereo panoramic video into a 3D model.
I’ve impelmented a simple stereo-disparity algorithm for stereo panoramic videos that generates a model. I hope to improve this algorithm with deep learning for my Computer Vision class project.
Here is an example model generated by processing stereo panoramic video from the Vuze stereo panoramic camera:
Here is an example of basketball video generated from 2D video processed with simple background subtraction / player position estimation:
Editing
Once the 3D model data is captured, it can be combined with other sources of data (video-clips), annotated, and narrated. This can be done using an AR/VR system like a mobile phone or an HTC Vive. We’ve prototyped editing studios in VR. For the thesis, I’d like to make a system which makes the process very easy.
Example of VR broadcast editing studio:
AR Interactive Widgets
The consumer end of spatial news is an interactive 3D expereince/video which the user can explore and play with. The experience will allow a user to explore the news in both space and time. Furthermore, the widget can be placed in a persistent location in the user’s world. This enables the user to take an issue they really care about and track it daily. By having the news topic take up room in the user’s physical space, it may lead to a bigger attention span for a story.
Example of a global news widget using Mobile AR and reuters data:
Example of a place widget, with google news from Syria:
Example of a place widget, with google news from the virgin islands:
Abstract topics
Often news revolves around abstract concepts, such as a healthcare bill. Interactive widgets can still be made that can represent what is going on in Congress. An example of that would be a house of representatives widget sitting on a car dashboard:
Evaluation
Short term explorational study:
Complete an A/B user test wherein some of the participants are given a standard 2D news-feed, and some are given the ability to explore the news in a spatial way. Both interfaces will be instrumented and the behavior of the users tracked. We can then observe the differences in clicking behavior / exploration between the different groups of users.
5-10 participants
Long term study:
Control group is asked to visit a website everyday for a week
Test group has a news widget installed on their desk, asked to access the app everyday for a week
Informational exam is administered at the end of the week (user knows they will be surveyed but not what the nature of the questions will be)
Results will evaluate whether the format facilitates news exploration/comprehension of a developing story
Creative Commons Attribution 4.0 International License (CC-BY 4.0)