Raphael Thys
A virtual tour is an interactive, immersive experience that allows users to explore and experience a location or property as if they were there in person. Virtual tours are typically created using a combination of 360-degree photographs with other multimedia elements, and can be accessed using a computer, smartphone, or other device.
“A photographer with 360 camera in a hotel” generated by MidJourney AI
In recent years, AI has made significant advancements in the ability to generate high-quality, realistic images. The progress has been driven by advances in machine learning and deep learning algorithms, which have enabled AI systems to learn and adapt from large amounts of data.
AI-generated images already found their way into book and article illustrations, and social media posts. There are cases when AI-generated art won fine arts competitions. But could AI produce 360-degree images for a virtual tour? Let's give it a try by creating some 360-degree photos. I will use MidJourney and DALL-E 2 for this purpose.
MidJourney
MidJourney is one of the most popular AI-based image generators. It is able to generate an image based on a text description. My first try with it was quite straightforward:
360-degree photo in equirectangular projection
Nice try, nice little planets! But I actually need something in the equirectangular projection.
Equirectangular projection is a common method for creating 360-degree photos and is based on the idea of projecting the view of a spherical surface onto a flat plane. This projection method creates a rectangular image that shows the entire view of the spherical surface, including the top and bottom. Here is an example of such photo:
Luckily we can control the aspect ratio of photo with MidJourney with — ar switch:
360-degree photo in equirectangular projection — v 3 — ar 2:1
With the aspect ratio set to 2:1 MidJourney was able to reproduce the general idea of 360 photo structure, but the subject became too futuristic. Lets limit AI imagination to something down to earth:
360-degree photo of restaurant in equirectangular projection — v 3 — ar 2:1
360-degree photo of hotel in equirectangular projection — v 3 — ar 2:1
Still too much creativity, but AI is getting a general idea of 360-degree photo in equirectangular projection.
DALL-E 2
DALL-E 2 is an AI-based image generator from OpenAI. Let's give him the same descriptions as we provided to MidJourney.
360-degree photo in equirectangular projection
DALL-E produced much better results right out of the box! But unfortunately, it can only produce photos with a 1:1 aspect ratio. It is great for Instagram, but not for our purpose. If you want a different size you have to go to its editor called “Outpainting” and append some more generated “squares” to your image:
But the final result looks like a real 360-degree photo! Lets try it on “restaurant” and “hotel” queries:
Good job DALL-E! The photos look very realistic!
Trying results in a virtual tour
To test the results in a virtual tour I created a site (https://viarlive.wixsite.com/ai-virtual-tour) on Wix and added the Virtual Tour app to create virtual tours based on my generated images. I created 2 demo tours: one with MidJourney photos and another with DALL-E’s ones. No doubt the DALL-E tour looks much closer to reality but it still has a show-stopping issue. Unfortunately at the moment, any AI is not really “understanding” what it is doing. It simply tries to compose an image based on its knowledge base. As the result, it does not care that the left and right edges of 360 photo should match and we are getting issues like the one in the screenshot below:
But other than that DALL-E did a good job. MidJourney is just too creative and mind-blowing :)
Conclusion
You still have to hire a flash-and-bones virtual tour photographer or 3D modeler :) AI still fails to generate proper 360-degree photos mainly because it still does not really “understand” what he is doing. But DALL-E seems to be pretty close. Maybe it is just a matter of some code tweaks or a special module to make it work.
Samples
https://viarlive.wixsite.com/ai-virtual-tour
Regards, Genadii Ganebnyi Founder @ Viar.Live