OpenAI’s latest tool Shap-E converts text or images into 3D models

OpenAI’s latest model allows users to create realistic 3D models from text inputs.

Shap-E is capable of creating outlines of objects and works on a feature known as neural radiance fields (NeRFs) to deliver accurate responses. (Image: Research Paper)

Listen to this article

00:00

1x 1.5x 1.8x

San Francisco-based AI powerhouse, OpenAI seems to be on a roll. After introducing the revolutionary ChatGPT and its subsequent version with enhanced capabilities, now the company seems to be gearing up to introduce its latest innovation.

As widely known GPT-4 is capable of generating responses in the fastest and most efficient manner from prompts in natural language, OpenAI has seemingly taken a leap forward and now introduced its text-to-3D model. Known as Shap-E, the model can turn text prompts into three-dimensional objects.

Reportedly, the latest generative AI tool has been claimed to produce far more accurate responses compared to a similar tool Point E that was introduced in 2022 by OpenAI. This is seen as yet another breakthrough from OpenAI after the success of its text-to-image generation tool Dall-E.

Story continues below this ad

What is Shap-E?

The latest tool from OpenAI has been developed to generate fine-grained textures and complex shapes from text inputs. Shap-E is capable of creating outlines of objects and works on a feature known as neural radiance fields (NeRFs) to deliver accurate responses. Interestingly, NeRFs are the technology employed in virtual and augmented reality to transform three-dimensional visuals into photorealistic environments.

Reportedly, Shap-E applies NeRFs to common diffusion models to create the shape and texture of the object from text prompts. Compared to Point E the process is significantly faster with Shap-E.

Also read | ChatGPT Plus too pricey? 6 websites that let you access GPT-4 for free

Shap-E is currently available for free on GitHub and it runs locally on PC. According to a report on Tomshardware.com, setting up the tool and getting it to run is quite a mammoth task. Just like existing models, Shap-E comes with a lot of potential, but its output as of now is passable. According to the report, most outputs were low-res and key details were missing. Text inputs lead to outputs that were either colour-animated GIFs or monochrome PLY files that could be later opened in programs such as Paint 3D.

Capabilities and use cases

The company has reportedly demonstrated the potential of Shap-E with examples of a bowl of food, a penguin, a campfire, etc. These 3D models were generated within seconds, offering users a faster way to create 3D assets. When it comes to application, Shap-E can be beneficial for movies, virtual reality experiences, movies, engineering simulations, and most importantly for scientific research. While more precise samples of 3D output from Shap-E are awaited, the tool is touted to be revolutionary in the domain of 3D modeling and rendering.

Tags:
artificial intelligence