Dream3D: Customer Spotlight
Why did you start Dream3D?
In college, I did a lot of computer graphics and VR storytelling, and I always remembered how painful the tools were. To even tell the simplest of stories in 3D would take weeks and weeks of work. This is why if you’re doing any type of animation and storytelling, it’s usually big studios with big budgets. So when all this progress was happening in generative AI, the first thing that came to mind was to wonder: what if I could build the tool I wanted in college that would help tell the stories?
Who is your ideal customer?
Dream3D is an AI-powered 3D design tool that makes it easy to create studio-quality computer graphics. The people we’re building for right now are individual designers, in-house designers at firms with rendering tasks– that is taking an existing 3D model turning into a photo-realistic visualization. We’re starting with a simple and expressive rendering tool that we’ll progressively expand to a complete 3D suite. It’s very much a model where you start building for individuals and it eventually gets adopted by big companies. We want to make something that’s very delightful and focused on our users.
What is the role of AI in your product?
AI is the crux of everything we do. One way to think about it is that we’re replacing rendering in the workflow. Rendering tools are currently simulation-based; input quality equals output quality. We’ve replaced that with a generative approach so you have a lot more flexibility to build out and structure a scene, set your camera, and then use natural language to get the final output that you want. Everything we do is AI-based and requires us to build custom models.
How did MosaicML help you build your models?
MosaicML is our training platform. We trained all our models on MosaicML, and the big reason is with models of this size if you try to train on your own machine it will take forever, so you need to do distributed training, and when you’re trying to train on a cluster, there’s a lot of complexity that goes into that. MosaicML was able to abstract away all the complexity of distributed model training.
How long did it take you to get up and running with MosaicML?
It was a matter of three weeks from our initial sales call to being up and running. We were training a model within the first week. In my previous company, I architected everything and I was the only one who knew how to manage and onboard new users to the system. With MosaicML, it’s so simple that my co-founder can easily deploy a job with a CLI command. The intangibles of making it easy for someone who is new on the team to get up to speed have been huge.
How does MosaicML help you scale up to hero runs?
We were looking to do on-demand auto scaling on AWS. Since we’re a startup, it doesn’t really make sense for us to go buy a year’s worth of reserved allocation yet. MosaicML made autoscaling easy. We hit “scale” and it goes up and down however we need it. I cannot begin to describe how simple MosaicML has made it. Plus, the Composer library was simple to use and felt more in line with how you typically work with PyTorch Lightning or normal PyTorch code.
How is your experience with MosaicML? What additional capabilities would you like to see?
It’s been great, your team has been super responsive. We’ve been trying to abstract away different clouds and get the best prices on different clouds. We’re trying to use on-demand instances as much as possible. But capacity varies a lot. Sometimes we get nodes right away, sometimes we wait a few weeks, so we’re setting up a backup cluster. It would be great to be able to find capacity on whatever cluster is available, so we can abstract away that last bit. Not worrying about capacity helps a lot as the team grows. If a bunch of people are trying to simultaneously run jobs, we can make the progress we want to make immediately instead of worrying about exceeding our capacity for the week.
What’s next for Dream3D?
This coming Monday, we’re launching our public beta, and we’re already thinking about how to make the tool increasingly powerful. Right now we have this core workflow where designers can create renders; how can we give users more and more control, while keeping the tool accessible? When we first started talking to users, we got a lot of feedback saying that Stable Diffusion and DALL-E are cool, but it’s hard to use them in a professional workflow because you don’t have a lot of control over the final output render. And this comes back to a lot of what we’re seeing in generative AI. Simply tacking on these tools to existing workflows is short-sighted and doesn’t fully capture their potential. Instead, when you have core technological unlock, you need to build new workflows.That’s what we are focused on today– rebuilding the rendering workflow to be AI-first. Very soon, this will expand to enabling animation and interactive 3D environments, building towards a landscape of a complete 3D design tool.
How important is it to train your own model vs. using off-the shelf models?
Generally, the models that are done in the research community can be quite different from models applied in an actual industry. They are related but tangential. You get inspired by research, but you need to tweak and tune the objectives for that model. You borrow the techniques and retrain for your specific task. That’s the only way to make a production tool that fits in production workflows.
On top of this, training your own models and owning your research stack allows you to take ownership over your product timelines. As a startup, you want to limit external dependencies and take control over your destiny. This is really hard to do if you’re exclusively relying on open source for your core technology. Mosaic’s training optimizations make it feasible to develop our own models as a startup. We align ourselves with where research is going, but then take focused bets with our own models to drive value for customers on our own clock, running our own race.
What are your thoughts on the future of Generative AI?
We’ve seen a bunch of cool demos that are really powerful, and we have all these foundation models that can do a lot. We’re going to slowly see these models used for professional use cases. Right now, people are behind the scenes building products that will redefine these workflows in a few months from now. As I mentioned before, currently only big studios can afford to go tell the stories they want to tell or create the content they want to create. We’re going to see an explosion of individual creators and smaller studios being able to compete. This is a big part of AI: You’re taking hard-to-use, expensive tools, making them more accessible, and giving individuals the power to unlock new levels of creativity. We’re going to see an explosion of use cases and redefined workflows. It’s part of the natural progression of technology.
Dream3D is now in public beta. Sign up and start creating today at www.dream3d.com!
