Natasja Paulssen is a standalone female in her field, leading innovation in augmented reality over the last ten years. She has founded not only one, but two Eindhoven-based companies: Dutch Rose Media and 4DR Studios.
She’s helping to define what this immersive technology can be used for and the possibilities seem endless. In a short period of time, she has developed an impressive client list and has pioneered the only mobile volumetric capture studio in the world. We discussed the technology, her career and what makes her excited about the field.
What you're doing is really ambitious. You're creating a digital universe. Can you explain volumetric capture a little bit? A lot of people aren’t familiar with it.
It's best shown because it's very hard to realize how difficult it is to create. It looks completely natural. Only when you start thinking about how we create this, that's when you start wondering. Want to see it?
Seeing a demonstration of the technology and walking through Paulssen’s studio is like walking into the future. A circular arena of green screens and high-resolution cameras surround the area where actors are filmed. The demo allowed us to rotate a screen to move where we wanted around the recorded model. While we were doing this, Paulssen explained it further.
You've seen it now. So, what is it? We have 32 cameras around the person we want to film. It captures the person from all angles. The video image of all the cameras is stitched together, so in the end, we create a 3D model. Volumetric capture is basically a video-quality 3D model or model animation. We create really high quality, lifelike holograms based on the actual movements of a real person. We film the movement and the liveliness in one go. What you just saw in the demo was Memphis Depay, one of the top players on the Dutch football team. He came in, did his moves and it was recorded by these cameras.
And how long was he here for you to film?
Ten minutes for these eight seconds of film. It doesn't take that long, but we have to prepare really well. For example, we are filming PSV and all 25 players are coming in. We have ten minutes per player, so we need to plan every breath.
Our studio is the only one in the world that's mobile. When we bought it, the first thing we did is take it apart and rebuild it in a different way so we could transport it to film the Dutch and the Spanish national football teams. It's cheaper to take the entire studio and travel to Madrid. I mean, these guys make millions a year, right? If they need to be here for days, that's more expensive than if we go there for a week.
I know you started 10 years ago, but how did you start a company before this technology was even conceivable?
What we started with was the augmented reality idea that we can put 3D objects and animations into real space. You are upgrading the real world. You can have content available anywhere, any place. It's something we need because our world is getting more complex. We need information to help us and having other people around all the time is too expensive and not feasible. That's the thought I started with.
In 2011, I wrote a blog about augmented reality and it got a lot of interest. I had a demo that when you looked at the Eindhoven city hall through your phone, it collapsed and some beautiful building came from under it. I showed it to an alderman and she said, “We're starting this thing called Dutch Technology Week. They need something to pimp up their communications. Go talk to them.”
In that same period, someone wrote a big article about us and all the outdoor agencies were calling us after that. And one of them organized augmented reality days for all their clients and they asked us to produce these augmented reality projects. So, within three months after the start of Dutch Rose Media, we had all the A-brands from the Netherlands in our portfolio. We had lift-off.
In the beginning, we were not able to create people in 3D. We could create puppets on a green screen, but it was 2D. So, as soon as the viewer started moving, they saw it wasn't real. That breaks the magic of the experience. It becomes less convincing because the cameraman decided what you watch. With our solution now, you decide what you watch. If you want to see it from this angle or that angle, you can decide.
It looks like you're already established with some fairly good clients, but you must have a vision because what you're doing is just in its infancy. Is that correct?
Yes, even though we've been doing it for over ten years now. We started in 2012 and it's still really new. I think that's incredible. Like you always hear, “Tech is moving so fast, blah, blah, blah.” Yo, guys, we've been at this for 10 years! It’s a lifetime in the development of tech.
And what will this technology be used for?
That is one of the most important questions I had in the beginning. I had ideas, but was the business case valid? We didn't know. So, we organized a research fund and had an open call for 20 projects to have one day free in the studio. We wanted to know if using this technology would be a feasible alternative for them in the future.
We’ve found many applications. We work on histories that come alive for museums for educational purposes. We also work with entertainment where it's all about recognizing idols in music and sports. Fashion is a really interesting one because it's all about the way the fabric moves. Creative directors from the fashion industry are really, really impressed with the quality of the photographs.
I can see there you could develop a virtual runway fashion show.
Exactly. A runway show is on a runway because we want 500 people to be able to sit around it, but that's not necessary anymore. You have to rethink the entire concept of a fashion show. And this is true for every area we use this technology. Everybody has to rethink why are we doing what we are doing.
Another important use case is workplace training. This is where the huge development is right now because of the war for talent. What you can do using immersive content and volumetric video is train people faster. You can make them more flexible in what they do. Older people can start coaching the younger ones, but the experts don't need to travel anymore. You can work smarter. Clean rooms could be a really interesting application. If you need to train people to work in clean rooms, it takes up a lot of time. We could virtualize that by using real people because then you see exactly how you have to behave in that environment.
This training is virtual, but it's also real. That opens up a lot of training possibilities. We are focusing on the current HR challenge. Can we train people faster? Can we help them shift jobs or tasks more easily? There are a lot of options where we have to get creative and we need to get creatives in contact with industry to start doing these things.
We know raising money for startups is tough, specifically in the Netherlands.
I can only tell you about my experience. For us, we just didn't fit anywhere. Tech investors said we were too creative and not scalable. Banks said we were too innovative and that for collateral, we seemed to be the only one who could use our equipment. The creative funds said, “Oh, you're investing in tech. We're not supporting that.” It took about two years to get the investment together.
You have a computing science degree. Were there a lot of women in your classes?
We started with 140 students. Seven were women. Two finished. You see that role models are really important. Look around this office. It’s 50/50 women and men.
Is that by design? Or did it just work out that those were the talented people you needed?
I don't actively hire; people just … find us and then we create a space for them. I have no problem getting enough people. It’s not because of the technology that they come. I think it's more about the wide end of what we do. We do it to make the world a little bit better. That's it. Nothing more, nothing less. We work for our customers, but we also work for ourselves. We have to work for many years to come, so we'd better be doing something we like.
Can you tell me a little bit about who you are looking for? What sort of skills do you want?
We need people with funny brains, basically. There’s the character Pippy Longstocking and she says, “I've never done it before, so I think I can do it.” We need those people. Those people hardly ever have a regular curriculum. They do all kinds of different things before they end up here. But besides people, we also need clients.
Who's your dream client?
Our dream clients are based in the region here and they are less sexy than doing a shoot in Madrid or Paris. It's the training sector. We want to help with the war for talent. The figures that you see are so unbelievable when you start using this technology. There are use cases that show 96% faster learning. You wouldn't believe it if it didn’t come from a valid source. Wherever you have a shortage of people, we can use this technology to create alternatives.