Photogrammetry and Star Wars Battlefront

[luchian]

Thanks to @Mr Whippy 's hint, I "stummbled" accros this presentation : Photogrammetry and Star Wars Battlefront. Forget about EA, forget about SW. Look at the process of creation. So inspiring. And so revealing. To see how much work people put into creating state-of-the-art content. And also to get an idea about the tools we have acces to nowdays. I strongly suggest you to watch the presentation. It gives an overwiew on the approach, and it's filled with small very usefull tips. So cool !

Here's a few words about it, by Johan Andersson @ Frostbite:

Overview
Photogrammetry has started to gain steam within the Games Industry in recent years. At DICE, this technique was first used on Battlefield and they fully embraced the technology and workflow for Star Wars: Battlefront. This talk will cover their research and development, planning and production, techniques, key takeaways and plans for the future. The speakers will cover photogrammetry as a technology, but more than that, show that it’s not a magic bullet but instead a tool like any other that can be used to help achieve your artistic vision and craft.

Takeaway
Come and learn how (and why) photogrammetry was used to create the world of Star Wars. This talk will cover Battlefront’s use of of the technology from pre-production to launch as well as some of their philosophies around photogrammetry as a tool. Many visuals will be included!

Intended Audience
A content creator friendly talk intended for pretty much any developer, especially those involved in 3D content creation. It is not a technical talk focused on the code or engineering of photogrammetry. The speakers will quickly cover all basics, so absolutely no prerequisite knowledge required.

Here is an excerpt from the presentation / wikipedia:
Photogrammetry - the science of making measurements from photographs, especially for recovering the exact positions of surface points. Moreover, it may be used to recover the motion pathways of designated reference points located on any moving object, on its components and in the immediately adjacent environment. Photogrammetry may employ high-speed imaging and remote sensing in order to detect, measure and record complex 2-D and 3-D motion fields (see also sonar, radar, lidar, etc.). Photogrammetry feeds the measurements from remote sensing and the results of imagery analysis into computational models in an attempt to successively estimate, with increasing accuracy, the actual, 3-D relative motions within the researched field.

So, if you are into modelling, I urge you - watch these. For the time spent (15-30 min) it will open a new world of possibilities.

Video 1 Final Result | Video 2 Level Contruction Kits | Video 3 Photo Scan Trips | PPT Presentation (with notes !)​

 

[Mr Whippy]

And this was their first pass using non-automated pipelines for a lot of the tasks!


I've been using some techniques for modelling a car recently... and for track data capture. I'll post up my results here in a bit.
They're not as all-encompassing or finished as the DICE stuff in the presentation, but the use of photogrammetry is still very relevant... especially in the roughing out/blocking out stages, and definitely in the texture creation process.

Also we can 'pinch' data from other sources nicely. I captured and processed some aerial imagery from Google 3D maps for ir Sindanco about 2 years ago now for blocking out his amazing AC track and I'm sure it helped him in those early stages to just get a 3D blockout to work onwards from!


They're very relevant processes right now and well worth getting familiar with and using where possible!

 

[Mr Whippy]

Some super-early progress on my car model.

I've used this as a test bed for this build process because I own the car, it's easy to go check reality vs my model, and it's quite a complex/tough model that I was struggling to get right with the available prints.
Also I think every other model I've seen of this car is wrong somewhere so even 3D models are not the best reference.

Here is a cross-section of the car to visualise the noise floor, which by all accounts is easily as good as the best blueprints I have any way (which often have questionable lines or missing details)
You can even see in this cross-section where the bonnet is lower than the wings at each side... such details might seem insignificant but it all adds up to enhanced realism in the end.



And just tinkering with laying over polygons onto point clouds, draping over meshes, baking normal maps etc (useful for the roof mostly)


I'm about ready to start doing big chunks of polygonal work now and building a proper final mesh.


I'm not sure how accurate it is outright but it's gonna be as good as a blueprint car generally, but with probably better surface curvatures because they're just not shown up on prints.


Cheers

Dave

 

[luchian]

Wow, this is just great @Mr Whippy . I've seen the other thread about the Honda on AC forums, but used to think it's overkill. And now, all of a sudden it all seems so obvious - there is no better way ! Having some blueprints for general volume and this as reference for perfect alignment it's perfect. Even more so on car interiors where there is rarely (if ever) plans with dimensions.

Great stuff, thank you for sharing.

PS: if it's not a very personal question - what do you do for a job ? You seem to always be on top of these things.. it can't be only a hobby. Or can it ?

 

[Mr Whippy]

Oh that Honda thread is interesting.

I'd done tests with paper too which work ok, but I wanted a method that uses less interference. I don't want to have to ask people if I can wrap their cars.

I found a nice enough method but want to refine it further and then I'll happily document a fairly foolproof approach.

I'll probably start a WIP thread going through most of the stuff post-point cloud creation too, as working off a point cloud is different vs prints, photo-matched, or a CAD mesh too!




Half of the really interesting stuff to do with cars and tracks is hobby but it pushes into commercial/freelance work.
I've been working freelance for just over 4 years now and have mainly done simmy/car/track work, but before that I was just a digital generalist I'd say, 2d/3d/video/mograph etc.

But I've always been into car/track/sims/games since the late 90s, and I've always been tinkering with this kinda stuff, trying new techniques, data sets, ways to get more accuracy, or do stuff quicker, or whatever else. I think I'm on my 20th year using 3DS now too haha. Ah using 3DS DOS with 8meg of ram



Photogrammetry is definitely a big game-changing leap for, err, games

The downsides of all these advancements is that you just have more details to make, and higher expectations of quality these days, so cars and tracks are probably harder to make today than any time before

 

[luchian]

Very interesting, thanks for the answer. It's great to be able to acces these techniques that some years were only for the big studios. I've know 3ds since its early days (486's and Pentium 150 Mhz era) but my day to day job always went in a slightly different direction. And even more so from March this year . But that's why I love AC - it's a great platform for this hobby.

Looking forward to seeing more !

Off-Topic: Bimmer lover here too . Had an E46 some years back, and very soon a 5er.

 

[luchian]

Inspired by the recent stuff, decided to make a quick test. Had some photos from last year, so this is what I got in about 15 min time (including processing).

And this is with default settings of the SW, and no masking applied to the photos. Just RAW > Import > Process. With some more work game assets can look stunning. Add some VR into the play, and it's the next best thing of being actually there .

 

[Willy Wale]

Inspired by the recent stuff, decided to make a quick test. Had some photos from last year, so this is what I got in about 15 min time (including processing).

.

Luchian, which software are you using? I was going to make a test with Arc3d at the weekend. I need to scan some statues for my circuit

 

[Mr Whippy]

That top image is Agisoft Photoscan.

The next ones are the Photoscan created model and texture in Blender.

Photoscan is well worth it. I think it was about £125 when I bought it, it's more like £200 now but still worth it imo.
You can do loads in the demo to see the results, just no exports.

If you really go to town with the process you can export a mesh, import it to your 3D app, create a nice mesh and uv unwrap, bring that back to Photoscan and project the HQ image data and normals etc back into the new mesh/uv, also in HDR too.

Plus the latest version has rolling shutter compensation so you can get better results from cheaper video gear, which may often be enough quality and faster depending on what you're doing.


It's far and away the best software right now for a professional doing almost any facet of photogrammetry work.

 

[Willy Wale]

Cheers, it's $179 USD for the standard version. I'll have a go with the 30 day trial.

As far as scanning cars goes, when I've seen it done with laser scanners the car has a layer of talc applied to avoid reflections. Then small reflective spot targets are applied to key features to make the post processing easier. I presume that's the idea of wrapping in paper you mentioned above. Easier to wash off the outside than the interior obviously.

annoyingly I have access to a large number of high quality scans (stl) at work that I can't make use of at home.

 

[Mr Whippy]

I did my car with water drops and it's fine.
Better than blueprints or cam matching.
Only an OEM cad model is better imo, or maybe laser scan (if someone else is paying!)

Interior should be close enough to accurate with mostly rough materials and good 'leading' for tracking points (so if soft reflections lead to changes, it's done gradually over many images)

Adding talc to a car is gonna be a hard sell for hobby purposes, vs rain or clean water on a freshly waxed car.

What we really need is photogrammetry with stereo cameras making depth map shots on the fly, which are then merged into a 3D model in post.
That'll probably be where things go for car sized scanning imo.

 

[Willy Wale]

Adding talc to a car is gonna be a hard sell for hobby purposes, vs rain or clean water on a freshly waxed car.

Definitely, I've never had to clean a car afterwards but I bet it takes a while.

 

[Mr Whippy]

It's a bonkers approach really. So expensive too.

It guarantees a repeatable result though which I suppose is one of the main things, but it feels like ramming a square peg in a round hole.


I remember getting my car covered in flour one Halloween, it then rained and dried before I picked up my car.
Jeez there was stuff everywhere in ringing bay and door shuts for months afterwards haha!



I'll post some links to my z4 build on here.
I did put a big thread on AC forums but they nuked their site so I removed it.

 

[Willy Wale]

You can get a car scanned for a couple of grand, which is huge for a hobbyist but peanuts for a car company. If you're scanning it to do CFD (for example) on a competitor then it's really good value. A day in the wind tunnel will cost £15k plus.

 

[Mr Whippy]

Aye that's it.

But I'm surprised industrial espionage 'lite' doesn't mean CAD models aren't leaked anyhow.

I've had car models in CAD, pre-sales, as someone supplying 3 levels down the line from the manufacturer.


But yeah, laser scanning still has it's place.

For us hobbyists though it's good to experiment and push photogrammetry

I bet the professionals will slowly use it too, just they might not be the ones innovating with techniques, or if they are they're keeping it trade secrets for obvious reasons!