‍🔹BANNER IMAGE

Add an image here that to be used as a banner image (Size 1920x1080, Max 2mb Size)

‍🔹LESSON CONTENT

‍LESSON OVERVIEW

In this lesson, we’ll go over the basics of photogrammetry from what equipment you need to get started, ideal capture conditions, as well as the process of delighting and processing your scan into a final highpoly. This will get you started on how to create your own scanned content!

‍WHAT IS PHOTOGRAMMETRY?

Photogrammetry is the process to recreate reality in 3D through taking photos. We do this by photographing an object or surface from all angles, and then running those photos through a software like RealityCapture or Agisoft Metashape to turn it into a 3D mesh. Photogrammetry can be used for more natural content like rocks, leaves and tree trunks, but can also be used to creating textures, or even man-made props!

‍EQUIPMENT OVERVIEW

To get started with photogrammetry you don’t need much. Any basic DSLR camera (or similar) will do. In fact, even your phone will do! (If you do have a camera however, a 35mm lens is ideal to avoid any distortion, although any lens will do the trick.) Aside from this, investing in a colorchecker passport will be neccessary once you start diving into color calibration and making your scans look great regardless of lighting conditions, allowing you to make your scans look good together if they were taken in areas with different ambient lighting, different times of day, etc. Finally, a yardstick will help you along with making sure your scans end up being the correct size in engine!

• Camera/Phone (+35mm lens if you have it):
• To capture your photos.

• Colorchecker Passport
• (highly recommended but not mandatory)
• :
• To colorcalibrate your images.

• Yardstick
• (optional)
• :
• To have a scale reference when making sure the scan is the proper size after processing.

‍DIFFERENT PROGRAMS

Highlight these different programs

Our main tool in photogrammetry is a software like RealityCapture, or Agisoft Metashape. Along with this we will also need a software capable of color correcting your photos such as Adobe Lightroom or similar. For delighting our textures we will also make use of Agisoft Delighter.

In this lesson we will make use of mostly free softwares apart for the color calibrating section. For our photogrammetry we will use RealityCapture, and for our delighting we will use Agisoft Delighter. For color calibrating the photos I will personally use Adobe Lightroom. But any photo editing softwhere where you can edit your white balance and levels will do the trick!

‍PREPARING FOR DATA CAPTURE

Alright, you have your camera! Now it’s time to get outside and get our first capture!

‍Picking the right weather conditions

When capturing for photogrammetry the weather is important. Ideally we want to do our capture on a cloudy overcast (but not rainy!) day. This to make our capture in as neutral lighting conditions as possible, with as little directional light as possible. While doing captures in sunny conditions are possible, it’s a lot easier without directional light and harsh shadows. Avoiding sunny days will also make the delighting and cleanup process a lot easier. Likewise, if we were to capture our object on a rainy day, the values would be off, and might even lead to a lot of reflections which won’t capture well.

‍Picking what to capture

Photogrammetry can be used to capture most static objects, be it textures, tree trunks, rocks, leaves, or anything else! However, there are a few caveats. Apart from lighting and weather that we’ve already covered, it’s good to avoid any sort of shiny or reflective objects. There are ways around it, but for the sake of the basics we’ll stick with non-reflective objects. The reason for this is that the scan software gets confused about reflections, and thus our scan won’t turn out good. There are workarounds like “scan sprays”, polarizing filters and ring flashes, but we won’t cover that in this lesson. When it comes to what you’re capturing the base principle is the same, the main key takeaways when making your choice however are the scale (larger object requires more photos) and the complexity of your object in terms of crevices (small crevices will be hard to capture well due to limited reach). Similarly, something overly noisy like grass will require hundreds of photos but even so may not turn out great. Instead a less noisy surface (like dirt) might be a better ground material, or creating your own grass in Substance Designer using scanned single grass straws for example.

Are there other limitations here? Scale of the object, complexity of the object, difference between capturing a tileable and prop? Doesn’t have to be long, but could be interesting to have - Prop: This needs to be kept in mind - Tileable: This needs to be blah blah

⚠️ To add: Image of shiny photogrammetry capture attempt.

All done? You have your object scouted out and the weather is just right? Great! Let’s get scanning!

‍Capturing Your Object

For our default camera settings, we’ll want to be in Manual Mode. This gives us full control over our camera and shooting settings. Start out with an aperture of F8 to get everything in focus, shutter speed at 1/100 or above to avoid shakeyness in our pictures, and your ISO as low as possible to avoid artificial noise. You might need to adjust these based on the lighting on the day, but the main thing is to keep your histogram as neutral as possible, avoiding any complete blacks or whites..

🖼️ ADD AN IMAGE HERE TO SHOW FSTOP SHUTTER SPEED ISO

More on camera settings can be found in the Camera Lesson.

⚠️To add: Link to Camera Lesson

🖼️ ADD AN IMAGE OF THE REFERENCE IMAGE HERE

To start with we always want a reference image of our object. This is mainly for ourselves so that we have a clear idea of what the object is. Then we also want to take a photo of our colorchecker in order to color correct our images later. This image we want to take from the top of the card, aiming our lens directly at the colorchecker.

‍Scanning Patterns

Next we’ll cover various patterns for scanning different objects.

‍Tileable Textures & Trims

🖼️ ADD AN IMAGE SCANNING PATTERN HERE

When it comes to scanning tileables or trims it’s relatively straightforward. You want to start at the bottom of your capture area, and work your way up towards the top row by row. Forward one step, capture a row of photos by moving sideways, then forward another step. All until you reach the top of your capture area This way, you make sure to not affect or ruin the capture area with your steps.

‍Prop

🖼️ ADD AN IMAGE SCANNING PATTERN HERE

When scanning a prop you want to capture as much of it as possible, generally moving around it, as well as up and down the object.

‍Vegetation

🖼️ ADD AN IMAGE SCANNING PATTERN HERE

For vegetation it’s quite similar to when we captured the colorchecker. Place your leaves, berries and whatever else on a dark background like a black piece of paper or cloth, and then take a picture directly from above. We can always be more fancy and building a lightrig for this capture, but for the basics what we have is enough. [[⚠️To add: Link to Billy’s Vegetation Workshop w. timestamp ]] And if you want to capture a tree trunk to go along with your leaves, circling around it 360 degrees like a prop is the way to do it.

‍Color Calibration and Delighting

Now that we’ve scanned our object, we need to calibrate the images to our color checker. Otherwise the scan will turn out colored from the ambient light, for example sunlight will yellow and bleach the object, whereas overcast weather might make your scan more cyan. We’ll also want to remove any extreme highlights or shadows that might’ve gotten captured along with our scan, such as in crevices or the like. For starters, we bring our images into Adobe Lightroom, using the ColorChecker image, we grab the color picker and select our whitebalance. Generally I go for a midgrey. From here we might want to tweak some other things like saturation, shadows, etc. But we can also do that when we have our mesh and texture baked out.

From here we can easily copy-paste the edited white balance to the rest of our photos, save them and bring them into RealityCapture.

As you can see in the example below, color calibrating your images can make quite the difference.

⚠️To add: REPLACE Lightroom Image with English

‍Creating Your Mesh in RealityCapture

After color calibrating our pictures, it’s time to bring them into RealityCapture where the mesh creation will take place. When first opening up the software you’ll be greeted by a UI looking like this. Thankfully, RealityCapture has its own step-by-step tutorial included. Just click on the “Let’s Get Started” box. If this window isn’t open by default, you can open it up by going under “Workflow” and then “Help” to the right of the tab.

The one thing that I like to use which isn’t covered by RealityCapture’s built-in walkthrough is the Lasso Tool. Under the “Scene 3D Tools” menu you’ll be able to select the lasso tool. Use this to select the parts around your scan which you don’t want to keep, and then hit Filter Selection to get rid of it. Finally, we can duplicate our mesh twice, export the first as your highpoly, and use the Simplify Tool to create a decimated highpoly for delighting (around 1 million triangles should be enough) as well as another mesh to use as a base for your lowpoly.

‍Final Delighting and Baking

Now that we have all our meshes exported it’s time for our final delighting. This is done through Agisoft Delighter. Import your decimated highpoly. Unless you have some really harsh shadows or highlights, Delighter handles the work automatically by clicking “Remove Shading”. This removes most of the lighting information in our mesh and leaves us with an albedo as flat as possible. If you still have visible highlights or shadows you can mark those using Brush Tool and then click “Remove Cast Shadows”. This should help remove what the automatic process couldn’t deal with.

Now that we’ve used Delighter we can export the new delit albedo and bake that down onto our lowpoly with proper UVs, along with Normals, AO, and whatever other maps you want to use to create your roughness, as well as other tweaks. It is also at this point that you might want to desaturate your textures as needed. All this can be done in your software of choice, I personally prefer Substance Designer. ⚠️To add: Link to baking lesson.

For final edits, the “skirt” left around your scan can be either deleted, or used to help blend your scan into your terrain using a shader. You can also “cap” your mesh by creating an underside for it if you’d like to. A straightforward way of doing this would be to duplicate your mesh in ZBrush or similar, rotate it 180 degrees and merge it together to create a fully watertight mesh.

By the end of this process, you should now have a finalized photoscanned asset, ready to be used in your environments!

‍Additional topics:

Guassian Splatting

Scanning reflective surfaces

Capping meshes

Lidar Scanning

Other more advanced equipment

Polarized Lenses

Linear and Circular Polarization

Creating background based on earth data

‍🔹EXERCISES

Add AT LEAST one exercise for people to improve their skills on the given topic. Practical exercises should ideally have any content to help them get started (blender file, examples, etc…)

‍🔹RESOURCES

Add any additional resources that you personally have used or have been vetted by yourself to add to the lesson.