Scaling common geoprocessing tasks with Spatial Indexes

So, you've decided to start scaling your analysis using Spatial Indexes - great! When using these grid systems, some common spatial processing tasks require a slightly different approach to when using geometries.

To help you get started, we've created a reference guide below for how you can use Spatial Indexes to complete common geoprocessing tasks - from buffers to clips. Once you're up and running, you'll be amazed at how much more quickly - and cheaply - these operations can run! Remember - you can always revert back to geometries if needed.

All of these tasks are undertaken with CARTO Workflows - our low-code tool for automating spatial analyses. Find more tutorials on using Workflows here.


Buffer

The humble buffer is one of the most basic - but most useful - forms of spatial analysis. It's used to create a fixed-distance ring around an input feature.

  • With geometries... use the ST Buffer tool.

  • With Spatial Indexes... convert the input geometry to a Spatial Index, then use a H3/Quadbin K-Ring component to approximate a buffer. Lookup H3 resolutions here and Quadbin resolutions here to work out the K-Ring size needed.

Clip/intersect

Where does geometry A overlap with geometry B? It’s one of the most common spatial tasks, but heavy geometries can make this straightforward task a pain.

  • With geometries... use the ST Intersection tool. This may look like a simple process, but it can be incredibly computationally expensive.

  • With Spatial Indexes... convert both input geometries to a Spatial Index, then use a Join (inner) to keep only cells which can be found in both inputs.

Difference

For a “difference” process, we want the result to be the opposite of the previous intersection, retaining all areas which do not intersect.

  • With geometries... use the ST Difference tool. Again, while this may look straightforward, it can be slow and computationally expensive.

  • With Spatial Indexes... again convert both input geometries to a Spatial Index, this time using a full outer Join. A Where component can then be used to filter only "different" cells (where h3 IS null AND h3_joined IS not null) - at a fraction of the calculation size.

Spatial Join

Spatial Joins are the "bread and butter" of spatial analysis. They can be used to answer questions like "how many people live within a 10-minute drive of store X?" or "what is the total property value in this flooded area?"

Our Analytics Toolbox provides a series of Enrichment tools which make these types of analyses easy. Enrichment tools for both geometries and Spatial Indexes are available - but we've estimated the latter of these are up to 98% faster!

  • With geometries... use the Enrich Polygons component.

  • With Spatial Indexes... use the Enrich H3 / Quadbin Grid component.

Check out the full guide to enriching Spatial Indexes here.

Aggregate within a distance

Say you wanted to know the population within 30 miles of

For instance, in the example below we want to create a new column holding the number of stores in a 1km radius.

  • With Geometries... create a Buffer, run a Spatial Join and then use Group by to aggregate the results.

  • With Spatial Indexes... have the inputs stored as a H3 grid with both the source and target features in the same table. Like in the earlier Buffer example, use the H3 K-Ring component to create your "search area." Now, you can use the Group by component - grouping by the newly created H3 K-Ring ID - to sum the number of stores within the search area.

This is a fairly simple example, but let's imagine something more complex - say you wanted to calculate the population within 30 miles of a series of input features. Creating and enriching buffers of this size - particularly when you have tens of thousands of inputs - will be incredibly slow, particularly when your input data is very detailed. This type of calculation could take hours - or even days - without Spatial Indexes.


Next up...

Last updated

Was this helpful?