SEED Houdini Landscape PDG

A tile-parallel procedural landscape system in Houdini PDG: cliffs, biomes and erosion cooked per tile and packed into a custom Unity terrain format.

Overview

A procedural landscape system built in Houdini with VEX, Python and the PDG (Procedural Dependency Graph). A full-map terrain is split into a grid of tiles that cook in parallel, each one running biome generation, mountain and cliff projection, and erosion, before the tiles are stitched back together and exported into the custom data format our Unity landscape system consumes.

The interesting problems here are not any single terrain operation: they are the ones that only appear once you go tile-parallel. How do you erode a tile so its edges still line up with its neighbours? How do you place a cliff that straddles a tile boundary without a visible seam? How do you keep 1500+ tiles cooking independently and still end up with one continuous world?

My role: I built large parts of the system, and am still maintaining it actively although it is slowly faded out for more runtime solutions. It is though a collaborative work of all Technical Artists at Klang, so I cannot take full credits for all the work. My contribution was: the cliffs, the tiling system, the export and the PDG data flow.

Gallery

Deep Dive

Landscape Tiles

A full landscape is too heavy to hold in memory and cook as a single heightfield, so the pipeline partitions the map into a grid of tiles and drives them through a PDG network. Each tile becomes a work item; the scheduler cooks them in parallel and only synchronises at the points where it has to.

The subtle part is the word independent. A naive split would cook each tile in isolation and then butt them together, which gives you a hard seam everywhere two tiles meet: erosion that flows off one edge has no idea there is terrain on the other side, and a cliff whose base sits on the boundary gets sliced in half.

The fix is to never cook a tile truly alone. Each tile is grouped with its eight neighbours and given a padded gutter of overlap, so every operation that reads a neighbourhood (erosion kernels, blurs, slope detection, cliff falloff) sees real terrain past its own border rather than empty space. After cooking, the padding is cropped back to the tile’s true bounds and the overlap is used to blend the seam.

Partitioning is driven in Python: from a tile’s grid index the network derives its neighbours, filters the ones that fall outside the map, and hands the group to the cook. In spirit it is nothing more than a bounded 3x3 stencil:

import hou
import math

node = hou.pwd()

# Partition by grid neighs
for work_item in work_items:
    stride = work_item.attribValue("map_size")
    #partition_holder.addItemToPartition(work_item, work_item.index)
    i = work_item.attribValue("tile_id")
    x = math.floor(i / stride)
    y = int(i % stride)

    grid_pos_neighs = list()
    for a in range(-1, 2):
        for b in range(-1, 2):
            if not (a == 0 and b == 0):
                nx = x + a
                ny = y + b
                if (nx >= 0 and nx < stride) and (ny >= 0 and ny < stride):
                    grid_pos_neighs.append((nx, ny))

    inid = 0
    for ix in range(stride):
        for iy in range(stride):
            for n in grid_pos_neighs:
                if ix == n[0] and iy == n[1]:
                    partition_holder.addItemToPartition(work_item, inid)
            inid += 1

Once every tile has cooked, a wait-for-all gate lets the merge stage stitch the seams using that padded overlap, the padding is cropped away, and the result is packed into the Unity terrain format: a heightmap, splatmaps for material blending, mask layers, and normals, written per tile.

import terrain.nodesCode.PDG_Terrain_Unity as tool
from importlib import reload
reload(tool)

map_name = work_item.attribValue('map_name')
height = work_item.attribValue('max_height')
map_size = work_item.attribValue('map_size')
import_path = work_item.attribValue('export_path')
tool.create_unity_map(map_name, height, map_size, map_size, import_path)

# terrain.nodesCode.PDG_Terrain_Unity

import os
import json
import shutil
from importlib import reload
import k_aip.core.environments as envs

reload(envs)

envs.build_klang_environment()

import k_aip.core.ini_handling as ih
import k_aip.core.git_utils as gutils
import k_aip.core.unity_utils as unity_utils

reload(ih)
reload(gutils)
reload(unity_utils)

UNITY_UTILS = unity_utils.UnityAIPUtils()

HOUDINI_TO_UNITY_COMMUNICATION = "HOUDINI_TO_UNITY_COMMUNICATION"
CREATE_MAP_COMMAND = (
    "Klang.Seed.Editor.Landscape.WindowTabLandscape.CreateLandscapeHoudini"
)
IMPORT_MAP_COMMAND = (
    "Klang.Seed.Editor.Landscape.WindowTabImport.ImportLandscapeHoudini"
)

LOG_PREFIX = "houdini_to_unity_log_"


def create_unity_map(map_name, height, size_x, size_y, import_path):
    print("Creating Unity Map")
    print("Map Name:", map_name)
    print("Height:", height)
    print("Size:", "X:", size_x, "Y:", size_y)

    log_path = os.path.join(import_path, f"{LOG_PREFIX}{map_name}_create.log")

    map_data = {"Name": map_name, "SizeX": size_x, "SizeY": size_y, "MaxHeight": height}

    print("Map Data", map_data)
    print("Log Path", log_path)

    os.environ[HOUDINI_TO_UNITY_COMMUNICATION] = json.dumps(map_data, indent=4)
    UNITY_UTILS.execute_aip_unity_command(
        "aip-landscape", CREATE_MAP_COMMAND, log_location=log_path, quit=False
    )


def import_unity_map(map_name, import_path, testsEnabled=False):
    log_path = os.path.join(import_path, f"{LOG_PREFIX}{map_name}_import.log")

    if not import_path.endswith("out/"):
        import_path = os.path.join(import_path, "out").replace("\\", "/")
    print("Importing Unity Map")
    print("Map Name:", map_name)
    print("Import Path: ", import_path)

    map_data = {"Name": map_name, "Path": import_path, "TestsEnabled": testsEnabled}

    print("Map Data", map_data)
    print("Log Path", log_path)
    os.environ[HOUDINI_TO_UNITY_COMMUNICATION] = json.dumps(map_data, indent=4)
    UNITY_UTILS.execute_aip_unity_command(
        "aip-landscape", IMPORT_MAP_COMMAND, log_location=log_path, quit=False
    )


def copy_harvestable_data(map_name, import_path, file_name):
    # get artsource
    art_source = os.environ.get("ARTSOURCE")
    if art_source == None:
        print("ERROR: ARTSOURCE not found!")
        return

    # collect files
    files = os.listdir(import_path)
    files_to_copy = list()
    for file in files:
        if file.contains(file_name):
            files_to_copy.append(file)

    copy_folder = os.path.join(
        art_source,
        f"aip_publishing/packages/seed-landscape-shared-{map_name}/Content/tileData",
    )
    for file in files_to_copy:
        src_path = os.path.join(import_path, file)
        dest_path = os.path.join(copy_folder, file)
        try:
            shutil.copy(src_path, dest_path)
        except Exception as e:
            print(f"ERROR: Failed copying {file}: {e}")

The production source is proprietary, so the code above is illustrative: it shows the techniques, not Klang’s implementation. The diagrams describe the pipeline’s logic at a conceptual level. All screenshots, and the real pipeline itself, belong to Klang Games GmbH.

Links

• Klang Games
• SEED Game