Terrain engine upgrade #1
The view from 1.5 meters altitude
I recently started upgrading the terrain engine. This is necessary work for the integration of the new planet classification system. The procedural generator uses a special file (palette.cfg) to assign "coloring" (a palette) to the generated planets. For each class of planets, there are several "presets" - different coloring options - in that file. Changing the classification system forced me to change this file as well. But I decided to do something even better: completely re-design the system of presets and eliminate its main disadvantages. There are known difficulties with palette modding - the mod can only replace the whole file, you cannot just add a new preset to the existing ones. The assignment of the probabilities for each particular preset in a file is also a troublesome task. In addition to the palette, for each planet the engine assigns a set of textures that are used to generate the smallest details on the surface: stones, sand, grass, rocks, etc. But this assignment is hard-coded - it is impossible to change the rules from the outside (using a mod). The detail textures themselves are stored in a texture atlas (planet_atlas.png) - only its modification allows changing the appearance of a planetary surface up close.
The new system introduces a separate file for each preset, where, in addition to the actual coloring, the user can specify paths to the textures, the rules for their assignment, scale and other parameters. Textures are now stored as separate files rather than in a single atlas - this simplifies modding (users can easily add new textures). The engine now uses a texture array internally instead of the atlas, which also eliminates some of disadvantages of the atlas (seams on the tile boundaries).
The system of biomes (surface types) has also changed. Previously, there were only eight biomes: sea, shelf, beach, desert, lowland, upland, rock, and snow, which were assigned in order depending on the local climate, which, in turn, depended on the latitude, height and density of the atmosphere. Therefore, a direct transition from desert to snow was impossible - there were always strips of intermediate biomes between them (lowland, upland, rock). Now biomes are independent of each other, they are assigned by a shader that generates textures. The shader knows what it is generating at the current pixel: a desert, a mountain, a beach or a polar cap, so it can force a certain biome for it. Some biomes modify (overlap) previous ones - for example, snow or grass. The transitions between biomes are still limited - too many biomes cannot overlap in one point, so for now I made a transition through a "default" biome (stony surface of neutral color). The new shader system is still under development, so this may change in the future.
The number of biomes is now much larger, and each of them can use its own color and 16 different detail textures. Textures are used in this way: 4 levels of slope (from a horizontal surface to an abrupt cliff) and 4 "layers" in each. "Layers" or "levels" are textures that are mixed together on different scales. The first two "large-scale" levels specify details of resolution down to 10 meters per pixel. This is what was used in SE before, but looked somewhat strange, due to the technical limitations of the engine (single-precision real numbers used in shaders do not allow generating details less than about ten meters on a planet the size of the Earth). Now these generated textures are mixed with two more layers of textures directly in the shader that draws the planet (in the previous versions of SE it was just a little random noise). This allows reaching a level of detail on the surface down to a few millimeters, despite the fact that the detailed geometry and generated textures remain the same - down to 10 meters per triangle or pixel. In the future I am planning to implement additional local tessellation to make the geometry smoother or more detailed.
For the "upper" two layers of detail textures, one can now use textures made from real space photographs of the Earth and Mars:
By carefully choosing the two most detailed levels of textures, their scale and mixing ratio, you can achieve very interesting terrains. Each texture has a height map that is used for mixing, so for example, pebbles from a smaller texture are visible only between boulders or "slabs" of the larger one:
Normal maps are used for lighting. In the future I am planning to implement POM (Parallax Occlusion Mapping) to give volume to details like stones. Some textures, such as snow and ice, have an increased level of specular reflection:
To eliminate the highly noticeable tiling (repeatability of the pattern), the method of random shifting of texture coordinates within random Voronoi cells is used, but unfortunately, it is very resource-consuming. Detail textures have a significant impact on performance. There will be several options for quality settings, including completely disabling them. Fortunately, detail textures are only visible at very close range, at distances of up to 100 meters. So for more distant parts of the landscape and for views from space, one can use another, faster shader that does not impose detailed textures.
To transfer information about the biome code and the texture codes that need to be superimposed, generation shaders must produce additional textures, in addition to the color and normals textures. I'm still experimenting with different approaches - transfer only the biome code and calculate everything else on-the-fly (just one byte per pixel of additional memory, but a large load in the planet rendering shader), or transfer texture codes and their blending coefficients (8-16 bytes per pixel of additional memory, but the shader is easier and faster). In any case, the use of additional memory cannot be avoided, but it's worth it. Fortunately, the fact that these additional "texture materials" are needed only for the most detailed (close) patches of the landscape helps here also.
Another very simple, but very useful effect is the addition of a small distortion to the texture coordinates of the basic, "large-scale" textures. This eliminates their visible pixelation, virtually increasing the resolution of the surface, even without imposing the two most detailed texture levels. This works even for Earth, Mars and other bodies with real textures. The imposition of detail textures for them is easy to implement - you need to define biomes somehow, for example, by analyzing the color of the pixel of the global texture. This will have to be postponed for the future.
There are many opportunities for modding now. For your planet, you can either use one of the existing presets of biome coloring and texturing, or edit them to your liking. The planet editor has an interface for this: you can select a texture for each of the four slopes and four layers of each biome (the biome itself is selected in the drop-down list). The choice is possible only among a limited set of textures - those that lie in the new detail textures folder (data/common/tiles/). For now, it contains the old atlas, split into individual tiles with a resolution of 256 * 256 (there are as many as 128 of them). But for the release I plan to switch to textures with a resolution of 1024 * 1024 and even more, but their number must be reduced. Modders of course will be able to add new textures, but keep in mind the amount of memory that they occupy, and their resolution (it should be the same for all textures - this is a limitation of OpenGL texture arrays). Of course, the textures should be seamless.
Above the table with biome textures there is the old palette choice - do not pay attention, I just have not completely switched to the new system. And above them there are buttons for loading and saving the preset: your customized preset can be saved, and it will be used for the procedural generation of other planets. The concept of "weights" is used to assign the probability of the appearance of a planet with this preset. Probability is the weight divided by the sum of the weights of all presets for planets of this class (for example, "temperate marine terra"). For example, if there are three presets with a weight of 1, and two with a weight of 0.5, then the probability of the first three will be 1/4, and the last two will be 1/8.
Discuss this post on the forum.