Planet textures

CubeMap is a software package for the conversion of textures in a cylindrical projection to a quadrilateralized spherical cube projection (a cubemap projection for short). It supports any resolution of input texture, any number of channels per pixel, 8-bit or 16-bit, and signed or unsigned integers. The format of the input textures is raw (plain uncompressed data array). The format of the output texture is raw and/or an array of tiles of different levels of detail in the raw, tga, dds, jpg or png format.

Download: CubeMap 1.06

Files:

CubeMap.exe – utility for conversion of textures in a cylindrical projection to a cubemap projection
Glue.exe – utility for gluing tiles into a single raw-file
default_cub.ccf – default configuration file for the CubeMap
default_glue.gcf – default configuration file for the Glue

The configuration file and command line

All parameters for the program are described in the default_cub.ccf file in the program’s folder. It is a simple text file that contains a set of lines of the form:

var value #comment

var – a name of a parameter, value – a value of a parameter, the sign # separates comments. A comment can be either in the beginning of the line (then the entire line is a comment), or in the end of line (after the parameter’s value). The value can be of four types:

string – a string of characters without quotation marks and spaces;
int – an integer number;
float – a real number;
bool – a binary value: true or 1, false or 0.

Both utilities can work with other configuration files, if they are passed to them as a command line parameter:


CubeMap myconfig.ccf
Glue myconfig.ccf

In this case the file default_cub.ccf or default_glue.ccf is ignored. It is recommended that you associate .ccf files with the program CubeMap.exe in the system, then opening a *.ccf file will launch the CubeMap program. A *.ccf file can be considered as a script for texture conversion with the CubeMap utility. The same is true for the .gcf extension used with the Glue utility.

You can specify paths to the input file and output folder in the command prompt. In this case the corresponding parameters from the configuration files are ignored:


CubeMap [myconfig.ccf] [-i InputFile] [-o OutFolder] [-o TempFolder] [-h]
Glue [myconfig.gcf] [-i InputFolder] [-o OutFile] [-h]

Option -h shows the prompt on the command line parameters.

Format of the planet textures in SpaceEngine

The path to the source cylindrical texture is specified by the InputFile parameter; the path to the output folder for the cubemap texture is specified by the OutFolder. If the path contains white spaces, it must be enclosed in quotation marks: “the path with spaces.” The TempFolder parameter specifies the path to the temporary folder, used by the utility to store temporary files. If it is not specified, the temporary files will be stored in the OutFolder. If the folder does not exist, it is created automatically; inside it the subfolders named neg_x, neg_y, neg_z, pos_x, pos_y, pos_z are created.

The cylindrical texture

The source cylindrical texture should be in raw format. The raw format is a plain two-dimensional array of pixels, the size and capacity (bit-depth) of which is described by the following parameters in the configuration file:

InputWidth – width of the source image
InputHeight – height of the source image
InputChannels – number of channels (Grayscale – 1, RGB – 3, RGBA – 4, etc.)
Input16bit – capacity (bit-depth): 16 or 8 bits per channel
InputByteSwap – for 16-bit only: true for little-endian (MAC), false for big-endian
InputUnsigned – for 16-bit only: true for unsigned, false for signed value
InputLatOffset – longitude offset, in degrees

The cubic texture is a set of folders and files which are organized as follows:


cubemap ———– folder specified in the OutFolder parameter
neg_x —– folder of the neg_x face
0_0_0.jpg —– 1 tile of the level 0
1_0_0.jpg –\
1_0_1.jpg –+– 4 tiles of the level 1
1_1_0.jpg –|
1_1_1.jpg –/
2_0_0.jpg –\
2_0_1.jpg –|
2_0_2.jpg –|
2_0_3.jpg –+– 16 tiles of the level 2
2_1_0.jpg –|
2_1_1.jpg –|
2_1_2.jpg –|
2_1_3.jpg –|
………
neg_y —– folder of the neg_y face
neg_z —– folder of the neg_z face
pos_x —– folder of the pos_x face
pos_y —– folder of the pos_y face
pos_z —– folder of the pos_z face

The cubemap texture consists of six faces, with names given to them in accordance with the semi-axes of the coordinate system, which they crosses. Let the origin be the center of the planet. Then the Y-axis is through the poles of the planet: the North pole is on the pos_y face, the South pole is on the neg_y face. The X-axis is passing from left to right, i.e. the left face is neg_x, the right one is pos_x. The Z axis is passing towards the observer, i.e. the far face is neg_z, the close face is pos_z. In the original cylindrical texture, this corresponds to: the upper bound is the North pole, the lower bound is the South pole, the junction of the left and right boundaries runs vertically through the neg_x face, the central meridian runs vertically through the pos_x face, and the center of the cylindrical texture is located in the center of the pos_x face. A scan of the cubemap texture is standard:

The cubemap texture

Some planetary maps are not centered on the zero meridian. For example, it may run onto the left boundary of the texture. In this case, you can set the longitude offset of 180°, to “rotate” the resulting cubic texture to the correct position: InputLatOffset 180.

The resolution of the cubemap faces is calculated as a quarter of the width of the original texture, rounded up to the nearest power of 2. For example if the original texture resolution is 86400 x 43200, it will give the resolution of each face of 32768 x 32768 (131072/4).

Each face is organized as a set of tiles. The resolution of the tile is determined by the parameter TileWidth and must be no more than the resolution of the face. For example, if TileWidth = 512, and the resolution of the face is 32768, then we’ll have 7 levels of tiles: Log2(32768/512) + 1 = 7. The levels are numbered from 0 to 6; the number of levels is the first digit in the file name of the tile: (0_0_0.jpg, …, 6_63_63.jpg). The total resolution of each level is doubled sequentially, and the number of tile files is quadruplicating. Thus, we will have one tile of level 0 (0_0_0.jpg) with resolution of 512 x 512. It will have the reduced image of the whole face. There will be 4 tiles of level 1 (1_0_0.jpg … 1_1_1.jpg), each comprising a quarter of the entire face image. Resolution of each tile is again 512 x 512, thus the common resolution of level 1 will be 1024 x 1024. Level 2 will have 16 tiles, and so on. The last (most detailed) level (6) will have 64 * 64 = 4096 tiles (512 x 512 each), with total resolution of 32768 x 32768. The second and third digits in the tile’s file name are vertical (v) and horizontal (u) indices. Indexing is carried out from the left to the right and from the top to the bottom. Here is the example picture of the first three levels (with tile resolution 128 x 128):

level 0
Level 0

level 1
Level 1

level 2
Level 2

The CubeMap utility

Conversion of the source cylindrical texture to the tiled cubemap texture is performed in several stages. It is possible to perform conversion for certain faces only. This is controlled by the following parameters in the configuration file (setting the parameter to false or 0 disables all stages for the corresponding face):

Code
Create_NEG_X true
Create_POS_X true
Create_NEG_Y false
Create_POS_Y false
Create_NEG_Z true
Create_POS_Z true

Execution of any stage can be disabled by setting the appropriate parameter in the config to false or 0 (see below):

Code
CreateBaseTex   true    # create a base texture
ComputeFaces    true    # process selected faces (compute temporary raw-files of the latest level)
DownsizeFaces   true    # compute LODs of selected faces (compute temporary raw-files of other levels back to 0)
UpdateEdges     true    # update the face edges
TileFaces       true    # split temporary raw-files of the selected faces into tiles
OptimizeTiles   false   # delete homogeneous tiles

Lets look at all the steps in order.

1) Creating a base cylindrical texture

To perform this operation, the configuration file must have CreateBaseTex true.

SpaceEngine uses a special small cylindrical texture (called “base texture”) to render planets when they are far away from the camera. Creating this texture is achieved by a simple downsampling of the source cylindrical texture by averaging the pixels in square blocks. The BaseTexDownSize parameter specifies how many times to reduce the source texture, and BaseTexFormat parameter specifies the format of the resulting base texture. For example, reducing the texture of 10800 x 5400 four times gives a texture of 2700 x 1350. For SpaceEngine it’s better to use a base texture of resolution of 512 x 256. But the Cubemap supports only integer numbers for BaseTexDownSize, so not all source textures can be converted to a 512 x 256 base textures. For example, 10800 / 512 = 21.09375, so a close possible value for BaseTexDownSize is 21, which yields to 10800 / 21 = 514. So it’s necessary to downscale the final base texture again using a third-party program, such as Photoshop. To save precision, make a base texture of resolution ~ 1000×500, then downscale it in the third-party program to 512×256.

2) Computing the faces of the cubemap texture

To perform this operation, the configuration file must have ComputeFaces true.

The conversion is performed by bilinear filtering: to derive the color of the current pixel in the face, the floating-point coordinates of the corresponding point in the cylindrical texture are computed, and the color is obtained by a linear interpolation of the four pixels closest to that point. This method yields very good results; the artifacts may be noticeable only at the poles. Setting up the DoublePrecision true slightly modifies the algorithm: the result color is obtained by averaging the four values obtained as described above, but with a doubly-detailed mesh.

The program uses a special cache in the RAM for storing pieces of the original cylindrical texture. Therefore it is possible to convert a very large source texture without a complete loading of it into the memory. For example, NASA’s texture for Earth, the Blue Marble Next Generation, has an unpacked size (in the raw format) of 86400 x 43200 x 3 = 10.4 GB. The cache size is set by the parameter MaxMem (size in MB). In the current version (1.06) setting up a large value (greater than 512 MB) can reduce the processing speed because it reduces long searches in the cache. The optimum value is in the range of 64-512 Mb, and depends on the size of the source texture to a small degree.

There are two types of cache – the Line and the Quad. Choosing a type is performed by the MethodSide parameter for the side faces of the cubemap (neg_x, neg_z, pos_x, pos_z) and by the MethodPolar for the polar faces (neg_y, pos_y). The allowed values are Line and Quad (without the quotes). The Line method reads the source file by small row pieces, while the Quad methods reads small square pieces. The Quad method should be more effective for the polar faces, but in the current version (1.06) there no significant difference.

A temporary subfolder called raw is created in the temporary folder specified by the TempFolder parameter. Inside it, subfolders neg_x, neg_y, neg_z, pos_x, pos_y, pos_z are created, where raw-files of the corresponding cubemap faces are saved. In our example, each face’s raw-file will have a name of level6.raw (because we will have 7 levels, starting from 0), with resolution of 32768 o 32768.

If the TileBorder parameter is non-zero, additional pixels are added on the boundary of each tile. These pixels are a copy of the boundary pixels of the adjacent tiles. The width of this additional border is defined by this parameter. Thus if the configuration file has TileWidth 512 and TileBorder 1, we will obtain the tiles of resolution of 514 x 514. The resolution of the temporary file level#.raw will be increased by 2 TileBorder * cubeFaceWidth / TileWidth, where cubeFaceWidth is the original resolution of the level#.raw file. In our example, level 6 will have a resolution of 32768 + 128 = 32896. These extra pixels allow the elimination of gaps in the landscape or sharp edges at the joints of tiles when rendering the planet in SpaceEngine. For terrain elevation maps, it is important to save the tiles to a lossless format such as 16-bit png. In the previous versions of CubeMap a boolean parameter AddExtraData was used instead of TileBorder (AddExtraData true was equal to TileBorder 1).

The elevation maps (InputChannels 1) can use normalization, i.e. changing values so they implement the full dynamic range of the bit-depth. For example, the 16-bit elevation map gebco_bathy.21601×10801.bin has a minimum and maximum pixel value of -10577 and 8430 respectively, which does not fully implement the dynamic range of 16-bit integers (-32768 … 32767). Specifying the Normalize true enables normalization. In that case, if the parameter EnterNormData is set to false, the minimum and maximum values are searched for by scanning all pixels in the source cylindrical texture. For large textures this can take considerable time, so it is enough to perform this search once, then set EnterNormData true and enter the data that the program derived in the previous launch:

Code
Normalize      true   # normalize the output image (the height map)
EnterNormData  true   # true to enter the normalization data manually, false to find them automatically
NormMinValue  -10577  # minimum pixel value
NormMaxValue   8430   # maximum pixel value
3) Computing a LODs of faces

To perform this operation, the configuration file must have DownsizeFaces true.

A consequential 2-times downscaling of the temporary raw-file is performed, resulting in the creation of the files (in our example) level5.raw, level4.raw, level3.raw, level2.raw, level1.raw, level0.raw in the temporary folder, with a resolution of 16384 x 16384 to 512 x 512 respectively.

If the TileBorder is non-zero, the resolution of the new temporary files level#.raw will be increased by 2 * TileBorder * cubeFaceWidth / TileWidth, where cubeFaceWidth is the original resoluton of the level#.raw file.

4) Updating of the face borders

To perform this operation, the configuration file must have UpdateEdges true and TileBorder set to non-zero.

If adding of the additional frame pixels is enabled (TileBorder is non-zero), to obtain the correct values of pixels located at the boundary of the face, the Cubemap need to take pixels from the adjacent faces. To do this, all the faces must be computed, operations ComputeFaces and DownsizeFaces must be done for all faces. Otherwise the program will crash.

5) Splitting the temporary raw-files into tiles

To perform this operation, the configuration file must have TileFaces true.

The temporary files are split into tiles with resolution of TileWidth (or TileWidth + 2 * TileBorder, if the TileBorder is non-zero), and are saved to the folder named after the face, in one of the following formats:

raw – 8 or 16 bits, any number of channels, no compression
tga – 8 bits, 1, 3, 4 channels, no compression
dds – 8 bits, 1, 3, 4 channels, no compression
jpg – 8 bits, 1, 3, channels, lossy compression
tif – 8 or 16 bits, 1, 3, 4 channels, lossy compression
png – 8 or 16 bits, 1, 3, 4 channels, lossless compression

The tile format is described by the following parameters:

OutFormat – the tile format (raw, tga, dds, jpg, png)
OutFormat2 – the tile format (raw, tga, dds, jpg, png) for the low-byte, or for the alpha-channel (see SeparateHiLo and SeparateRGBA)
TileWidth – the tile resolution
TileBorder – the width of a border with additional pixels
Out16bit – capacity (bit-depth): 16 or 8 bits per channel (16 is allowed only for raw and png)
OutByteSwap – for 16-bit only: true for little-endian (MAC), false for big-endian
OutUnsigned – for 16-bit only: true for unsigned, false for signed value
OutInvertAlpha – invert the alpha channel for the RGBA images
OutJPEGquality – the quality for saving to JPEG (0…100)
SeparateRGBA – save RGBA tiles as a two separate files: RGB and alpha
SeparateHiLo – save 16-bit tiles as a two separate files: the high-byte and the low-byte

It is necessary to dwell on the parameters OutFormat2, SeparateRGBA and SeparateHiLo. SpaceEngine supports a special way to store RGBA textures for the planetary surface in the form of two separate texture files – a color file for the RGB components and a black-and-white file for the alpha channel. This can be used to save disk space. For example, Earth’s surface requires RGBA textures with color specified in the RGB channels, and a water mask specified in the alpha channel. The compressed texture formats which support 4 channels (RGBA) are only png and tif. But files of these formats are quite large, while the color information inside could be compressed in jpg with a sufficiently good quality, and the water mask could be compressed in the lossless png very well. Such a combined set of two textures jpg + png takes much less space than a single 4-channel png. To use this method, set the SeparateRGBA true, and specify the format for the RGB texture by the OutFormat, and for the alpha texture by the OutFormat2:

Code
OutFormat    jpg
OutFormat2   png
SeparateRGBA true

With this configuration, the CubeMap will save two files for each tile: the RGB file with the suffix _c and the alpha file with the suffix _a, for example:
3_2_16_c.jpg
3_2_16_a.png

For the 16-bit elevation maps, it can also be used for the separation of the 16-bit tile image into two files: a low-byte and a high-byte. Sometimes saving them separately into 8-bit files gives an advantage in size compared to saving in a single 16-bit image (png or tif). To do this, set the SeparateHiLo true, and specify the format for the high-byte texture by the OutFormat, and for the low-byte texture by the OutFormat2:

Code
OutFormat    png
OutFormat2   jpg
SeparateHiLo true

With this configuration, the CubeMap will save two files for each tile: the high-byte file with the suffix _h and the low-byte file with the suffix _l, for example:
3_2_16_h.png
3_2_16_l.jpg

Note: the high-byte file must be saved in a lossless format, such as png. Otherwise, the terrain will have large artifacts! The lower-byte file may be saved in a format with lossy compression, such as jpg, but it also leads to artifacts. Only the combination png + png does not result in artifacts, and sometimes it leads to a better file size; in this case then, this is the recommended method to use.

SpaceEngine automatically detects and loads the dual-file textures, no additional tweaking of the planet script is required.

If the tile files already exist in the folder, they are overwritten.

6) Optimizing tiles

To perform this operation, the configuration file must have OptimizeTiles true.

SpaceEngine supports the tiled cubemap textures with any level of detail at any place. This means that you can, for example, have an Earth texture with a resolution of 6 x 32768 x 32768 (obtained from Blue Marble Next Generation), and in some places such as the Grand Canyon, the level detail can be improved by adding additional tiles with higher levels. Or vice versa, the large homogeneous areas with no detail – the oceans – can have only lower-level tiles (from 0 to 2) with the rest removed, which saves disk space and improves loading time. The Cubemap can try to identify such homogeneous areas and remove them. Thus, it is recommended to use this optimization only with textures with large homogeneous areas such as the oceans; it is not recommended to do it for other textures.

Detecting of homogeneity is done by calculating the variation of the pixel colors across the tile (the weighted sum of the absolute value of the difference between the color of a pixel and the average color of the tile). If this variation is less than the value specified by the parameter MinVar in the configuration file, the tile is considered homogeneous and is moved to a temporary location.

The subfolder called optimized is created in the temporary folder specified by the TempFolder parameter. Inside it, subfolders called neg_x, neg_y, neg_z, pos_x, pos_y, pos_z are created. Any detected homogeneous tiles are moved into those subfolders. They are candidates for removal.

General guidelines for optimization are the next. After generating the tiles without optimization (OptimizeTiles false), start from some value MinVar and run CubeMap with OptimizeTiles true while the rest of the stages are disabled (parameters CreateBaseTex to TileBaces are set to false). It is better to do that for only one face, choosing it by the parameters Create_NEG_X … Create_POS_Z. Next, look at which tiles were moved to the temporary folder. If not you are not satisfied with the results, move the tiles back (manually), adjust the MinVar value, and repeat until you get an acceptable result. Then do the optimization of the other faces using the optimal value of the MinVar you have found.

The current version (1.06) does not use a good algorithm, so errors are possible – e.g. tiles with a small island and homogeneous ocean around it can be considered as homogeneous, while tiles with a slight color gradient can be considered as heterogeneous. Therefore it is recommended to look at all the tiles in the temporary folder and move back those which seem to not be suitable for removal. It is also recommended to save these removed tiles somewhere, so you can quickly restore incorrectly deleted tiles, without having to launch a full conversion process with the CubeMap again (especially if the source cylindrical texture is large).

Optimization is not supported for the dual-file tiles (SeparateRGBA or SeparateHiLo).

The Glue utility

The Glue utility is designed for glueing up the tiles back to a single raw-file. The utility has its own configuration file with the following parameters:

InputFolder – path to a folder with tiles
OutFile – path to an output raw-file
TilePrefix – string, a prefix in the file names of the tiles (the level number)
TileExt – string, the file extension for the tiles
StartU – initial U index
StartV – initial V index
SwapUV – swap U and V indices
TileWidth – tile width
TileHeight – tile height
OutWidth – output raw-image width
OutHeight – output raw-image height
ChannelsPerPixel – number of channels (Grayscale – 1, RGB – 3, RGBA – 4, etc.)
BytesPerChannel – bytes per channel: 1 for 8-bit, 2 for 16-bit, etc.)
ConvertToGrayscale – convert to grayscale (for 8-bit only)
MaxMem – memory cache size in MB

The tile file name takes the form of: TilePrefix_v_u.TileExt; for example, for this configuration file:

Code
TilePrefix 5_
TileExt    png
StartU     2
StartV     3
SwapUV     false

The first tile will have a name 5_3_2.png; the names of the next tiles will be obtained by increasing the indices u and v until resolution of the output texture specified in the configuration file is reached. Using an extension of * indicates that the program should look for tiles in any of the supported formats, in the following order: dds, png, jpg, jpeg, tif, tiff, tga, raw. For example, if the folder contains the two files 3_1_2.jpg and 3_1_2.tga, the first one will be loaded.

If the extension does not match one of the above, the tool switches to the raw-mode: the input tiles are considered to have the raw format, i.e. a simple uncompressed pixel arrays without a title. The extension can be .raw, .img or any other, renaming of the tiles to the .raw is not required. In raw-mode, the number of channels (ChannelsPerPixel) and the capacity (BytesPerChannel) are not limited. It can be used for glueing of arbitrary arrays of data (not necessarily textures), as if they are a rectangular blocks. Tiles do not necessarily have to be square-shaped – thus the separate parameters TileWidth and TileHeight are provided.

The parameter ConvertToGrayscale converts multi-channel tiles (RGB, RGBA and so on) into a grayscaled raw-texture by a simple averaging of channel values. It is only allowed for 8-bit tiles.

General recommendations

1) Placing the source texture, the temporary folder and output folder on different physical drives can significantly increase performance. Performance is also increased if the source file and temporary raw-files are defragmented.

2) Write the information about the format of the source raw-texture in its name, for example: Earth-surface-32k-RGBA.raw is an RGBA texture of 32768 x 16384; Mars-bump-16k-16bit.raw is a 16-bit elevation map of 16384 x 8192. This will help to avoid confusion.

3) Conversion of the texture of any format to raw format can be done in the Photoshop, IrfanView and other programs. Using them can also view the temporary raw-files (if you have enough RAM so that these programs are able to load large raw-files).

4) Save the configuration files for each converted texture, they can by useful in the future. Associate the .ccf file extension with the program CubeMap.exe, and the .gcf with the Glue.exe in the operating system, then double-clicking on a configuration file will automatically launch the corresponding utility. You can also drag and drop the configuration file onto a shortcut of the CubeMap or Glue.

5) The CubeMap can be interrupted by the Ctrl+C combination, and continue its work from the same point on the next launch. For example, if you do not know the format of a 16-bit elevation map, i.e. which values should be specified in the parameters InputByteSwap and InputUnsigned, first you can make conversion of one face and look at the result (tiles). But if the texture is very large, such a test will take a long time. Therefore, you can interrupt the conversion at the ComputeFaces stage, and view the finished part of the temporary file in the Photoshop or another program, or set the ComputeFaces false and run CubeMap again, waiting for the generation of some tiles.

6) It is recommended to view the result of the conversion of the polar faces pos_y and neg_y, and, if there are noticeable artifacts present, try and set DoublePrecision true, or edit the raw-files of the polar faces in Photoshop or another program. To do this, run the ComputeFaces for polar faces only, edit the raw-files, disable ComputeFaces and perform the remaining operations.

7) In order to avoid loss of precision it is not recommended to normalize the height map, the dynamic range of which is “almost implemented”: for example (1 … 252) for the 8-bit and (-32000 … 32000) for the 16-bit.

8) If you want to save the tiles into a format not supported by Cubemap, it is necessary to perform the conversion into a lossless format such as tga, and then perform a batch conversion of all tiles into the desired format using third-party software such as IrfanView or ImageMagic.