The RenderWare binary file format is a standard tagged format that is built on the RwStream concept. A chunk is made up of a chunk tag, then a chunk header containing at least the size of the chunk and finally the chunk data. A stream may contain many chunks, perhaps with chunks embedded in other chunks.
The format of a binary file is as follows
| Chunk Type |
| Chunk Header |
| Chunk Data |
| Next Chunk Type |
| Next Chunk Header |
| Next Chunk Data |
| etc. |
The chunk type and header are currently only 32 bit identifiers which indicate the tag and size respectively of the chunk. These should always be accessed via the relevant API functions however to allow for future expansion of these fields and the associated backwards and forwards compatibility issues.
For compatibility across all platforms the length of a chunk should always be a multiple of 4.
Standard RenderWare chunks are prefixed with rwCHUNK and followed with the type. For example a light has a chunk identifier rwCHUNKLIGHT.
For example a rwCHUNKSTRING for a string "Hello" is of the following format
rwCHUNKSTRING
8
H,e,l,l,o,\0,\0,\0
The length of the chunk given after the chunk type, can be used to skip all of the information held with the chunk. When performing a chunk read with RwReadStreamChunk(), the stream must be pointing to the header entry. That is the chunk type must have been read. For an example of a more complicated chunk structure here is the structure of a rwCHUNKCAMERA
|
|
|
|
|
|
For most uses of the binary file format there is no need to know the details of how RenderWare saves types. But when a user creates their own chunks it is important that the structure of the saved chunk is correct.
Writing a standard RenderWare type to a stream simply requires a call to the RwWriteStreamChunk() function as follows:
{
RwStream *stream;
stream = RwOpenStream(rwSTREAMFILENAME, rwSTREAMWRITE,
"clump.rwb");
if (stream)
{
if (!RwWriteStreamChunk(stream, rwCHUNKCLUMP, clump, 0))
{
/* Error writing the clump chunk */
RwCloseStream(stream);
return(FALSE);
}
RwCloseStream(stream);
}
else
{
printf("Error opening the stream\n");
return(FALSE);
}
}
The RwWriteStreamChunk() function takes four parameters. The first is the stream to be written to the second is the chunk type the third is a chunk type dependent parameter and the last is a flag field for requesting what should be saved in the chunk. The flags that are currently supported are listed in the next section.
It is very important when writing chunks that the correct chunk dependent parameters are given - the results otherwise are undefined and could potentially crash the machine.
Reading of chunks is a similar process to writing them although it is essential that the stream is at the correct position before reading occurs. Three functions can be used to help with this. The simplest method is to use RwFindStreamChunk() which will search for a particular chunk type. Once found it will leave the stream in the correct position for chunk reading.
{
RwStream *stream;
FILE *fp;
RwLight
*light;
fp =
fopen("light.rwb", "rb+");
if (!fp)
{
return(FALSE);
}
stream =
RwOpenStream(rwSTREAMFILE, rwSTREAMREAD, fp);
if (stream)
{
if (RwFindStreamChunk(stream, rwCHUNKLIGHT))
{
/* The chunk has been found, we can now read it */
if
(RwReadStreamChunk(stream,
rwCHUNKLIGHT, &light, 0)))
{
/* Stream has been correctly read */
RwAddLightToScene(scene, light);
}
}
RwCloseStream(stream);
}
else
{
/* The stream was not opened */
return(FALSE);
}
fclose(fp);
return TRUE;
}
The other method uses the RwReadStreamChunkType() and RwSkipStreamChunk() functions. RwReadStreamChunkType() reads a chunk identifier from the stream. It does this just by reading a number of bytes equal to the identifier length and then converting the data read into a form suitable for the platform RenderWare is running on.
RwSkipStreamChunk() reads the chunk length from the header and then skips that number of bytes. For RwSkipStreamChunk() to function correctly it is essential that the stream is correctly positioned at the start of a chunk header. This will be the case if the chunk type has previously been read via one of the RwFindStreamChunk() or RwReadStreamChunkType() functions.
The following example looks for an rwCHUNKMATRIX chunk within a stream
{
RwStream *stream;
stream = RwOpenStream(rwSTREAMFILENAME,
rwSTREAMREAD, "stream.rwb");
if (stream)
{
RwInt32 nChunkType;
if
(!RwReadStreamChunkType(stream, &nChunkType))
{
if (nChunkType == rwCHUNKMATRIX)
{
/* The chunk has been found */
return TRUE;
}
if
(!RwSkipStreamChunk(stream))
{
/* The skip has caused an error -> probably a
badly structured stream */
return
FALSE;
}
}
else
{
printf("Unable to open the stream. \n");
return FALSE;
}
/* The chunk was not found */
return
FALSE;
}
The contents of a chunk
The actual contents of a RenderWare chunk depends on the type of chunk you are using. The data is saved such that the data for the chunk type can be reconstructed efficiently. This is significant especially when it comes to textures. When the binary file format saves off a texture it does so in the same raw file format that the texture is held in memory. This raises two significant points
· The data for a raster is always at a single depth. This will be the render depth that was in effect when it was saved.
· The data is held in the format suitable for rendering on a particular platform.
When RenderWare loads a raster from the native texture file format, some conversion will take place to translate the raster into a form that can be used for rendering. For example if RenderWare has started in an 8 bit rendering mode, a subsequently loaded texture will be palette matched down to an 8 bit image (irrespective of the depth of the source image). Also the texture will be scaled to the texture map size being used. The binary file format only saves textures in the form they are stored in memory. This has the following significant effects when handling rasters with the binary file format
· A raster should only be read from the binary file format to RenderWare operating at the same depth as it was stored as.
· A raster used as a texture should only be read if RenderWare has a driver which is operating at the same texture map width and height as when the raster was saved off.
· A raster used in 8 bit modes should only be loaded if the palette used is the same as when the raster was written off.
Typically a binary file format should have a palette saved before any raster is saved in 8 bit
These limitations are present in order to allow rasters to be loaded very quickly - as no translation from color spaces is necessary, or any scaling. To avoid problems if the binary file is to be used in many different environments, there are facilities provided to locate textures automatically in a similar fashion to that used for the ASCII file format. This is done by looking up textures in the texture dictionary or by loading textures into the texture dictionary automatically from their native format.
When a texture is saved using the binary file format its name in the texture dictionary will be saved off also. When the texture is read, the name of the texture is checked in the texture dictionary. If it is there the texture in the dictionary will be used irrespective of whether there is a raster embedded in the stream.
If the name of the texture is not in the texture dictionary, RenderWare will try and load the texture from a file if there isn't an embedded raster. Obviously the loading of a texture will mean that potentially the image will have to be resized and placed in the correct color space, and thus will load far more slowly than a raster which has been saved in the binary file format.
This checking of the texture dictionary and reading can be overwritten with the rwLOADCHUNKRASTERS flag.
The default information saved in a chunk is the minimum that can be saved. The following example will save and reload a clump, but the raster for the texture will not be saved in the file although the name of the texture will.
{
RwClump *clump;
RwStream *stream;
/* Note
that all textures loaded via RwReadShape will be
placed in the texture dictionary */
clump =
RwReadShape("shape.rwx"); /* script which
references a
texture */
stream =
RwOpenStream(rwSTREAMFILENAME, rwSTREAMWRITE,
"shape.rwb");
if (stream)
{
/* Need to save off UV coordinates as this object is
textured. Don't save off the rasters - we’ll get
them
from the dictionary */
if(!RwWriteStreamChunk(stream, rwCHUNKCLUMP,
clump, rwSAVECHUNKUVS))
{
printf("Unable to write the binary clump.\n");
RwCloseStream(stream);
return FALSE;
}
/* Close the write stream */
RwCloseStream(stream);
}
else
{
printf("Unable to open write stream\n");
return FALSE;
}
return TRUE;
}
Chunk Scope
When a chunk is saved it will try and save off all the information associated with it so that it can later, when read, be fully reconstructed. RenderWare provides the concepts of a texture dictionary and materials by reference. These aim to provide flexibility but also to minimize resource usage.
The idea of saving off everything that a chunk requires so it can be rebuilt means that everything in the 'scope' of an object must also be saved. For example if a chunk is saved then all the materials, all of the textures and all of the rasters associated with the clump must be saved also. RenderWare tries to minimize repetition of data by two methods. Firstly by saving texture names and secondly by having scope. Scope in this context means everything that is required by the chunk to be rebuilt. Thus if a there are two clumps referencing one material, and the clumps are saved off one by one the material will be saved twice - because the material is referenced by the scope of each object. If the clumps are added to a scene and the scene saved off, the material will be saved only once for both clumps, as the material is only accessed once in the scope of a scene.
By saving texture names (the name a texture has in the dictionary) RenderWare can check the current dictionary, and if the texture is already present it ignores any saved texture in a binary file.
To save materials by reference, RenderWare will save off with a clump all of the materials referenced by a chunk. This means that if there are two clumps which share a single material and the clumps are saved one after another, the materials will be saved twice and when the clumps are later reloaded they will have their own copy of a material.
Clump A Clump B
\ /
\ /
\ /
Material 1
When Clump A is saved the information saved will be
| Material 1 |
| Clump A |
When Clump B is saved the information saved will be
| Material 1 |
| Clump B |
When both clumps are loaded the materials will be referenced
Material 1 Material 2
| |
| |
Clump A Clump B
Where material 1 and material 2 will be exactly the same.
This has two problems
· More resources are used up
· If I change Material 1 it will only effect clump A, whereas previously it would effect both clumps.
To get around this problem it is necessary to save off a chunk with scope covering both clumps. A scene is such a type. Therefore if both clumps are added to a scene, the scene is saved and then loaded, the material will still be shared by both clumps. Thus if a 'dummy' scene is created and all of the chunks are added to this scene, and it is saved, the scope will cover both clumps and thus the material will be correctly shared.
When the scene is read the clumps can be removed from the scene (and added to the required scene) and the dummy scene can be destroyed.
For example, the following section of code, creates two clumps which share a material. These clumps are then added to a dummy scene and then saved off so that their material remains shared.
static
RwClump *
SceneClumpRecurse(RwClump *clump, void *data)
{
RwScene *scene = (RwScene *)data;
RwAddClumpToScene(scene, clump);
return
clump;
}
static
RwBool
Scope(void)
{
RwClump *clumpa, *clumpb;
RwMaterial *material;
RwScene *dummyscene;
RwModelBegin();
RwClumpBegin();
RwSetSurface(CREAL(0.75), CREAL(0.0), CREAL(0.0));
RwSetSurfaceColor(CREAL(1.0), CREAL(1.0), CREAL(0.0));
RwSetSurfaceGeometrySampling(rwSOLID);
RwBlock(CREAL(1.0), CREAL(1.0), CREAL(1.0));
RwClumpEnd(&clumpa);
RwModelEnd();
clumpb = RwDuplicateClump(clumpa);
/* We have two clumps which share materials */
RwPushScratchMatrix();
RwTranslateMatrix(RwScratchMatrix(),
CREAL(1), CREAL(0), CREAL(0), rwREPLACE);
RwTransformClump(clumpa, RwScratchMatrix(), rwREPLACE);
RwTranslateMatrix(RwScratchMatrix(), CREAL(-1),
CREAL(0), CREAL(0), rwREPLACE);
RwTransformClump(clumpb, RwScratchMatrix(), rwREPLACE);
RwPopScratchMatrix();
/* Add the clumps to the scene */
dummyscene = RwCreateScene();
if
(!dummyscene)
{
RwDestroyClump(clumpa);
RwDestroyClump(clumpb);
}
RwAddClumpToScene(dummyscene, clumpa);
RwAddClumpToScene(dummyscene, clumpb);
/* Find the material assigned */
/* I know
all the tags are set to 0 initially and that
all the polygons are using the same material, therefore
finding a polygon with a tag of 0s material will be the
material used by both clumps */
material = RwGetPolygonMaterial(RwFindTaggedPolygon(clumpa, 0));
RwSetMaterialColor(material, CREAL(1), CREAL(0), CREAL(0));
/* Save off
the dummy scene to a stream */
{
RwStream *stream;
stream =
RwOpenStream(rwSTREAMFILENAME, rwSTREAMWRITE,
"scope.rwb");
if
(!stream)
{
RwDestroyScene(dummyscene);
return FALSE;
}
/* Write off the dummy scene */
if
(!RwWriteStreamChunk(stream, rwCHUNKSCENE,
dummyscene, 0L))
{
RwCloseStream(stream, NULL);
RwDestroyScene(dummyscene);
return FALSE;
}
/* Close the stream */
RwCloseStream(stream, NULL);
}
/* Destroy the scene and the clumps */
RwDestroyScene(dummyscene);
/* Read back the scene and the clumps */
{
RwStream *stream;
stream = RwOpenStream(rwSTREAMFILENAME,
rwSTREAMREAD, "scope.rwb");
if
(!stream)
{
printf("Unable to read stream\n");
return(FALSE);
}
/* Find the scene in the stream */
if
(!RwFindStreamChunk(stream, rwCHUNKSCENE))
{
RwCloseStream(stream, NULL);
printf("Unable to find the scene chunk\n");
return FALSE;
}
/* Read back the scene */
if
(!RwReadStreamChunk(stream, rwCHUNKSCENE,
&dummyscene, 0))
{
RwCloseStream(stream, NULL);
printf("Unable to read the scene from the
stream\n");
return FALSE;
}
/* Close the stream */
RwCloseStream(stream, NULL);
/* Remove
the chunks */
}
/* Check that the material is actually shared */
{
RwMaterial *material;
/* Test that the material is shared */
material = RwGetPolygonMaterial(RwFindTaggedPolygon(
RwFindTaggedClump(Scene,0),0));
RwSetMaterialColor(material,
CREAL(1), CREAL(0), CREAL(0));
}
/* Put the clumps in the normal scene */
RwForAllClumpsInScenePointer(dummyscene,
SceneClumpRecurse, RwDefaultScene());
/* Can destroy the dummy scene */
RwDestroyScene(dummyscene);
return TRUE;
}
Write Flags
The chunk write flags are used when calling RwWriteStreamChunk(), they are passed as the last parameter of the function. Multiple flags can be set by using the C OR operator (|) . If none of the extra information needs to be saved in a chunk the flags should be set to 0.
Note that the flags only effect certain chunk types. For example rwSAVECHUNKNORMALS only effects rwCHUNKSCENE and rwCHUNKCLUMP chunk types.
The chunk flags passed will be saved with chunks which have used this information. When the chunk is read back, the correct information will automatically be read back from the chunk. The rwSAVECHUNK flags should only be used when using RwWriteStreamChunk() or RwGetChunkSize(), they should never be used when reading a chunk.
• rwSAVECHUNKNORMALS Save off polygon and vertex normals. This will only affect rwCHUNKSCENE and rwCHUNKCLUMP chunk types.
• rwSAVECHUNKUVS Save off vertex texture 'u' and 'v' coordinates. This will only affect rwCHUNKSCENE and rwCHUNKCLUMP chunk types.
• rwSAVECHUNKLUMINANCES Save off polygon and vertex lighting luminance values. This will only affect rwCHUNKSCENE and rwCHUNKCLUMP types.
• rwSAVECHUNKRASTERS Save off any rasters. This will only effect rwCHUNKSCENE, rwCHUNKCAMERA, rwCHUNKCLUMP, rwCHUNKMATERIAL and rwCHUNKTEXTURE chunks.
• rwSAVECHUNKTAGS Save off clump and polygon tags. This will only affect rwCHUNKSCENE and rwCHUNKCLUMP chunk types.
• rwSAVECHUNKALL Is the combination of all these flags and so will save off all information possible for any chunk type.
Read Flags
These need only be set when a chunk is read via the RwReadStreamChunk() function. The last parameter holds the required read flags.
• RwLOADCHUNKRASTERS Forces rasters embedded in textures in the input stream to be extracted even if a reference to an already loaded texture has been found.
When a texture is saved off, the name in the texture dictionary will be saved off with it also. To improve performance and to allow a set of textures to be loaded before the binary chunks are read, normally when a texture is found its name is first looked up in the current texture dictionary. If the name is there that texture will be used - and the raster for the texture embedded in the stream (if there is one) will be ignored. This flag will force the raster in the stream to be used.
Chunk types
The following lists the chunk types currently supported by the binary file format. All of the major RenderWare types are supported.
• rwCHUNKCAMERA data should be a RwCamera *. Setting flags to rwSAVECHUNKRASTERS will cause the cameras backdrop raster to be saved.
• rwCHUNKMATRIX data should be a RwMatrix4d *. flags should be 0.
• rwCHUNKMATERIAL data should be a RwMaterial *. flags should be 0.
• rwCHUNKPALETTE data should be NULL on write. data should be a RwPalette on read. On success data will contain a valid RwPalette. flags should be 0.
• rwCHUNKRASTER data should be a RwRaster *. flags should be 0.
• rwCHUNKTEXTURE data should be a RwTexture *. rwSAVECHUNKRASTERS will cause the textures raster to be saved off on the stream. On reading the stream if a texture is already present in the texture dictionary with the same name as the texture being loaded the texture present will be returned. If rwLOADTEXTURERASTERS is passed when reading the stream, then a raster must be on the stream and it will be made the texture. Note if the textures name is already in the dictionary the texture will be placed in the dictionary with an empty name.
• rwCHUNKCLUMP data should be a RwClump *. Any of the write or read flags can be used to modify the operation of rwCHUNKCLUMP.
• rwCHUNKLIGHT data should be a RwLight *. flags should be 0.
• rwCHUNKSCENE data should be a RwScene *. Any of the write or read flags can be used to modify the operation of rwCHUNKCLUMP.
• rwCHUNKDATA data should be a RwMemory *. flags should be 0.
• rwCHUNKRECT data should be a RwRect *. flags should be 0.
• rwCHUNKV3D data should be RwV3d *. flags should be 0.
• rwCHUNKSTRING data should be a char *. flags should be 0.
• rwCHUNKUSER This chunk type is not directly supported by RenderWare. The developer may use it for creating their own chunk types. All chunk types of rwCHUNKUSER + n where n is a number between 0 and 1000 may be used by the user for their own data.
Writing User Chunks
Chunks may be created by the user, and may contain arbitrary types of data. RenderWare defines a range of chunk types to identify user chunks. These are identified by rwCHUNKUSER + n where n is a number between 0 and 1000. All other chunk identifiers are reserved for use by RenderWare.
RenderWare chunks can be embedded within user chunks also. This brings up the problem of how big the final user chunk will be. To find out the size of a RenderWare chunk the function RwGetChunkSize() can be used. This will return the size of the chunk in bytes as it would be written to the stream. Using this, the size of a user chunk can be easily calculated by adding up the sizes of the contents of the used chunk.
After a user chunk has been found it is necessary to read the remainder of the chunk header - this will return the length of the entire chunk saved.
/* Find the chunk identifier */
if
(!RwFindStreamChunk(stream, rwCHUNKUSER))
{
printf("Chunk not found\n");
return FALSE;
}
/* Read the rest of the chunk mark */
length = RwReadStreamChunkHeader(stream);
if (!length)
{
printf("Error reading the chunk mark\n");
return FALSE;
}
/* Now the contents of the chunk can be read */
The following example writes a user chunk to the stream containing both a clump and a light, and then reads them back again.
{
RwClump *clump;
RwLight *light;
RwStream *stream;
/* Build a test clump to save */
RwModelBegin();
RwClumpBegin();
RwSetSurface(CREAL(0.75), CREAL(0.0), CREAL(0.0));
RwSetSurfaceColor(CREAL(0.0), CREAL(1.0), CREAL(1.0));
RwSetSurfaceGeometrySampling(rwSOLID);
RwSphere(CREAL(1.0), 2);
RwClumpEnd(&clump);
RwModelEnd();
/* Build a test light */
light =
RwCreateLight(rwDIRECTIONAL,
CREAL(1), CREAL(0), CREAL(0), CREAL(1));
/* Open a stream */
stream =
RwOpenStream(rwSTREAMFILENAME,
rwSTREAMWRITE, "user.rwb");
if
(!stream)
{
printf("Unable to open the stream\n");
return FALSE;
}
/* Write the user stream mark */
if
(!RwWriteStreamChunkHeader(stream,
rwCHUNKUSER,
RwGetChunkSize(rwCHUNKLIGHT, light, 0)+
RwGetChunkSize(rwCHUNKCLUMP, clump, 0)))
{
printf("Unable to save chunk header\n");
return FALSE;
}
/* Write off the chunks */
if
(!RwWriteStreamChunk(stream, rwCHUNKLIGHT, light, 0))
{
printf("Unable to write light chunk\n");
return FALSE;
}
if
(!RwWriteStreamChunk(stream, rwCHUNKCLUMP, clump, 0))
{
printf("Unable to write clump chunk\n");
return FALSE;
}
/* Close the stream */
if
(!RwCloseStream(stream, NULL))
{
printf("Unable to close the write stream\n");
return FALSE;
}
/* Destroy the objects ready for reading ! */
RwDestroyLight(light);
RwDestroyClump(clump);
/* Read the user clump back */
stream =
RwOpenStream(rwSTREAMFILENAME,
rwSTREAMREAD, "user.rwb");
if
(!stream)
{
printf("Unable to open the read stream\n");
return FALSE;
}
if
(!RwFindStreamChunk(stream, rwCHUNKUSER))
{
printf("Unable to find the user chunk\n");
return FALSE;
}
/* Read the mark */
if
(!RwReadStreamChunkHeader(stream))
{
printf("Error reading chunk mark.\n");
return FALSE;
}
/* Find the embedded light */
if
(!RwFindStreamChunk(stream, rwCHUNKLIGHT))
{
printf("Unable to find the light chunk\n");
return FALSE;
}
/* Read back the light */
if
(!RwReadStreamChunk(stream, rwCHUNKLIGHT, &light,0))
{
printf("Unable to read light\n");
return FALSE;
}
/* Find the embedded clump */
if
(!RwFindStreamChunk(stream, rwCHUNKCLUMP))
{
printf("Unable to find the clump chunk\n");
return FALSE;
}
/* Read back the clump */
if
(!RwReadStreamChunk(stream, rwCHUNKCLUMP, &clump, 0))
{
printf("Unable to read clump\n");
return FALSE;
}
if
(!RwCloseStream(stream, NULL))
{
printf("Unable to close read stream\n");
return FALSE;
}
return TRUE;
}
Platform independence
Data can be written off to a user chunk in any format. Unfortunately different machines have different number format and different 'endianness'. The binary file format uses standard number formats and endianesses for the built in chunk types. If a user requires platform independence RenderWare provides services which will allow for the writing of platform independent user chunks.
The functions which enable this platform independence are as follows
These allow for the reading and writing of integers and RwReals in a manner where they can be read correctly on arbitrary platforms. To write an array of 5 RwReals the following can be used,
RwReal
numbers[]={CREAL(1), CREAL(2), CREAL(3),
CREAL(4), CREAL(5)};
if
(RwWriteStreamReal(stream, numbers, sizeof(numbers))
{
printf("Error writing reals\n");
return FALSE;
}
Note that the number of RwReals is passed in bytes, this is so that the sizeof operator can be used (as in sizeof(numbers)).
These numbers could be read back with the following.
RwReal readnumbers[5];
if
(RwReadStreamReal(stream, readnumbers,
sizeof(readnumbers))
{
printf("Error reading reals\n");
return FALSE;
}
Note that the RwReals saved off will probably have to have their format changed so that they are in the correct platform independent format. This may mean that numbers written and read may well not be exactly the same as the original value - due to precision differences in the binary file format number format.
A number read via the binary file format, can be written with no loss of information (so repeated reading and writing will not cause numbers to 'degenerate').
The binary file format only supports 32 bit C 'longs', and thus other integer formats must be translated into this form if they are to be saved off in a platform independent manner.
The Binary Representation
The binary file format number format has the following characteristics
· All values are stored big endian. The most significant byte of a long word is saved first in the stream (this is the opposite of how Intel architecture machines represent numbers - they use the little endian format where the least significant byte of a long word is stored first).
· All RwReals are saved as IEEE 32 bit floating representation.
· 32 bit integers are the only integer type supported.
· All chunks should be of a size of a multiple of 4. The format will function on most platforms if this is not held, but certain systems do not like data straddling boundaries (such as a long word on an odd address). They either crash or have to perform an exception. Thus chunks should be kept of a multiple of 4 bytes wherever possible. All the built in chunk types conform to this convention.
·