Difference between revisions of "ManiaPlanet internals"

From Mania Tech Wiki
Jump to navigation Jump to search
(Moved the class / struct NodMeta as SGameCtnIdentifier to the basic data structures)
Line 82: Line 82:
 
* If bit 30 *or* bit 31 is set, the Id is textual. In this case it contains a row and column index into the global string table that can be used to look up the text.
 
* If bit 30 *or* bit 31 is set, the Id is textual. In this case it contains a row and column index into the global string table that can be used to look up the text.
 
* If bits 30 and 31 are set, the Id is empty (null). Typically ''value'' is -1 in this case.
 
* If bits 30 and 31 are set, the Id is empty (null). Typically ''value'' is -1 in this case.
 +
 +
Id's are managed by class CMwId.
 +
 +
===Identifier===
 +
Stores three Id's that together describe an object. Typically, this is the name of the object, the collection ID, and the author name.
 +
 +
struct SGameCtnIdentifier
 +
{
 +
    Id uid;        // Name. Could be the name of a map, name of a block, path of a file etc.
 +
    Id collection;  // Environment. In ManiaPlanet, 0xC = Canyon and 0xCA = Storm.
 +
    Id author;      // Author name (typically "Nadeo")
 +
};
 +
 +
Identifier are managed by class SGameCtnIdentifier.
  
 
===Delegate===
 
===Delegate===
Line 176: Line 190:
 
     virtual bool Is(int classID);  // Returns true if the object's class is equal to or derives from
 
     virtual bool Is(int classID);  // Returns true if the object's class is equal to or derives from
 
                                     // the specified class
 
                                     // the specified class
     virtual NodMeta* GetMeta();    // Returns meta information about the object, like name and author
+
     virtual SGameCtnIdentifier* GetIdentifier();    // Returns meta information about the object, like name and author
     virtual void SetUID(char* psz); // Sets the name of the object (affects GetMeta())
+
     virtual void SetUID(char* psz); // Sets the name of the object (affects GetIdentifier())
 
     virtual void m1C();
 
     virtual void m1C();
 
     virtual void m20();
 
     virtual void m20();
Line 208: Line 222:
 
  };
 
  };
 
   
 
   
struct NodMeta
 
{
 
    Id uid;        // Name. Could be the name of a map, name of a block, path of a file etc.
 
    Id collection;  // Environment. In ManiaPlanet, 0xC = Canyon and 0xCA = Storm.
 
    Id author;      // Author name (typically "Nadeo")
 
};
 
 
 
GetPropertyIndirect() and CallMethodIndirect() should not be used directly. These only exist for handling calculated properties (where the value doesn't come directly from a field in the class but is calculated). Instead, use the nonvirtual wrapper methods GetProperty() and CallMethod() which handle both field properties and calculated properties.
 
GetPropertyIndirect() and CallMethodIndirect() should not be used directly. These only exist for handling calculated properties (where the value doesn't come directly from a field in the class but is calculated). Instead, use the nonvirtual wrapper methods GetProperty() and CallMethod() which handle both field properties and calculated properties.
  

Revision as of 19:27, 7 June 2017

This page details the technical inner workings of ManiaPlanet (and the original TrackMania). It describes the game's classes, data structures and functions, and is as such geared towards reverse engineers and programmers.

This documentation goes together well with TM2Unlimiter.

Basic data structures

This section describes the basic primitive data structures used in ManiaPlanet.

FastArray

Simple list of items. Resizing the list consists of allocating exactly the amount of memory needed for the new size, copying the original data to it, and freeing the original buffer.

struct FastArray<T>
{
    int size;     // Number of elements in list
    T* pElems;    // Pointer to array of elements
};

FastBuffer

Simple list of items, comparable to std::vector. Resizing the list consists of first checking the current capacity; if newSize <= capacity, no reallocation is needed. Otherwise, a higher capacity is calculated (usually higher than newSize) and the list is reallocated to this new capacity.

struct FastBuffer<T>
{
    int size;      // Number of elements in list
    T* pElems;     // Pointer to memory buffer
    int capacity;  // Maximum number of elements that can be stored in the list before
                   // it has to be reallocated
};

String

Zero-terminated string of ASCII characters. An empty string is denoted by a size of 0 and psz pointing to a specific empty ASCII string (a 0 byte).

struct String
{
    int size;   // Number of characters, excluding terminating 0
    char* psz;  // Pointer to text
};

FastString

Intermediate class used for assigning char*'s/Strings to Strings (e.g.: String str, str2; str = "abc"; str2 = str).

Rather than passing the char* or String argument directly to String::operator=, a temporary FastString object is created holding the char* pointer and size. In case of assigning a String, the size is known; in the case of a char*, it is calculated using strlen. Then, this FastString object is passed by reference to String::operator=.

struct FastString
{
    char* psz;
    int size;
    String* pStr;
};

Note that the order of psz and size is reversed compared to String!

StringInt

"International" string: zero-terminated string of UTF16 characters (wchar_t). As with String, an empty string is denoted by a size of 0 and pwsz pointing to a specific empty UTF16 string (a 0 word).

struct StringInt
{
    int size;
    wchar_t* pwsz;
};

FastStringInt

The UTF16 equivalent of FastString.

struct FastStringInt
{
    wchar_t* pwsz;
    int size;
    bool bAscii;      // Indicates whether the string only contains characters
                      // that can be converted to ASCII
};

Id

Stores a numerical or textual identifier.

Everytime a string is assigned to an Id, this string is placed into a global 2-dimensional table with its position depending on its hash (if it wasn't already in the table). This position is then stored in the Id. Because of this mechanism, Ids can be used for very fast string comparisons (like a hash) while still being reversible.

struct Id
{
    int value;
};
  • If bits 30 and 31 are not set, the Id is numerical and value is the ID.
  • If bit 30 *or* bit 31 is set, the Id is textual. In this case it contains a row and column index into the global string table that can be used to look up the text.
  • If bits 30 and 31 are set, the Id is empty (null). Typically value is -1 in this case.

Id's are managed by class CMwId.

Identifier

Stores three Id's that together describe an object. Typically, this is the name of the object, the collection ID, and the author name.

struct SGameCtnIdentifier
{
    Id uid;         // Name. Could be the name of a map, name of a block, path of a file etc.
    Id collection;  // Environment. In ManiaPlanet, 0xC = Canyon and 0xCA = Storm.
    Id author;      // Author name (typically "Nadeo")
};

Identifier are managed by class SGameCtnIdentifier.

Delegate

Stores a reference to a member function.

struct Delegate
{
    void*    pInstance;    // Pointer to the object on which the method should be called
    void*    pMethod;      // Pointer to the method code
};

Color

Stores a colour.

struct Color
{
    int r;
    int g;
    int b;
    int a;
};

Vectors/matrices

ManiaPlanet of course has several vector types:

struct int2     // And uint2 for unsigned int
{
    int x;
    int y;
};
struct int3     // And uint3 for unsigned int
{
    int x;
    int y;
    int z;
};
struct Vec2D
{
    float fX;
    float fY;
};
struct Vec3D
{
    float fX;
    float fY;
    float fZ;
};
struct Vec4D
{
    float fX;
    float fY;
    float fZ;
    float fW;
};
struct Quaternion     // Structure is uncertain
{
    float fAngle;
    Vec3D vecAxis;
};
struct Matrix33
{
    Vec3D rotRow1;
    Vec3D rotRow2;
    Vec3D rotRow3;
};
struct Matrix34 : public Matrix33
{
    Vec3D position;
};

Object system

ManiaPlanet has its own custom object system that supports the following features:

  • Reflection
    • Runtime type identification: finding out the class of an object at runtime.
    • Late binding: accessing properties and methods by their name rather than their address. Mainly used for ManiaScript.
  • Serialization: reading and writing objects from .gbx files.

The root class of the object system is called CMwNod. All classes that participate in the object system derive from it, directly or indirectly. The equivalent in other languages like C# or Java would be "Object".

class CMwNod
{
public:
    virtual m0();
    virtual ~CMwNod();
    virtual CMwClassInfo* GetClassInfo();      // Gets information about the object's class
    virtual int GetClassID();
    virtual bool Is(int classID);   // Returns true if the object's class is equal to or derives from
                                    // the specified class
    virtual SGameCtnIdentifier* GetIdentifier();     // Returns meta information about the object, like name and author
    virtual void SetUID(char* psz); // Sets the name of the object (affects GetIdentifier())
    virtual void m1C();
    virtual void m20();
    virtual void GetPropertyIndirect(CMwStack* pStack, void** ppValue);   // Gets the value of a property
    virtual void CallMethodIndirect(CMwStack* pStack, void* pValue);      // Calls a method or sets a property
    virtual void m2C();
    virtual void m30();
    virtual void ReadWriteChunks(CClassicArchive* pArchive);              // (De)serializes the nod from a stream
    virtual void ReadWriteChunk(CClassicArchive* pArchive, int chunkID);  // (De)serializes a single chunk from a stream
    virtual void m3C();
    virtual int GetChunkLoadFlags(int chunkID);        // Gets chunk-specific flags, e.g. if it is skippable
    virtual int GetChunkIDByIndex(int index);          // Used during serialization: which chunk to write next
    virtual int GetNumChunks();                        // Used during serialization: how many chunks to write
    virtual void OnFinishReadWriteChunks();            // Called after the last chunk has been read/written
    virtual void m50();
    virtual void m54();
    virtual void m58();
    virtual void m5C();
    virtual void m60();
    virtual void WriteHeaderChunk(CClassicArchive* pArchive, int* pHeaderChunkID, bool* pbLight);
                                                      // Writes a header chunk to a stream
    virtual int GetHeaderChunkIDByIndex(int index);   // Used during serialization: which header chunk to write next
    virtual int GetNumHeaderChunks();                 // Used during serialization: how many header chunks to write

private:
    int refcount;                // Reference count. If this reaches zero, the object is destroyed.
    CSystemFid* pFid;            // File which the object originates from (if it was deserialized; NULL otherwise)
    FastBuffer<CMwNod*>* pList;  // Optional list of child objects
    int unknown;
};

GetPropertyIndirect() and CallMethodIndirect() should not be used directly. These only exist for handling calculated properties (where the value doesn't come directly from a field in the class but is calculated). Instead, use the nonvirtual wrapper methods GetProperty() and CallMethod() which handle both field properties and calculated properties.

Runtime class inspection

Each CMwNod-derived class is described by a CMwClassInfo instance, which contains the class ID, pointer to the parent class, and list of members (properties/methods). The classes are grouped into "engines" (CMwEngineInfo) which can be thought of as namespaces in other languages. Finally, all available engines are listed in a singleton CMwEngineManager instance.

class CMwEngineManager
{
public:
    virtual ~CMwEngineManager();

private:
    FastArray<CMwEngineInfo*> engines;
};
class CMwEngineInfo
{
public:
    virtual ~CMwEngineInfo();

private:
    int engineID;
    char* pszName;
    FastArray<CMwClassInfo*> classes;
};
class CMwClassInfo
{
public:
    virtual ~CMwClassInfo();

private:
    int classID;
    CMwClassInfo* pParentClassInfo;
    int unknown0;
    int unknown1;
    char* pszName;
    CMwClassInfo* pNextClassInfo;
    CMwNod* (*pInstantiate)();      // Pointer to a function that creates an instance of the class
    int unknown2;
    int unknown3;
    int unknown4;
    int unknown5;
    int unknown6;
    MemberInfo** ppMemberInfos;
    int numMemberInfos;
};
struct CMwMemberInfo    // Base class for class member descriptions
{
    enum eType
    {
        ACTION,          // Method that takes no arguments and has no return value; CMwMethodInfo
        BOOL,
        BOOLARRAY,
        BOOLBUFFER,
        BOOLBUFFERCAT,
        CLASS,           // CMwClassMemberInfo
        CLASSARRAY,      // CMwClassArrayMemberInfo
        CLASSBUFFER,     // CMwClassArrayMemberInfo
        CLASSBUFFERCAT,  // CMwClassArrayMemberInfo
        COLOR,           // Color struct
        COLORARRAY,
        COLORBUFFER,
        COLORBUFFERCAT,
        ENUM,            // CMwEnumInfo
        ENUMARRAY,
        ENUMBUFFER,
        ENUMBUFFERCAT,
        INT,
        INTARRAY,
        INTBUFFER,
        INTBUFFERCAT,
        INTRANGE,
        ISO4,            // Matrix34
        ISO4ARRAY,
        ISO4BUFFER,
        ISO4BUFFERCAT,
        ISO3,            // Matrix33
        ISO3ARRAY,
        ISO3BUFFER,
        ISO3BUFFERCAT,
        ID,              // Id
        IDARRAY,
        IDBUFFER,
        IDBUFFERCAT,
        NATURAL,         // unsigned int
        NATURALARRAY,
        NATURALBUFFER,
        NATURALBUFFERCAT,
        NATURALRANGE,
        REAL,            // float
        REALARRAY,
        REALBUFFER,
        REALBUFFERCAT,
        REALRANGE,
        STRING,          // String
        STRINGARRAY,
        STRINGBUFFER,
        STRINGBUFFERCAT,
        STRINGINT,       // StringInt
        STRINGINTARRAY,
        STRINGINTBUFFER,
        STRINGINTBUFFERCAT,
        VEC2,            // Vec2D
        VEC2ARRAY,
        VEC2BUFFER,
        VEC2BUFFERCAT,
        VEC3,            // Vec3D
        VEC3ARRAY,
        VEC3BUFFER,
        VEC3BUFFERCAT,
        VEC4,            // Vec4D
        VEC4ARRAY,
        VEC4BUFFER,
        VEC4BUFFERCAT,
        INT3,            // int3
        INT3ARRAY,
        INT3BUFFER,
        INT3BUFFERCAT,
        NAT3,            // uint3
        NAT3ARRAY,
        NAT3BUFFER,
        NAT3BUFFERCAT,
        QUAT,            // Quaternion
        QUATARRAY,
        QUATBUFFER,
        QUATBUFFERCAT,
        PROC             // Method with arguments and/or a return value; CMwMethodInfo
    };
    
    eType type;
    int memberID;
    CMwParam* pParam;     // Pointer to an object used for accessing the member
    int fieldOffset;      // Offset of the field within the class (0 for methods and code-based properties)
    char* pszName;
    int flags;
    int flags2;
};
struct CMwClassMemberInfo : public CMwMemberInfo        // Specialized class for fields/properties that reference an object (CMwNod*)
{
    CMwClassInfo* pClassInfo;                           // Class of the object being referenced
};
struct CMwClassArrayMemberInfo : public CMwMemberInfo   // Specialized class for fields/properties that are a list
                                                        // of object references (FastArray<CMwNod*>/FastBuffer<CMwNod*>)
{
    int unknown0;
    char* pszFriendlyClassName;
    int unknown1;
    CMwClassInfo* pClassInfo;                           // Class of the objects in the list
};
struct CMwMethodInfo : public CMwMemberInfo             // Specialized class for member methods
{
    void* pMethod;                                      // Pointer to the method code
    int numArgs;
    int* pArgClassIDs;                                  // Pointer to an array of numArgs ints
    char** ppszArgNames;                                // Pointer to an array of numArgs char*'s
    int* pArgFlags;                                     // Pointer to an array of numArgs ints
};
struct CMwEnumInfo : public CMwMemberInfo               // Specialized class for members that store an enum value
{
    int unknown;
    int numValues;
    char** ppszValueNames;                              // Pointer to an array of numValues char*'s
};

Late binding

Once you have the CMwMemberInfo of a property of a class, you can access this property without knowing the class's internal structure. Similarly, if you have a CMwMethodInfo, you can call the method without knowing its address. In effect, this lets you use ManiaPlanet functionality as easily from C++ as from ManiaScript; except that by directly accessing the class system, a lot more functionality is available (most methods are actually blocked for use from ManiaScript).

Performing late binding in ManiaPlanet requires three objects:

  • A CMwNod: the object on which you want to perform the action.
  • A CMwMemberInfo: the object that describes the member you want to access. This can be retrieved by getting the CMwNod's CMwClassInfo (pNod->GetClassInfo()) and looping over the class's members until you find the one with the name you are looking for.
  • A CMwStack: wraps the CMwMemberInfo. In addition, when calling a method, this contains any method arguments.

A CMwStack is a list of items, where each item has a pointer to a value and a type ID (indicating whether the value is a CMwMemberInfo*, a CMwNod*, a float*, a bool*...). The name of this class is misleading: it doesn't work like a stack (LIFO: the item that was added last is consumed first), but like a queue (FIFO: the item that was added first is consumed first). In addition, it's not a generic container class; it's only used for late binding.

The three late binding scenarios work in the following way:

Getting properties

  • Create a CMwStack (let's call it "stack").
  • Push the CMwMemberInfo* of the property you want to read.
  • Allocate memory large enough to hold the property's value (let's call it "result"). If you are retrieving a primitive value or a CMwNod*, simply allocate 4 bytes. For complex property values like FastBuffer or String, allocate 4 bytes for a pointer to the value, immediately followed by memory for the value itself.
  • Call pNod->GetProperty(&stack, &result).

Example:

List < class CPlugSolid* >& CFuncClouds::GetSolidFids () const
{
    static CMwMemberInfo* pMemberInfo = GetClassInfo ()->GetMemberInfo ( "SolidFids" );
    struct
    {
        List < class CPlugSolid* >* pResult;
        List < class CPlugSolid* > storage;
    } result;
    CMwStack stack;
    stack.Push ( pMemberInfo );
    GetProperty ( &stack, &result );
    return *result.pResult;
}

Setting properties

  • Create a CMwStack.
  • Push the CMwMemberInfo* of the property you want to write.
  • Call pNod->CallMethod(&stack, &value).

Example:

void CFuncClouds::SetSolidFids ( List < class CPlugSolid* >& value )
{
    static CMwMemberInfo* pMemberInfo = GetClassInfo ()->GetMemberInfo ( "SolidFids" );
    CMwStack stack;
    stack.Push ( pMemberInfo );
    CallMethod ( &stack, &value );
}

Calling methods

  • Create a CMwStack.
  • Push the CMwMemberInfo* of the method you want to call.
  • Push any method arguments in the reverse order as they appear in the signature (right to left).
  • If the method has a return value, allocate a variable for it and push this variable.
  • Call pNod->CallMethod(&stack, NULL).

Example:

uint CGameCtnEditorPluginScriptHandler::CanPlaceBlock ( class CGameCtnBlockInfo* pBlockModel, int3 coord, uint dir )
{
    static CMwMemberInfo* pMemberInfo = GetClassInfo ()->GetMemberInfo ( "CanPlaceBlock" );
    uint uiVariantIndex;
    CMwStack stack;
    stack.Push ( pMemberInfo );
    stack.Push ( dir );
    stack.Push ( coord );
    stack.Push ( *reinterpret_cast < CMwNod** > ( &pBlockModel ) );
    stack.Push ( uiVariantIndex );
    CallMethod ( &stack, NULL );
    return uiVariantIndex;
}

Internally, a CMwStack has some optimizations for efficient list storage. The first two items are always stored inside the object itself (m_ContainedItems member). If more items are pushed after these two, a buffer is allocated on the heap and the additional items are stored in there. "Pushing" an item consists of appending it to the back of the list. After the stack is set up, it is passed to GetProperty/CallMethod which processes the items in the same order in which they were pushed.

class CMwStack
{
public:
    enum eItemType
    {
        ITEM_MEMBER     = 0,            // CMwMemberInfo*
        ITEM_BOOL       = 0x10000001,
        ITEM_OBJECT     = 0x10000002,   // CMwNod*
        ITEM_ENUM       = 0x10000003,
        ITEM_ISO4       = 0x10000004,   // Matrix34*
        ITEM_VEC2       = 0x10000005,
        ITEM_VEC3       = 0x10000006,
        ITEM_INT3       = 0x10000007,
        ITEM_UINT3      = 0x10000008,
        ITEM_INT        = 0x10000009,
        ITEM_UINT       = 0x1000000A,
        ITEM_FLOAT      = 0x1000000B,
        ITEM_STRING     = 0x1000000C,   // String*
        ITEM_WSTRING    = 0x1000000D    // StringInt*
    };

    struct Item
    {
        void*           m_pValue;       // Always a pointer, even for primitives
        eItemType       m_Type;
    };

private:
    Item                m_ContainedItems[2];    // The first two items in the queue
    Item*               m_pExtraItems;          // Pointer to an array of additional items (item 3 and above)
    short               m_sSize;                // The total number of items in the queue (contained + extra)
    short               m_sExtraItemsCapacity;  // The capacity of the "extra items" array (not counting m_ContainedItems)
    int                 m_iCurrentPos;          // Current reading position in the item queue (starts at 0 and increases)
};

Serialization

Every CMwNod can be read from and written to a file. Typically this is a .gbx file, but it's also possible to "deserialize" a .png file into a CPlugFilePng instance. Serializing and deserializing is easy to do with the following methods:

void CreateNodFromFid(CMwNod** ppResultNod, CSystemFid* pFid, int flags = 7);
void ReadNodFromFile(StringInt* pwstrFilePath, CMwNod** ppResultNod, CSystemFids* pFolder, int flags);
void WriteNodToFile(StringInt* pwstrFilePath, CMwNod* pNod, CSystemFids* pFolder, int flags);

I/O

File and folder structure

ManiaPlanet maintains several filesystem trees in its memory. The files it represents can come from the actual file system, or from archives (.zip/.pak/.Pack.Gbx).

At the root of each file hierarchy, there is a "drive". This should not be confused with a PC harddrive or partition like C:\; instead, it is an isolated ManiaPlanet folder. The following drives exist:

  • Resource: contents of ManiaPlanet\Packs\Resource.pak. Contains common fonts, shaders, textures etc.
  • Personal: the "Personal" folder containing the user's maps, settings etc.
  • ManiaPlanet: the "ManiaPlanet" folder containing ManiaPlanet.exe and the GameData folder.
  • Common: the "Common" folder containing the PacksCache folder etc.
  • Temp: temporary folder.

.gbx files within a drive can freely reference each other, but cross-drive references are not possible. There is one exception to this rule: through a special file format mechanism, any .gbx file can reference a file in the Resource drive.

ManiaPlanet starts off by creating an in-memory representation of the file structure of each of the drives, based on the actual files on the harddrive. Next it loads the .pak and .zip files in ManiaPlanet\Packs and Common\PacksCache. The files in these archives are merged into the previously created file structures. For example, the ManiaPlanet drive has the following (partial) structure:

ManiaPlanet\
    GameData\
        Translations\       (from harddrive)
        Vehicles\
            Media\
                Texture\    (from ManiaPlanet.zip)
                Material\   (from ManiaPlanet.pak)

The root drives are pointed to by an instance of CSystemEngine, which is in turn pointed to by the singleton instance of CMwEngineMain. The CMwEngines in CMwEngineMain correspond to the CMwEngineInfos in CMwEngineManager; each CMwEngine provides functionality related to that CMwEngineInfo.

class CMwEngine : public CMwNod {};
class CMwEngineMain : public CMwEngine
{
    enum eEngine
    {
        ENGINE_FOUNDATIONS      = 0x01,
        ENGINE_GAME             = 0x03,
        ENGINE_GRAPHIC          = 0x04,
        ENGINE_FUNCTION         = 0x05,
        ENGINE_HMS              = 0x06,
        ENGINE_CONTROL          = 0x07,
        ENGINE_MOTION           = 0x08,
        ENGINE_PLUG             = 0x09,
        ENGINE_SCENE            = 0x0A,
        ENGINE_SYSTEM           = 0x0B,
        ENGINE_VISION           = 0x0C,
        ENGINE_AUDIO            = 0x10,
        ENGINE_SCRIPT           = 0x11,
        ENGINE_NET              = 0x12,
        ENGINE_INPUT            = 0x13,
        ENGINE_XML              = 0x14,
        ENGINE_TRACKMANIA       = 0x24,
        ENGINE_QUESTMANIA       = 0x2C,
        ENGINE_SHOOTMANIA       = 0x2D
    };

    int unknown0;
    int unknown1;
    FastArray<CMwEngine*> engines;     // To get a particular engine, index this array using the relevant eEngine value.
                                       // E.g. engines[ENGINE_SYSTEM] returns a pointer to the CSystemEngine instance.
};
class CSystemEngine : public CMwEngine
{
private:
    int unknown0;
    void* pSystemManagerFile;
    StringInt unknown1;
    CSystemFidsDrive* pResourceDrive;
    CSystemFidsDrive* pPersonalDrive;
    CSystemFidsDrive* pManiaPlanetDrive;
    CSystemFidsDrive* pCommonDrive;
    CSystemFidsDrive* pTempDrive;
    FastBuffer<Delegate> fidResolvers;
    FastArray<CSystemFid*> resourceFids;    // List of files in Resource.pak. A resourceIndex in a .gbx file
                                            // indexes this array (see GBX wiki page).
    int unknown2;
    CSystemFidsFolder* pGameDataFolder;     // Pointer to the ManiaPlanet\GameData folder
};
class CSystemFid : public CMwNod          // Base class for files
{
private:
    int unknown0;
    CSystemFids* pFolder;                 // Folder containing the file
    int flags1;
    int flags2;
    CMwNod* pNod;                         // Object that was deserialized from this file, if any
    FastBuffer<CSystemFid*> childFids;    // Versions of this file that were loaded with different parameters
                                          // (only filled in if this is the main version of the file)
    CSystemFid* pMainFid;                 // Pointer to the main version of the file
                                          // (only filled in if this is a child version of the file)
    byte fidParameters[0x30];
    int unknown1;
    int classID;                          // Class ID of the content of the file (i.e. of pNod)
    void* pHeaderChunks;                  // Header chunk data, directly from the .gbx file,
                                          // starting with numHeaderChunks
    CLoader* pLoader;
    int index;                            // Index of the file in its containing archive (.pak/.zip), if any
};
class CSystemFidFile : public CSystemFid  // Class representing a named file (as opposed to an anonymous in-memory "file")
{
private:
    StringInt wstrName;                   // Name of the file (without folder path)
    int index;
    int sizeLow;                          // Low 32 bits of the file size
    int sizeHigh;                         // High 32 bits of the file size (0 for files < 4GB)
};
class CSystemFidMemory : public CSystemFid
{
private:
    CClassicBuffer* pBuffer;
};
class CSystemFids : public CMwNod         // Base class for folders
{
private:
    int unknown0;
    CSystemFids* pParentFolder;
    CSystemFidsDrive* pDrive;
    FastBuffer<CSystemFid*> files;
    FastBuffer<CSystemFids*> subFolders;
    int unknown1;
};
class CSystemFidsFolder : public CSystemFids    // Named folder
{
private:
    int unknown0;
    int unknown1;
    int unknown2;
    StringInt wstrName;
};
class CSystemFidsDrive : public CSystemFidsFolder {};

Loading files

Loading a file in ManiaPlanet is quite easy: you get the CSystemFid's pLoader and call its GetBuffer() method, which returns a stream object ready for reading (see Streams section). Once you're done reading, call the loader's FreeBuffer() method to release the stream.

class CLoader
{
public:
    virtual CClassicBuffer* GetBuffer(CSystemFid* pFid, int mode, CClassicBuffer* pInnerBuffer = NULL);   // mode: 1 = read, 2 = write
    virtual void FreeBuffer(CSystemFid* pFid, CClassicBuffer* pBuffer);
};

Streams

ManiaPlanet uses various stream-like objects for reading and writing files. This means that each stream is associated with an array of bytes (like a file), has a certain position within that array, and can either read from or write to the array (optionally making it larger).

CClassicBuffer

Abstract base class that represents a stream. Comparable to C#'s Stream class. Contrary to what the name might suggest, this class does not necessarily contain a memory buffer.

class CClassicBuffer
{
public:
    virtual ~CClassicBuffer();
    virtual int ReadBytes(void* pTargetBuffer, int length);
    virtual int WriteBytes(void* pData, int length);
    virtual void Close();
    virtual void m10()
    virtual int GetPosition();
    virtual int GetSize();
    virtual bool IsMemoryMapped();
    virtual void SetPosition(int position);
    virtual void m24();
    virtual void Seek(int offset);   // SetPosition(GetPosition() + offset);
    virtual int GetCapacity();

private:
    int mode;                // 1 = reading, 2 = writing
    bool bInitializing;      // If set to true, writing to the CClassicBuffer will
                             // not actually write to the underlying stream.
                             // Instead, the data provided for writing will be used
                             // to affect decryption.
};

CClassicBufferPart

Exposes a subsection of an existing stream as a new stream.

class CClassicBufferPart : public CClassicBuffer
{
private:
    CClassicBuffer* pInnerBuffer;
    int offset;
    int size;
};

CClassicBufferMemory

In-memory data stream, comparable to C#'s MemoryStream.

class CClassicBufferMemory : public CClassicBuffer
{
private:
    void* pData;
    int size;
    int position;
    int capacity;
    int capacityIncrement;    // Step size by which to increase capacity when the buffer gets full
};

CSystemFile

Buffered file stream. The class reads 4 KB from the designated file into memory and then services read requests from that buffer, until the buffer is exhausted and the next 4 KB is read.

class CSystemFile : public CClassicBuffer
{
private:
    HANDLE hFile;     // Obtained using Win32's CreateFile()
    int unknown;
    StringInt wstrFilePath;
    int position;
    int remainingReadahead;    // Remaining unread bytes in the buffer; 4096 - readaheadOffset
    int readaheadOffset;       // Current read offset within the buffer
    byte readaheadBuffer[4096];
};

CSystemFileMemMapped

Memory-mapped file stream.

class CSystemFileMemMapped : public CClassicBuffer
{
private:
    StringInt wstrFilePath;
    int position;
    int fileSize;
    HANDLE hMapping;     // Handle returned by Win32's CreateFileMapping()
    HANDLE hFile;        // Handle returned by Win32's CreateFile()
    void* pMappedData;   // Data pointer, returned by MapViewOfFile()
};

CClassicBufferZlib

Compresses (when writing) or decompresses (when reading) data from an underlying stream. Comparable to C#'s DeflateStream.

class CClassicBufferZlib : public CClassicBuffer
{
private:
    int lastError;          // Last zlib error code
    CClassicBuffer* pInnerBuffer;
    z_stream_s z_stream;    // See zlib.h
    int unknown;
    int unknown;
    byte inBuffer[256];
    byte outBuffer[1024];
    int unknown;
    int uncompressedSize;
    bool bCompressing;
};

CClassicBufferCrypted

Encrypts (when writing) or decrypts (when reading) data from an underlying stream.

class CClassicBufferCrypted : public CClassicBuffer
{
private:
    int unknown;
    CClassicBuffer* pInnerBuffer;
    int position;
    int ivSize;      // Size of the IV (initialisation vector)
    int algo;        // 0: custom xor-based algorithm
                     // 1: Blowfish in CBC mode (most commonly used)
                     // 2: Blowfish in CTR mode
    int dataSize;
    byte key[16];
    void* pBlowfishState;
    int unknown;
    byte iv[8];
    byte ivXor[8];
    bool bEncrypt;
    int unknown;
    byte buffer[256];
    int bufferIndex;
};

There are some nasty details in the way TrackMania/ManiaPlanet handle encryption. For details, see the page on .pak files (for TrackMania). ManiaPlanet further complicates things by adding custom, non-standard behaviour to the CBC mode.

CClassicArchive

Wraps a CClassicBuffer and exposes handy methods for reading/writing bytes, words, dwords, floats etc. in one go. It's like a C# BinaryReader and BinaryWriter in one: there are both reading and writing methods, and also ReadWrite methods that call either the Read or Write method depending on the bWriting field.

Unlike the name might suggest, this class has nothing to do with .zip or .pak archives.

class CClassicArchive
{
public:
    virtual ~CClassicArchive();
    virtual ReadWriteNodRef(CMwNod** ppNod);    // Reads or writes a reference to an object.
                                                // If this is the first time the object is encountered,
                                                // it is (de)serialized at this point.
    virtual ReadWriteNodRefNoCreate(CMwNod** ppNod);   // Reads or writes a reference to an object.
                                                       // The object is expected to be already known
                                                       // (no deserialization happens).

private:
    CClassicBuffer* pInnerBuffer;
    bool bWriting;
    bool bTextMode;         // Whether to read/write in readable ASCII text or in binary
    bool bUnknown;
    void* pStringIndices;   // Pointer to cache of known string IDs
};

For details on ReadWriteNodRef, see the definition of "noderef" on the GBX page.

CSystemArchiveNod

CClassicArchive-derived class used specifically for reading and writing .gbx files.

class CSystemArchiveNod : public CClassicArchive
{
private:
    word version;                                      // .gbx file version
    word unused;
    CSystemEngine* pSystemEngine;
    FastArray<CSystemFids*> externalFolders;           // References to other folders
    FastBuffer<ExternalNodEntry> externalNodEntries;   // References to other files
    FastBuffer<NodEntry> nodEntries;                   // List of objects, both in the current .gbx file
                                                       // and from external files
    FastBuffer<CSystemFids*> folders;
    int classID;                                       // Class ID of the main object of the .gbx file
    CSystemFid* pFid;                                  // Reference to the .gbx file

    struct ExternalNodEntry
    {
         CSystemFid* pFid;      // Reference to the file
         int unknown;
         int nodIndex;          // Index in nodEntries of where the external object will be placed
         int folderIndex;       // Index in externalFolders to the folder containing the .gbx file
         bool bUseFid;          // If false, the file pFid will be deserialized and nodEntries is
                                // populated with the resulting object.
                                // If true, nodEntries is populated with pFid itself.
    };

    struct NodEntry
    {
        CMwNod* pNod;           // Pointer to the object
        int fileOffset;         // Index in the .gbx file of the object
        bool bRelease;
    };
};

A .gbx file contains a serialized main object instance whose type is indicated by classID. This main object can reference other objects, which may be contained in the same .gbx file or in other files.

All objects, both from the current .gbx file and from external files, are stored in the nodEntries list. The main object of the .gbx file is always at index 0. Every time a subobject is read from the .gbx file, or an external file is referenced, the nodEntries index is specified where the object should be stored. The process of reading .gbx files is explained in detail on the GBX page.

Tools

  • TM2Unlimiter (@archive.org) - an open source tool that contains all the code listed above and was kept up to date with the newest ManiaPlanet function addresses.