ManiaPlanet internals

From Mania Tech Wiki
Revision as of 23:21, 30 June 2017 by Electron (talk | contribs) (Added basic data types)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

This page details the technical inner workings of ManiaPlanet (and the original TrackMania). It describes the game's classes, data structures and functions, and is as such geared towards reverse engineers and programmers.

Basic data types

Nadeo uses its own primitive data types in its game engine. Since they can't be reversed, they are not used on this page and are listed here only for the sake of completeness. 

typedef int Integer;
typedef unsigned int Bool;
typedef unsigned char Nat8;
typedef unsigned short Nat16;
typedef unsigned int Natural, Nat32;
typedef unsigned __int64 Nat64;
typedef float Real, Real32;

Basic data structures

This section describes the basic primitive data structures used in ManiaPlanet.

FastArray

Simple list of items. Resizing the list consists of allocating exactly the amount of memory needed for the new size, copying the original data to it, and freeing the original buffer. 

struct FastArray<T>
{
    int size;     // Number of elements in list
    T* pElems;    // Pointer to array of elements
};

FastBuffer

Simple list of items, comparable to std::vector. Resizing the list consists of first checking the current capacity; if newSize <= capacity, no reallocation is needed. Otherwise, a higher capacity is calculated (usually higher than newSize) and the list is reallocated to this new capacity. 

struct FastBuffer<T>
{
    int size;      // Number of elements in list
    T* pElems;     // Pointer to memory buffer
    int capacity;  // Maximum number of elements that can be stored in the list before
                   // it has to be reallocated
};

String

Zero-terminated string of ASCII characters. An empty string is denoted by a size of 0 and psz pointing to a specific empty ASCII string (a 0 byte). 

struct String
{
    int size;   // Number of characters, excluding terminating 0
    char* psz;  // Pointer to text
};

FastString

Intermediate class used for assigning char*'s/Strings to Strings (e.g.: String str, str2; str = "abc"; str2 = str).

Rather than passing the char* or String argument directly to String::operator=, a temporary FastString object is created holding the char* pointer and size. In case of assigning a String, the size is known; in the case of a char*, it is calculated using strlen. Then, this FastString object is passed by reference to String::operator=. 

struct FastString
{
    char* psz;
    int size;
    String* pStr;
};

Note that the order of psz and size is reversed compared to String!

StringInt

"International" string: zero-terminated string of UTF16 characters (wchar_t). As with String, an empty string is denoted by a size of 0 and pwsz pointing to a specific empty UTF16 string (a 0 word). 

struct StringInt
{
    int size;
    wchar_t* pwsz;
};

FastStringInt

The UTF16 equivalent of FastString. 

struct FastStringInt
{
    wchar_t* pwsz;
    int size;
    bool bAscii;      // Indicates whether the string only contains characters
                      // that can be converted to ASCII
};

Id

Stores a numerical or textual identifier.

Everytime a string is assigned to an Id, this string is placed into a global 2-dimensional table with its position depending on its hash (if it wasn't already in the table). This position is then stored in the Id. Because of this mechanism, Ids can be used for very fast string comparisons (like a hash) while still being reversible. 

struct Id
{
    int value;
};
  • If bits 30 and 31 are not set, the Id is numerical and value is the ID.
  • If bit 30 *or* bit 31 is set, the Id is textual. In this case it contains a row and column index into the global string table that can be used to look up the text.
  • If bits 30 and 31 are set, the Id is empty (null). Typically value is -1 in this case.

Ids are managed by the class CMwId: 

class CMwId
{
public:
    CMwId(void);  // Initializes the ID with the value -1 (Unassigned).
    CMwId(CMwId const &);  // Initializes the id with the ID of the passed class.
    ~CMwId(void);

    static CMwId * Unassigned;  // Points to the first entry in the table (Unassigned).
    static void StaticInit(void);  // Called from CGbxApp::Init. Creates the name table. Creates a first ID with the value -1 (Unassigned) and adds it to the table.
    static void StaticRelease(void);  // Called from CGbxApp::Destroy. Releases the name table.

    static CMwId CreateFromLocalIndex(unsigned long);   // Calls CMwId::CMwId and sets the ID to the passed index.
    static CMwId CreateFromTypeAndIndex(unsigned long); // Calls CMwId::CMwId and sets the ID to the passed index.
    static CMwId CreateFromLocalName(char const *);     // Calls CMwId::CMwId and adds the passed name to the name table using SetLocalName (bit 30 of the ID is set).
    static CMwId CreateFromUUIdName(char const *);      // Calls CMwId::CMwId and adds the passed name to the name table using AddName (bit 31 of the ID is set).

    void SetLocalName(char const *);  // Sets the ID to -1 if the name is "Unassigned"; otherwise, adds the name to the table using AddName and sets bit 30 of the ID.
    void SetLocalName(CFastStringInt const &);  // Calls CFastStringInt::GetUtf8 and then SetLocalName.

    char const * GetString(void)const;  // Gets the string of the ID if bit 30 or 31 is set; otherwise, NULL is returned.

    void GetName(CFastString &)const;  // Returns either the name of the ID using GetString, "Unassigned" if the ID is -1 or "Id<number>" if ID represents a number.
    void GetName(CFastStringInt &)const;
    class CFastString const GetName(void)const;

    void Archive(CClassicArchive &);  // Serializes the ID. See "lookbackstring" on the GBX page for some details.

private:
    static SMwIdInternal * s_NameTable;  // Points to the name table.
    static unsigned long AddName(char const *);  // Adds the name to the name table.
    static void DeleteArchiveUserData(CClassicArchive *);
};

Identifier

Stores three Ids that together describe an object. Typically, this is the name of the object, the collection ID, and the author name. 

struct Ident
{
    Id id;          // Name. Could be the name of a map, name of a block, path of a file etc.
    Id collection;  // Environment. In ManiaPlanet, 0xC = Canyon and 0xCA = Storm.
    Id author;      // Author name (typically "Nadeo")
};

Identifiers are managed by the class SGameCtnIdentifier: 

struct SGameCtnIdentifier
{
public:
    SGameCtnIdentifier(void);  // Calls CMwId::CMwId for all three IDs of the Ident.
    SGameCtnIdentifier(SGameCtnIdentifier const &);  // Calls CMwId::CMwId(class CMwId const &) for all three IDs of the Ident.
    ~SGameCtnIdentifier(void);

    int operator==(SGameCtnIdentifier const &)const;  // Compares the first two IDs of the Ident.
    static int sCompareCollectionAndId(SGameCtnIdentifier const *, SGameCtnIdentifier const *);  // Compares the names of the first two IDs of the Ident using CMwId::GetName.

    void Archive(CClassicArchive &);  // Serializes the Ident. See "meta" on the GBX page for details.
};

Delegate

Stores a reference to a member function. 

struct Delegate
{
    void*    pInstance;    // Pointer to the object on which the method should be called
    void*    pMethod;      // Pointer to the method code
};

Color

Stores a colour. 

struct Color
{
    float r;
    float g;
    float b;
};

Vectors/matrices

ManiaPlanet of course has several vector types: 

struct Int2     // And Nat2 for unsigned int
{
    int x;
    int y;
};

struct Int3     // And Nat3 for unsigned int
{
    int x;
    int y;
    int z;
};

struct Vec2
{
    float x;
    float y;
};

struct Vec3
{
    float x;
    float y;
    float z;
};

struct Vec4
{
    float x;
    float y;
    float z;
    float w;
};

struct Quat
{
    float w;
    float x;
    float y;
    float z;
};

struct Iso3
{
    const float AxeXx;
    const float AxeXy;
    const float AxeYx;
    const float AxeYy;
    float tx;
    float ty;
};

struct Iso4
{
    const float AxeXx;
    const float AxeXy;
    const float AxeXz;
    const float AxeYx;
    const float AxeYy;
    const float AxeYz;
    const float AxeZx;
    const float AxeZy;
    const float AxeZz;
    float tx;
    float ty;
    float tz;
};

Object system

ManiaPlanet has its own custom object system that supports the following features:

  • Reflection
    • Runtime type identification: finding out the class of an object at runtime.
    • Late binding: accessing properties and methods by their name rather than their address. Mainly used for ManiaScript.
  • Serialization: reading and writing objects from .gbx files.

The root class of the object system is called CMwNod. All classes that participate in the object system derive from it, directly or indirectly. The equivalent in other languages like C# or Java would be "Object". 

class CMwNod
{
public:
    CMwNod(void);
    CMwNod(CMwNod const &);
    virtual ~CMwNod(void);
    virtual CMwClassInfo const * MwGetClassInfo(void)const; // Gets information about the object's class
    virtual unsigned long GetMwClassId(void)const;
    virtual bool MwIsKindOf(unsigned long classID)const;    // Returns true if the object's class is equal to or derives from the specified class
    virtual CMwId const * MwGetId(void)const;               // Returns meta information about the object, like name and author
    virtual void SetIdName(char const * psz);               // Sets the name of the object (affects MwGetId())
    virtual void MwIsKilled(CMwNod *);
    virtual void MwIsUnreferenced(CMwNod *);
    virtual unsigned long VirtualParam_Get(CMwStack * pStack, CMwValueStd * ppValue); // Gets the value of a property
    virtual unsigned long VirtualParam_Set(CMwStack * pStack, void * pValue);         // Calls a method or sets a property
    virtual unsigned long VirtualParam_Add(CMwStack * pStack, void * pValue);
    virtual unsigned long VirtualParam_Sub(CMwStack * pStack, void * pValue);
    virtual CMwNod * Archive(CClassicArchive & pArchive);                   // (De)serializes the nod from a stream
    virtual void Chunk(CClassicArchive & pArchive, unsigned long chunkID);  // (De)serializes a single chunk from a stream
    virtual unsigned long GetChunkInfo(unsigned long chunkID)const;         // Gets chunk-specific flags, e.g. if it is skippable
    virtual unsigned long GetUidChunkFromIndex(unsigned long index)const;   // Used during serialization: which chunk to write next
    virtual unsigned long GetChunkCount(void)const;  // Used during serialization: how many chunks to write
    virtual void OnNodLoaded(void);                  // Called after the last chunk has been read/written
    virtual void CreateDefaultData(void);
    virtual void OnPackAvailable(CSystemFid *, CSystemFid *);
    virtual void ApplyFidParameters(CSystemFidParameters const *, CSystemFidParameters *, CFastBuffer<SManuallyLoadedFid> &);
    virtual int OnCrashDump(CFastString &);
    virtual void CopyFrom(CMwNod *);
    virtual void HeaderChunk(CClassicArchive & pArchive, unsigned long & pHeaderChunkID, bool & pbLight); // Writes a header chunk to a stream
    virtual unsigned long GetUidHeaderChunkFromIndex(unsigned long index)const; // Used during serialization: which header chunk to write next
    virtual unsigned long GetHeaderChunkCount(void)const;                       // Used during serialization: how many header chunks to write
    virtual unsigned long GetUidMessageFromIndex(unsigned long index)const;
    virtual unsigned long GetMessageCount(void)const;
    virtual void MwReceiveMessage(unsigned long const &, CMwNod *, unsigned long *);

    unsigned long Param_Get(CMwStack * pStack, CMwValueStd * ppValue);
    unsigned long Param_Set(CMwStack * pStack, void * pValue);
    unsigned long Param_Add(CMwStack * pStack, void * pValue);
    unsigned long Param_Sub(CMwStack * pStack, void * pValue);
    unsigned long Param_Check(CMwStack * pStack);
    unsigned long Param_Set(CFastString const &, CFastStringInt const &);

    unsigned long MwAddRef(void);      // Increments countRef
    unsigned long MwRelease(void);     // Decrements countRef. If zero, the object and all dependants are destroyed.

    void MwAddDependant(CMwNod *)const;
    void MwAddReceiver(CMwNod *);
    void MwSubDependant(CMwNod *)const;
    void MwSubDependantSafe(CMwNod *)const;
    void MwFinalSubDependant(CMwNod *)const;
    void DependantSendMwIsKilled(void);

private:
    int countRef;                      // Reference count. If this reaches zero, the object is destroyed.
    CSystemFid* pFid;                  // File which the object originates from (if it was deserialized; NULL otherwise)
    FastBuffer<CMwNod*>* pDependants;  // Optional list of child objects
    int unknown;
};

VirtualParam_Get() and VirtualParam_Set() should not be used directly. These only exist for handling calculated properties (where the value doesn't come directly from a field in the class but is calculated). Instead, use the nonvirtual wrapper methods Param_Get() and Param_Set() which handle both field properties and calculated properties.

Runtime class inspection

Each CMwNod-derived class is described by a CMwClassInfo instance, which contains the class ID, pointer to the parent class, and list of members (properties/methods). The classes are grouped into "engines" (CMwEngineInfo) which can be thought of as namespaces in other languages. Finally, all available engines are listed in a singleton CMwEngineManager instance. 

class CMwEngineManager
{
public:
    virtual ~CMwEngineManager();

private:
    FastArray<CMwEngineInfo*> engines;
};

class CMwEngineInfo
{
public:
    virtual ~CMwEngineInfo();

private:
    int engineID;
    char* pszName;
    FastArray<CMwClassInfo*> classes;
};

class CMwClassInfo
{
public:
    virtual ~CMwClassInfo();

private:
    int classID;
    CMwClassInfo* pParentClassInfo;
    int unknown0;
    int unknown1;
    char* pszName;
    CMwClassInfo* pNextClassInfo;
    CMwNod* (*pInstantiate)();      // Pointer to a function that creates an instance of the class
    int unknown2;
    int unknown3;
    int unknown4;
    int unknown5;
    int unknown6;
    MemberInfo** ppMemberInfos;
    int numMemberInfos;
};

struct CMwMemberInfo    // Base class for class member descriptions
{
    enum eType
    {
        ACTION,          // Method that takes no arguments and has no return value; CMwMethodInfo
        BOOL,
        BOOLARRAY,
        BOOLBUFFER,
        BOOLBUFFERCAT,
        CLASS,           // CMwClassMemberInfo
        CLASSARRAY,      // CMwClassArrayMemberInfo
        CLASSBUFFER,     // CMwClassArrayMemberInfo
        CLASSBUFFERCAT,  // CMwClassArrayMemberInfo
        COLOR,           // Color struct
        COLORARRAY,
        COLORBUFFER,
        COLORBUFFERCAT,
        ENUM,            // CMwEnumInfo
        ENUMARRAY,
        ENUMBUFFER,
        ENUMBUFFERCAT,
        INT,
        INTARRAY,
        INTBUFFER,
        INTBUFFERCAT,
        INTRANGE,
        ISO4,            // Iso4
        ISO4ARRAY,
        ISO4BUFFER,
        ISO4BUFFERCAT,
        ISO3,            // Iso3
        ISO3ARRAY,
        ISO3BUFFER,
        ISO3BUFFERCAT,
        ID,              // Id
        IDARRAY,
        IDBUFFER,
        IDBUFFERCAT,
        NATURAL,         // unsigned int
        NATURALARRAY,
        NATURALBUFFER,
        NATURALBUFFERCAT,
        NATURALRANGE,
        REAL,            // float
        REALARRAY,
        REALBUFFER,
        REALBUFFERCAT,
        REALRANGE,
        STRING,          // String
        STRINGARRAY,
        STRINGBUFFER,
        STRINGBUFFERCAT,
        STRINGINT,       // StringInt
        STRINGINTARRAY,
        STRINGINTBUFFER,
        STRINGINTBUFFERCAT,
        VEC2,            // Vec2
        VEC2ARRAY,
        VEC2BUFFER,
        VEC2BUFFERCAT,
        VEC3,            // Vec3
        VEC3ARRAY,
        VEC3BUFFER,
        VEC3BUFFERCAT,
        VEC4,            // Vec4
        VEC4ARRAY,
        VEC4BUFFER,
        VEC4BUFFERCAT,
        INT3,            // Int3
        INT3ARRAY,
        INT3BUFFER,
        INT3BUFFERCAT,
        NAT3,            // Nat3
        NAT3ARRAY,
        NAT3BUFFER,
        NAT3BUFFERCAT,
        QUAT,            // Quaternion
        QUATARRAY,
        QUATBUFFER,
        QUATBUFFERCAT,
        PROC             // Method with arguments and/or a return value; CMwMethodInfo
    };
    
    eType type;
    int memberID;
    CMwParam* pParam;     // Pointer to an object used for accessing the member
    int fieldOffset;      // Offset of the field within the class (0 for methods and code-based properties)
    char* pszName;
    int flags;
    int flags2;
};

struct CMwClassMemberInfo : public CMwMemberInfo        // Specialized class for fields/properties that reference an object (CMwNod*)
{
    CMwClassInfo* pClassInfo;                           // Class of the object being referenced
};

struct CMwClassArrayMemberInfo : public CMwMemberInfo   // Specialized class for fields/properties that are a list
                                                        // of object references (FastArray<CMwNod*>/FastBuffer<CMwNod*>)
{
    int unknown0;
    char* pszFriendlyClassName;
    int unknown1;
    CMwClassInfo* pClassInfo;                           // Class of the objects in the list
};

struct CMwMethodInfo : public CMwMemberInfo             // Specialized class for member methods
{
    void* pMethod;                                      // Pointer to the method code
    int numArgs;
    int* pArgClassIDs;                                  // Pointer to an array of numArgs ints
    char** ppszArgNames;                                // Pointer to an array of numArgs char*'s
    int* pArgFlags;                                     // Pointer to an array of numArgs ints
};

struct CMwEnumInfo : public CMwMemberInfo               // Specialized class for members that store an enum value
{
    int unknown;
    int numValues;
    char** ppszValueNames;                              // Pointer to an array of numValues char*'s
};

Late binding

Once you have the CMwMemberInfo of a property of a class, you can access this property without knowing the class's internal structure. Similarly, if you have a CMwMethodInfo, you can call the method without knowing its address. In effect, this lets you use ManiaPlanet functionality as easily from C++ as from ManiaScript; except that by directly accessing the class system, a lot more functionality is available (most methods are actually blocked for use from ManiaScript).

Performing late binding in ManiaPlanet requires three objects:

  • A CMwNod: the object on which you want to perform the action.
  • A CMwMemberInfo: the object that describes the member you want to access. This can be retrieved by getting the CMwNod's CMwClassInfo (pNod->MwGetClassInfo()) and looping over the class's members until you find the one with the name you are looking for.
  • A CMwStack: wraps the CMwMemberInfo. In addition, when calling a method, this contains any method arguments.

A CMwStack is a list of items, where each item has a pointer to a value and a type ID (indicating whether the value is a CMwMemberInfo*, a CMwNod*, a float*, a bool*...). The name of this class is misleading: it doesn't work like a stack (LIFO: the item that was added last is consumed first), but like a queue (FIFO: the item that was added first is consumed first). In addition, it's not a generic container class; it's only used for late binding.

The three late binding scenarios work in the following way:

Getting properties

  • Create a CMwStack (let's call it "stack").
  • Push the CMwMemberInfo* of the property you want to read.
  • Allocate memory large enough to hold the property's value (let's call it "result"). If you are retrieving a primitive value or a CMwNod*, simply allocate 4 bytes. For complex property values like FastBuffer or String, allocate 4 bytes for a pointer to the value, immediately followed by memory for the value itself.
  • Call pNod->GetProperty(&stack, &result).

Example: 

List < class CPlugSolid* >& CFuncClouds::GetSolidFids () const
{
    static CMwMemberInfo* pMemberInfo = MwGetClassInfo ()->GetMemberInfo ( "SolidFids" );
    struct
    {
        List < class CPlugSolid* >* pResult;
        List < class CPlugSolid* > storage;
    } result;
    CMwStack stack;
    stack.Push ( pMemberInfo );
    GetProperty ( &stack, &result );
    return *result.pResult;
}

Setting properties

  • Create a CMwStack.
  • Push the CMwMemberInfo* of the property you want to write.
  • Call pNod->CallMethod(&stack, &value).

Example: 

void CFuncClouds::SetSolidFids ( List < class CPlugSolid* >& value )
{
    static CMwMemberInfo* pMemberInfo = MwGetClassInfo ()->GetMemberInfo ( "SolidFids" );
    CMwStack stack;
    stack.Push ( pMemberInfo );
    CallMethod ( &stack, &value );
}

Calling methods

  • Create a CMwStack.
  • Push the CMwMemberInfo* of the method you want to call.
  • Push any method arguments in the reverse order as they appear in the signature (right to left).
  • If the method has a return value, allocate a variable for it and push this variable.
  • Call pNod->CallMethod(&stack, NULL).

Example: 

uint CGameCtnEditorPluginScriptHandler::CanPlaceBlock ( class CGameCtnBlockInfo* pBlockModel, int3 coord, uint dir )
{
    static CMwMemberInfo* pMemberInfo = MwGetClassInfo ()->GetMemberInfo ( "CanPlaceBlock" );
    uint uiVariantIndex;
    CMwStack stack;
    stack.Push ( pMemberInfo );
    stack.Push ( dir );
    stack.Push ( coord );
    stack.Push ( *reinterpret_cast < CMwNod** > ( &pBlockModel ) );
    stack.Push ( uiVariantIndex );
    CallMethod ( &stack, NULL );
    return uiVariantIndex;
}

Internally, a CMwStack has some optimizations for efficient list storage. The first two items are always stored inside the object itself (m_ContainedItems member). If more items are pushed after these two, a buffer is allocated on the heap and the additional items are stored in there. "Pushing" an item consists of appending it to the back of the list. After the stack is set up, it is passed to GetProperty/CallMethod which processes the items in the same order in which they were pushed. 

class CMwStack
{
public:
    enum eItemType
    {
        ITEM_MEMBER     = 0,            // CMwMemberInfo*
        ITEM_BOOL       = 0x10000001,
        ITEM_OBJECT     = 0x10000002,   // CMwNod*
        ITEM_ENUM       = 0x10000003,
        ITEM_ISO4       = 0x10000004,
        ITEM_VEC2       = 0x10000005,
        ITEM_VEC3       = 0x10000006,
        ITEM_INT3       = 0x10000007,
        ITEM_UINT3      = 0x10000008,
        ITEM_INT        = 0x10000009,
        ITEM_UINT       = 0x1000000A,
        ITEM_FLOAT      = 0x1000000B,
        ITEM_STRING     = 0x1000000C,   // String*
        ITEM_WSTRING    = 0x1000000D    // StringInt*
    };

    struct Item
    {
        void*           m_pValue;       // Always a pointer, even for primitives
        eItemType       m_Type;
    };

private:
    Item                m_ContainedItems[2];    // The first two items in the queue
    Item*               m_pExtraItems;          // Pointer to an array of additional items (item 3 and above)
    short               m_sSize;                // The total number of items in the queue (contained + extra)
    short               m_sExtraItemsCapacity;  // The capacity of the "extra items" array (not counting m_ContainedItems)
    int                 m_iCurrentPos;          // Current reading position in the item queue (starts at 0 and increases)
};

Serialization

Every CMwNod can be read from and written to a file. Typically this is a .gbx file, but it's also possible to "deserialize" a .png file into a CPlugFilePng instance. Serializing and deserializing is easy to do with the following methods: 

int CSystemArchiveNod::LoadFromFid(CMwNod** ppResultNod, CSystemFid* pFid, enum EArchive = 7);
int CSystemArchiveNod::LoadFileFrom(CFastStringInt* pwstrFilePath, CMwNod** ppResultNod, CSystemFids* pFolder, enum EArchive);
int CSystemArchiveNod::SaveFile(CFastStringInt* pwstrFilePath, CMwNod* pNod, CSystemFids* pFolder, enum EArchive, int);

I/O

File and folder structure

ManiaPlanet maintains several filesystem trees in its memory. The files it represents can come from the actual file system, or from archives (.zip/.pak/.Pack.Gbx).

At the root of each file hierarchy, there is a "drive". This should not be confused with a PC harddrive or partition like C:\; instead, it is an isolated ManiaPlanet folder. The following drives exist:

  • Resource: contents of ManiaPlanet\Packs\Resource.pak. Contains common fonts, shaders, textures etc.
  • Personal: the "Personal" folder containing the user's maps, settings etc.
  • ManiaPlanet: the "ManiaPlanet" folder containing ManiaPlanet.exe and the GameData folder.
  • Common: the "Common" folder containing the PacksCache folder etc.
  • Temp: temporary folder.

.gbx files within a drive can freely reference each other, but cross-drive references are not possible. There is one exception to this rule: through a special file format mechanism, any .gbx file can reference a file in the Resource drive.

ManiaPlanet starts off by creating an in-memory representation of the file structure of each of the drives, based on the actual files on the harddrive. Next it loads the .pak and .zip files in ManiaPlanet\Packs and Common\PacksCache. The files in these archives are merged into the previously created file structures. For example, the ManiaPlanet drive has the following (partial) structure: 

ManiaPlanet\
    GameData\
        Translations\       (from harddrive)
        Vehicles\
            Media\
                Texture\    (from ManiaPlanet.zip)
                Material\   (from ManiaPlanet.pak)

The root drives are pointed to by an instance of CSystemEngine, which is in turn pointed to by the singleton instance of CMwEngineMain. The CMwEngines in CMwEngineMain correspond to the CMwEngineInfos in CMwEngineManager; each CMwEngine provides functionality related to that CMwEngineInfo. 

class CMwEngine : public CMwNod {};

class CMwEngineMain : public CMwEngine
{
    enum eEngine
    {
        ENGINE_MWFOUNDATIONS    = 0x01,
        ENGINE_DATA             = 0x02,
        ENGINE_GAME             = 0x03,
        ENGINE_GRAPHIC          = 0x04,
        ENGINE_FUNCTION         = 0x05,
        ENGINE_HMS              = 0x06,
        ENGINE_CONTROL          = 0x07,
        ENGINE_MOTION           = 0x08,
        ENGINE_PLUG             = 0x09,
        ENGINE_SCENE            = 0x0A,
        ENGINE_SYSTEM           = 0x0B,
        ENGINE_VISION           = 0x0C,
        ENGINE_PSY              = 0x0D,
        ENGINE_EDIT             = 0x0E,
        ENGINE_NODEDIT          = 0x0F,
        ENGINE_AUDIO            = 0x10,
        ENGINE_SCRIPT           = 0x11,
        ENGINE_NET              = 0x12,
        ENGINE_INPUT            = 0x13,
        ENGINE_XML              = 0x14,
        ENGINE_MOVIE            = 0x15,
        ENGINE_PTP              = 0x16,
        ENGINE_CYBERDRIVE       = 0x20,
        ENGINE_VIRTUALSKIPPER   = 0x21,
        ENGINE_ADVENTURE        = 0x22,
        ENGINE_LANFEUST         = 0x23,
        ENGINE_TRACKMANIA       = 0x24,
        ENGINE_SORCIERES        = 0x25,
        ENGINE_MG               = 0x26,
        ENGINE_GBXVIEWER        = 0x27,
        ENGINE_GBE              = 0x28,
        ENGINE_MEDIATRACKERAPP  = 0x29,
        ENGINE_RENDERBOX        = 0x2A,
        ENGINE_FBX              = 0x2B,
        ENGINE_QUESTMANIA       = 0x2C,
        ENGINE_SHOOTMANIA       = 0x2D,
        ENGINE_GAMEDATA         = 0x2E
    };

    int unknown0;
    int unknown1;
    FastArray<CMwEngine*> engines;     // To get a particular engine, index this array using the relevant eEngine value.
                                       // E.g. engines[ENGINE_SYSTEM] returns a pointer to the CSystemEngine instance.
};

class CSystemEngine : public CMwEngine
{
private:
    int unknown0;
    void* pSystemManagerFile;
    StringInt unknown1;
    CSystemFidsDrive* pResourceDrive;
    CSystemFidsDrive* pPersonalDrive;
    CSystemFidsDrive* pManiaPlanetDrive;
    CSystemFidsDrive* pCommonDrive;
    CSystemFidsDrive* pTempDrive;
    FastBuffer<Delegate> fidResolvers;
    FastArray<CSystemFid*> resourceFids;    // List of files in Resource.pak. A resourceIndex in a .gbx file
                                            // indexes this array (see GBX wiki page).
    int unknown2;
    CSystemFidsFolder* pGameDataFolder;     // Pointer to the ManiaPlanet\GameData folder
};

class CSystemFid : public CMwNod          // Base class for files
{
public:
    int ArchiveHeaderUserData(CClassicArchive *);         // Reads or writes header user data
    void BuildHeaderUserData(CSystemArchiveNod const &);  // Prepares the header chunk data for writing

private:
    int HeaderUserDataCreateFromArchive(CClassicArchive *, unsigned long size);  // Reads the header chunk data

private:
    int unknown0;
    CSystemFids* pFolder;                 // Folder containing the file
    int flags1;
    int flags2;
    CMwNod* pNod;                         // Object that was deserialized from this file, if any
    FastBuffer<CSystemFid*> childFids;    // Versions of this file that were loaded with different parameters
                                          // (only filled in if this is the main version of the file)
    CSystemFid* pMainFid;                 // Pointer to the main version of the file
                                          // (only filled in if this is a child version of the file)
    byte fidParameters[0x30];
    int unknown1;
    int classID;                          // Class ID of the content of the file (i.e. of pNod)
    void* pHeaderChunks;                  // Header chunk data, directly from the .gbx file,
                                          // starting with numHeaderChunks
    CLoader* pLoader;
    int index;                            // Index of the file in its containing archive (.pak/.zip), if any
};

class CSystemFidFile : public CSystemFid  // Class representing a named file (as opposed to an anonymous in-memory "file")
{
private:
    StringInt wstrName;                   // Name of the file (without folder path)
    int index;
    int sizeLow;                          // Low 32 bits of the file size
    int sizeHigh;                         // High 32 bits of the file size (0 for files < 4GB)
};

class CSystemFidMemory : public CSystemFid
{
private:
    CClassicBuffer* pBuffer;
};

class CSystemFids : public CMwNod         // Base class for folders
{
private:
    int unknown0;
    CSystemFids* pParentFolder;
    CSystemFidsDrive* pDrive;
    FastBuffer<CSystemFid*> files;
    FastBuffer<CSystemFids*> subFolders;
    int unknown1;
};

class CSystemFidsFolder : public CSystemFids    // Named folder
{
private:
    int unknown0;
    int unknown1;
    int unknown2;
    StringInt wstrName;
};

class CSystemFidsDrive : public CSystemFidsFolder {};

Loading files

Loading a file in ManiaPlanet is quite easy: you get the CSystemFid's pLoader and call its OpenBuffer() method, which returns a stream object ready for reading (see Streams section). Once you're done reading, call the loader's CloseBuffer() method to release the stream. 

class CLoader
{
public:
    virtual CClassicBuffer * OpenBuffer(CSystemFid const * pFid, enum EMode, int)const;   // EMode: 1 = read, 2 = write
    virtual void CloseBuffer(CSystemFid const * pFid, CClassicBuffer * pBuffer)const;
};

Streams

ManiaPlanet uses various stream-like objects for reading and writing files. This means that each stream is associated with an array of bytes (like a file), has a certain position within that array, and can either read from or write to the array (optionally making it larger).

CClassicBuffer

Abstract base class that represents a stream. Comparable to C#'s Stream class. Contrary to what the name might suggest, this class does not necessarily contain a memory buffer. 

class CClassicBuffer
{
public:
    CClassicBuffer(void);
    virtual ~CClassicBuffer(void);
    virtual int IsStoringSecured(void)const;
    virtual int IsSeekable(void)const;
    virtual void SetCurOffset(unsigned long);
    virtual CFastStringInt const * GetFileName(void);

    int ReadAll(void * data, unsigned long size);        // Reads the specified number of bytes
    int WriteAll(void const * data, unsigned long size); // Writes the specified number of bytes

    CClassicBufferMemory * CreateUncompressedBlock(void);
    void AddCompressedBlock(CClassicBufferMemory const &);

    unsigned long Skip(unsigned long);
    int IsEqualBuffer(CClassicBuffer &);
    int CopyFrom(CClassicBuffer *);

    static int s_IsBufferExceptions;

private:
    enum EMode;              // 1 = reading, 2 = writing
    bool bDummyWrite;        // If set to true, writing to the CClassicBuffer will
                             // not actually write to the underlying stream.
                             // Instead, the data provided for writing will be used
                             // to affect decryption.
};

CClassicBufferPart

Exposes a subsection of an existing stream as a new stream. 

class CClassicBufferPart : public CClassicBuffer
{
private:
    CClassicBuffer* pInnerBuffer;
    int offset;
    int size;
};

CClassicBufferMemory

In-memory data stream, comparable to C#'s MemoryStream. 

class CClassicBufferMemory : public CClassicBuffer
{
public:
    CClassicBufferMemory(void);
    virtual ~CClassicBufferMemory(void);
    virtual unsigned long Read(void * pTargetBuffer, unsigned long length);
    virtual unsigned long Write(void const * pData, unsigned long length);
    virtual int Close(void);
    virtual unsigned long m10(); // Stub that returns 0
    virtual unsigned long GetCurOffset(void)const;
    virtual unsigned long GetActualSize(void)const;
    virtual bool IsSeekable(void)const;
    virtual void SetCurOffset(unsigned long position);
    virtual unsigned long m24(); // Stub that returns 0
    virtual void AdvanceOffset(unsigned long offset); // SetCurOffset(GetCurOffset() + offset);
    virtual unsigned long GetAllocatedSize(void)const;

    void PreAlloc(unsigned long);
    void Attach(void *, unsigned long);
    void CopyAndDetachBufferMemory(CClassicBufferMemory &);
    void Reset(void);
    void Empty(void);
    void EmptyAndFreeMemory(void);
    int IsEqualBuffer(CClassicBufferMemory const &)const;
    unsigned long WriteVoid(unsigned long);
    unsigned long WriteCopy(CClassicBuffer *, unsigned long);

private:
    void* pData;
    int size;
    int position;
    int capacity;
    int capacityIncrement; // Step size by which to increase capacity when the buffer gets full
};

CSystemFile

Buffered file stream. The class reads 4 KB from the designated file into memory and then services read requests from that buffer, until the buffer is exhausted and the next 4 KB is read. 

class CSystemFile : public CClassicBuffer
{
private:
    HANDLE hFile;     // Obtained using Win32's CreateFile()
    int unknown;
    StringInt wstrFilePath;
    int position;
    int remainingReadahead;    // Remaining unread bytes in the buffer; 4096 - readaheadOffset
    int readaheadOffset;       // Current read offset within the buffer
    byte readaheadBuffer[4096];
};

CSystemFileMemMapped

Memory-mapped file stream. 

class CSystemFileMemMapped : public CClassicBuffer
{
private:
    StringInt wstrFilePath;
    int position;
    int fileSize;
    HANDLE hMapping;     // Handle returned by Win32's CreateFileMapping()
    HANDLE hFile;        // Handle returned by Win32's CreateFile()
    void* pMappedData;   // Data pointer, returned by MapViewOfFile()
};

CClassicBufferZlib

Compresses (when writing) or decompresses (when reading) data from an underlying stream. Comparable to C#'s DeflateStream. 

class CClassicBufferZlib : public CClassicBuffer
{
private:
    int lastError;          // Last zlib error code
    CClassicBuffer* pInnerBuffer;
    z_stream_s z_stream;    // See zlib.h
    int unknown;
    int unknown;
    byte inBuffer[256];
    byte outBuffer[1024];
    int unknown;
    int uncompressedSize;
    bool bCompressing;
};

CClassicBufferCrypted

Encrypts (when writing) or decrypts (when reading) data from an underlying stream. 

class CClassicBufferCrypted : public CClassicBuffer
{
private:
    int unknown;
    CClassicBuffer* pInnerBuffer;
    int position;
    int ivSize;      // Size of the IV (initialisation vector)
    int algo;        // 0: custom xor-based algorithm
                     // 1: Blowfish in CBC mode (most commonly used)
                     // 2: Blowfish in CTR mode
    int dataSize;
    byte key[16];
    void* pBlowfishState;
    int unknown;
    byte iv[8];
    byte ivXor[8];
    bool bEncrypt;
    int unknown;
    byte buffer[256];
    int bufferIndex;
};

There are some nasty details in the way TrackMania/ManiaPlanet handle encryption. For details, see the page on .pak files (for TrackMania). ManiaPlanet further complicates things by adding custom, non-standard behaviour to the CBC mode.

CClassicArchive

Wraps a CClassicBuffer and exposes handy methods for reading/writing bytes, words, dwords, floats etc. in one go. It's like a C# BinaryReader and BinaryWriter in one: there are both reading and writing methods, and also ReadWrite methods that call either the Read or Write method depending on the bWriting field.

Unlike the name might suggest, this class has nothing to do with .zip or .pak archives. 

class CClassicArchive
{
public:
    CClassicArchive(void);
    virtual ~CClassicArchive(void);
    virtual void DoNodPtr(CMwNod * &);
    virtual void DoFid(CSystemFidFile * &);
    virtual void DoFolder(CSystemFidsFolder * &, int, CSystemFidsFolder *);
    virtual int GetArchivingFileName(CFastStringInt &)const;

    void AttachBuffer(CClassicBuffer *);
    CClassicBuffer * DetachBuffer(int);

    void ReadData(void *, unsigned long);
    void ReadBool(bool *, unsigned long);
    void ReadMask(unsigned long *, unsigned long);
    void ReadNat8(unsigned char *, unsigned long, int);
    void ReadNat16(unsigned short *, unsigned long, int);
    void ReadNat32(unsigned long *, unsigned long, int);
    void ReadNatural(unsigned long *, unsigned long, int);
    void ReadNat64(unsigned __int64 *, unsigned long, int);
    void ReadInteger(int *, unsigned long, int);
    void ReadReal(float *, unsigned long);
    void ReadString(CFastString *, unsigned long);
    void ReadString(CFastStringInt *, unsigned long);

    void WriteData(void const *, unsigned long);
    void WriteBool(bool const *, unsigned long);
    void WriteMask(unsigned long const *, unsigned long);
    void WriteNat8(unsigned char const *, unsigned long, int);
    void WriteNat16(unsigned short const *, unsigned long, int);
    void WriteNat32(unsigned long const *, unsigned long, int);
    void WriteNatural(unsigned long const *, unsigned long, int);
    void WriteNat64(unsigned __int64 const *, unsigned long, int);
    void WriteInteger(int const *, unsigned long, int);
    void WriteReal(float const *, unsigned long);
    void WriteString(CFastString const *, unsigned long);
    void WriteString(CFastStringInt const *, unsigned long);

    void DoMarker(char const *);  // Reads or writes markup, like <Thumbnail.jpg> or </Thumbnail.jpg>
    void DoData(void *, unsigned long count);  // Reads or writes the specified number of bytes
    void DoBool(bool *, unsigned long count);  // Reads or writes a truth value
    void DoMask(unsigned long *, unsigned long count);  // Reads or writes a hex value (typically a class ID)
    void DoNat8(unsigned char *, unsigned long count, int isHex);
    void DoNat16(unsigned short *, unsigned long count, int isHex);
    void DoNat32(unsigned long *, unsigned long count, int isHex);
    void DoNatural(unsigned long *, unsigned long count, int isHex);
    void DoNat64(unsigned __int64 *, unsigned long count, int isHex);
    void DoNat128(struct SNat128 *, unsigned long count);
    void DoInteger(int *, unsigned long count, int isHex);
    void DoReal(float *, unsigned long count);
    void DoString(CFastString *, unsigned long count);
    void DoString(CFastStringInt *, unsigned long count);
    void DoStringI18nComment(CFastStringInt *, char const *);

    void SkipData(unsigned long); // Writes the specified number of zeros, or calls CClassicBuffer::Skip() while reading

    static void (__cdecl* s_ThrowCorruptedArchive)(void);
    static int s_AllowUnprintableStrings;

protected:
    int ReadLine(void);    // Reads data from a .gbx file in text format
    void WriteLine(void);  // Writes data to a .gbx file in text format

    static void (__cdecl* s_DeleteMwIdUserDataCallBack)(CClassicArchive *);

private:
    CClassicBuffer* pInnerBuffer;
    bool bWriting;
    bool bTextMode;         // Whether to read/write in readable ASCII text or in binary
    bool bBucr3IsR;
    void* pStringIndices;   // Pointer to cache of known string IDs
};

CSystemArchiveNod

CClassicArchive-derived class used specifically for reading and writing .gbx files. 

class CSystemArchiveNod : public CClassicArchive
{
public:
    CSystemArchiveNod(void);
    virtual ~CSystemArchiveNod(void);
    virtual void DoNodPtr(CMwNod * &); // Reads or writes a reference to an object. If this is the first time the object is encountered, it is (de)serialized at this point.
    virtual void DoFid(CSystemFidFile * &);
    virtual void DoFolder(CSystemFidsFolder * &, int, CSystemFidsFolder *);
    virtual int GetArchivingFileName(CFastStringInt &)const;

    void DoDecode(unsigned long);
    int DoFindNod(CMwNod * &, CSystemFid *);
    int DoIsFileSame(CSystemFid *, CClassicBufferMemory &)
    void DoFormatFromFid(void);
    int AddInternalRef(CMwNod *, unsigned long &, char const *);
    int InsertExternalLocations(CSystemFids *, CFastBuffer<CSystemFids *> *);
    int ExtractExternalLocations(CFastBuffer<CSystemFids *> * &, CSystemFidsDrive *);
    int LoadCurrentHeader(enum EVersion);  // Reads the file header of a .gbx file
    int DoLoadHeader(void);      // Reads the magic 'GBX' and version of a .gbx file, and then calls LoadCurrentHeader()
    int DoSaveHeader(void);      // Writes the header of a .gbx file
    int DoLoadRef(void);         // Reads the reference table of a .gbx file
    int DoSaveRef(void);         // Writes the reference table of a .gbx file
    int DoLoadAllRef(void);      // Reads the header and the reference table of a .gbx file
    int DoLoadBody(CMwNod * &);  // Reads the body of a .gbx file
    int DoSaveBody(void);        // Writes the body of a .gbx file
    int DoLoadAll(CMwNod * &);   // Reads the header, the reference table and the body of a .gbx file
    int DoSaveAll(void);         // Writes the header, the reference table and the body of a .gbx file
    int DoSaveBodyMemory(CMwNod *);
    int DoSave(CMwNod *, unsigned long, enum EArchive, int);
    int DoLoadResource(unsigned long, CMwNod * &);
    int DoFidLoadFile(CMwNod * &);
    int DoFidSaveFile(CMwNod *);
    int DoFidSaveFileSafe(CMwNod *, unsigned long);
    int DoFidLoadRefs(enum EArchive, CClassicBuffer *);
    int DoFidLoadMemory(CMwNod * &);
    int DoFidSaveMemory(CMwNod *);
    int DoLoadFile(CFastStringInt const &, CMwNod * &, CSystemFids *, enum EArchive);
    int DoSaveFile(CFastStringInt const &, CMwNod *, CSystemFids *, unsigned long, enum EArchive, int);
    int DoLoadFromFid(CMwNod * &);
    int DoLoadMemory(CClassicBufferMemory *, CMwNod * &);
    int DoSaveMemory(CClassicBufferMemory *, CMwNod *, unsigned long);
    int DoLoadMemoryTemp(CClassicBufferMemory *, CMwNod * &);
    int DoSaveMemoryTemp(CClassicBufferMemory *, CMwNod *, unsigned long, int);

    static void DoFolder(CClassicArchive &, CSystemFidsFolder * &, CSystemFidsFolder *);
    static int SaveFile(CFastStringInt const &, CMwNod *, CSystemFids *, unsigned long, enum EArchive, int);
    static int LoadFromFid(CMwNod * &, CSystemFid *, enum EArchive);
    static int SaveToFid(CSystemFid *, CMwNod *, unsigned long, int);
    static int LoadFileFrom(CFastStringInt const &, CMwNod * &, CSystemFids *, enum EArchive);
    static int LoadFileToMemory(CSystemFid *, CClassicBufferMemory *)
    static int SaveMemoryToFile(CSystemFid *, CClassicBufferMemory *);
    static int SaveMemory(CClassicBufferMemory *, CMwNod *, unsigned long);
    static int LoadMemoryTemp(CClassicBufferMemory *, CMwNod * &);
    static int SaveMemoryTemp(CClassicBufferMemory *, CMwNod *, unsigned long, int);
    static int LoadResource(unsigned long, CMwNod * &);
    static int Save(CMwNod *, unsigned long, int);
    static int Compare(CMwNod *, CMwNod *, int &);
    static int Duplicate(CMwNod * &, int);
    static void ComputeCrcNat32(CMwNod *, unsigned long &);
    static void ComputeCrcString(CMwNod *, CFastString &);

private:
    void ParametrizedFinalization(CMwNod *);
    void ParametrizedFindOrAddFid(CSystemFid *, CMwNod *);

private:
    word version;                                      // .gbx file version
    word unused;
    CSystemEngine* pSystemEngine;
    FastArray<CSystemFids*> externalFolders;           // References to other folders
    FastBuffer<ExternalNodEntry> externalNodEntries;   // References to other files
    FastBuffer<NodEntry> nodEntries;                   // List of objects,  both in the current .gbx file
                                                       // and from external files
    FastBuffer<CSystemFids*> folders;
    int classID;                                       // Class ID of the main object of the .gbx file
    CSystemFid* pFid;                                  // Reference to the .gbx file

    struct ExternalNodEntry
    {
         CSystemFid* pFid;      // Reference to the file
         int unknown;
         int nodIndex;          // Index in nodEntries of where the external object will be placed
         int folderIndex;       // Index in externalFolders to the folder containing the .gbx file
         bool bUseFid;          // If false,  the file pFid will be deserialized and nodEntries is
                                // populated with the resulting object.
                                // If true,  nodEntries is populated with pFid itself.
    };

    struct NodEntry
    {
        CMwNod* pNod;           // Pointer to the object
        int fileOffset;         // Index in the .gbx file of the object
        bool bRelease;
    };
};

For details on DoNodPtr, see the definition of "noderef" on the GBX page.

A .gbx file contains a serialized main object instance whose type is indicated by classID. This main object can reference other objects, which may be contained in the same .gbx file or in other files.

All objects, both from the current .gbx file and from external files, are stored in the nodEntries list. The main object of the .gbx file is always at index 0. Every time a subobject is read from the .gbx file, or an external file is referenced, the nodEntries index is specified where the object should be stored.

The process for reading a complete *.gbx file looks like this: 

LoadFileFrom -> DoLoadFile -> DoFidLoadFile -> DoLoadAll -> { DoLoadHeader { -> LoadCurrentHeader } -> DoLoadRef -> DoLoadBody }

Reading the file header and body is explained in detail on the GBX page.

Tools

  • TM2Unlimiter (@archive.org) - an open source tool that contains all the code listed above and was kept up to date with the newest ManiaPlanet function addresses.
Retrieved from "https://wiki.xaseco.org/w/index.php?title=ManiaPlanet_internals&oldid=218"