Changes

TrackMania '''.pak files''' are archives that contain a collection of other files, much like .zip archives. They are found in the "Packs" folder in the game installation directory. In ManiaPlanet, there are also '''.Pack.Gbx''' files with the same purpose.

TrackMania '''.pak files''' are archives that contain a collection of other files, much like .zip archives. They are found in the "Packs" folder in the game installation directory. In ManiaPlanet, there are also '''.Pack.Gbx''' files with the same purpose.

In older TM versions, .pak files are both compressed and encrypted (version 3); even the file index containing the file names and directory structure is encrypted. In ManiaPlanet, they include an uncompressed/uncrypted section as well (versions 6+).

In older TM versions, .pak files are both zlib-compressed and encrypted (version 3); even the file index containing the file names and directory structure is encrypted. In ManiaPlanet, they include an uncompressed/uncrypted section as well (versions 6+). Since version 18, compression has switched to {{wp|LZ4 (compression algorithm)|LZ4}} with a specific dictionary.

== Encryption ==

== Encryption ==

.pak files are encrypted using {{wp|Blowfish (cipher)|Blowfish}} in {{wp|Block cipher mode of operation#Cipher Block_Chaining .28CBC.29|CBC mode}} with a 16-byte key. When an encrypted block begins, decryption is initialized by reading an 8-byte, plaintext IV ({{wp|initialization vector}}) from the file. From then on, Blowfish decryption commences.

.pak files are encrypted using {{wp|Blowfish (cipher)|Blowfish}} in {{wp|Block cipher mode of operation#Cipher Block_Chaining .28CBC.29|CBC mode}} with a 16-byte key. When an encrypted block begins, decryption is initialized by reading an 8-byte, plaintext IV ({{wp|initialization vector}}) from the file. From then on, Blowfish decryption commences.

Each .pak file has its own encryption key. The keys for the different packs are found in [[packlist.dat]].

Each .pak file has its own encryption key. For older games, keys for the different packs are found in [[packlist.dat]]. Since ManiaPlanet, these keys are sent from the master server as "sub-keys", then stored in profile chunks for offline purpose, and calculated into final keys during runtime. ManiaPlanet.pak and Resource.pak are the only two that have these sub-keys hardcoded in a function. ManiaPlanet also added an additional key for decrypting the files inside that is calculated differently by hashing the sub-key:

 

There is one gotcha where Nadeo deviates from regular CBC: on the first read and after every 256 bytes read, the current IV is xor'd with a value we'll call ivXor (this happens before the IV is applied to the Blowfish-decrypted block). ivXor is initialized to zero and is also reset to zero every time it is applied, so most times it doesn't have an effect. Crucially though it does get assigned a non-zero value while reading the .pak header, of which the effect usually kicks in somewhere in the middle of the file list. So if you don't take this into account, half your file table will be messed up (which is likely what Nadeo was intending with this trick). The same goes for [[GBX|.gbx files]] embedded in packs.

There is one gotcha where Nadeo deviates from regular CBC: on the first read and after every 256 bytes read, the current IV is xor'd with a value we'll call ivXor (this happens before the IV is applied to the Blowfish-decrypted block). ivXor is initialized to zero and is also reset to zero every time it is applied, so most times it doesn't have an effect. Crucially though it does get assigned a non-zero value while reading the .pak header, of which the effect usually kicks in somewhere in the middle of the file list. So if you don't take this into account, half your file table will be messed up (which is likely what Nadeo was intending with this trick). The same goes for [[GBX|.gbx files]] embedded in packs.

  {

  {

     uint128 headerMD5;

     uint128 headerMD5;

     uint32 metaDataOffset;

     uint32 gbxHeadersStart; // offset to metadata section

     uint32 dataOffset;

     uint32 dataStart;

     if version >= 2:

     if version >= 2:

     {

     {

         uint32 metaDataUncompressedSize;

         uint32 gbxHeadersSize;

         uint32 metaDataCompressedSize;

         uint32 gbxHeadersComprSize;

     }

     }

     if version >= 3:

     if version >= 3:

         uint32 compressedSize;

         uint32 compressedSize;

         uint32 offset;

         uint32 offset;

         uint32[[Class IDs | classID]]; // indicates the type of the file

         uint32 [[Class IDs|classID]]; // indicates the type of the file

         uint64 flags;

         uint64 flags;

     }

     }

There is one special exception with .gbx files. If the specific class ID is 0x07031000 (Control::CControlText), 0x07001000 (Control::CControlBase) is used as input instead.

There is one special exception with .gbx files. If the specific class ID is 0x07031000 (Control::CControlText), 0x07001000 (Control::CControlBase) is used as input instead.

===Header versions 6-8===

A few chunks also do ivXor additionally - Plug::CPlugSurfaceGeom 0900F004 and Scene::CSceneVehicleTuning 0A02E000 / Plug::CPlugVehiclePhyTuning 090EB000. In the code, the ivXor execution is referred to as m_DummyWrite.

byte magic[8]: "NadeoPak"

uint32 version (6, 7, 8)

  uint256    ContentsChecksum;  // Checksum Sha256 of the pack contents starting at next byte

===Header versions 9+===

===Header versions 6+===

  byte magic[8]: "NadeoPak"

  byte magic[8];  // "NadeoPak"

  uint32 version (9 or higher)

  uint32 version;

  if (version >= 6)

  if (version >= 6)

  {

  {

  uint256 ContentsChecksum; // Checksum Sha256 of the pack contents starting at next byte

    uint256 ContentsChecksum;   // Checksum Sha256 of the pack contents starting at next byte

     struct SHeaderFlagsUncrypt

     struct SAuthorInfo

     {

     {

      uint32 version;

        uint32 IsHeaderPrivate    : 1;

      string Login;

        uint32 UseDefaultHeaderKey : 1;

      string Nick;

        uint32 IsDataPrivate      : 1;

      string Zone;

        uint32 IsImpostor          : 1;

      string ExtraInfo;

        uint32 __Unused__          : 28;

     }

     };

    string      ManialinkUrl;

     if (version >= 15)

     if (version >= 13)

        uint32 HeaderMaxSize;   // 0x4000 = Small (16 KB), 0x100000 = Big (1 MB), 0x1000000 = Huge (16 MB)

      string   DownloadUrl;

     if (version >= 7)

     if (version >= 12)

     {

     {

      string   Xml;

      string   TitleID;

                string Xml;

                string TitleID;

     }

     }

     string      UsageSubDir; // to known the kind of pack it is

     Blowfish encrypted: // Unencrypted if neither SHeaderFlagsUncrypt.IsHeaderPrivate nor SHeaderFlagsUncrypt.UseDefaultHeaderKey are set

    if (version >= 10)

     {

     {

      uint32  NbIncludedPacks;

        uint32 GbxHeadersStart; // Offset to the metadata section

      {

         uint256    ContentsChecksum; // Sha256

            uint32 DataStart;     // If version >= 15: DataStart = HeaderMaxSize

         string      Name;

         SAuthorInfo AuthorInfo;

        {

         string     InfoManialinkUrl;

         uint64      CreationDate;

         string     Name;

        if (version >= 11)

          uint32    IncludeDepth;

        } IncludedPacks[];

         if version >= 16:

            uint32 FileSize;

         if version >= 3:

            uint128 unused;

         if version == 6:

            SAuthorInfo;

         uint32 Flags;

            string FolderName;

         }

        uint32 NumFiles;

         FileDesc Files[NumFiles]

            string FileName;

                uint32 Size;

            if version >= 14:

                uint32 IsSeekable        : 1;

                uint32 OpaqueUserData   : 16;

                uint32 __Unused2__      : 13;

            };

     }

     }

  }

  }

===Data===

===Data===

The content of each file starts at Header.dataOffset + FileDesc.offset in the .pak file. First, an 8-byte plaintext IV is read. Then, FileDesc.compressedSize bytes are read and decrypted using Blowfish in CBC mode, using the same key that was used to decrypt the header. If FileDesc.flags & 0x7C is not zero, the file is compressed and should be decompressed using zlib deflate after decryption (it will end up at FileDesc.uncompressedSize bytes).

The content of each file starts at Header.dataStart + FileDesc.offset in the .pak file. From version 15, the data block starts at HeaderMaxSize. First, an 8-byte plaintext IV is read. Then, FileDesc.compressedSize bytes are read and decrypted using Blowfish in CBC mode, using the same key that was used to decrypt the header. If FileDesc.flags & 0x7C is not zero, the file is compressed and should be decompressed using zlib deflate after decryption (it will end up at FileDesc.uncompressedSize bytes).

The type of the file can be found from the extension in the name, or, if this is not available (many file names are actually just hashes), from the [[Class IDs|class ID]].

The type of the file can be found from the extension in the name, or, if this is not available (many file names are actually just hashes), from the [[Class IDs|class ID]].

====GBX compression intricacies====

====GBX compression intricacies====

There are some not-so-obvious details about compressed .gbx files, which may not seem important at first sight but actually make a world of difference when extracting them. Not addressing these ''will'' result in corrupt data. (Note: this is about the deflate compression specific to .pak files, not the LZO compression specific to .gbx data sections. .gbx files in a .pak don't have LZO-compressed data sections (BUUR header)).

There are some not-so-obvious details about compressed .gbx files, which may not seem important at first sight but actually make a world of difference when extracting them. Not addressing these ''will'' result in corrupt data. (Note: this is about the deflate compression specific to .pak files, not the {{wp|Lempel–Ziv–Oberhumer|LZO}} compression specific to .gbx data sections. .gbx files in a .pak don't have LZO-compressed data sections (BUUR header)).

* Files from a .pak are not decrypted and then decompressed in their entirety before they are parsed. Instead, the parser requests data from the decompressor as it goes along, which in turn requests data from the decrypter. Whenever the decompressor's buffer is empty and new data is requested, it requests blocks of 0x100 bytes from the decrypter and decompresses each block into its buffer of 0x400 bytes, until the buffer is full. Then, whenever the parser requests more data, it is simply copied over from the buffer – until the buffer is empty again.

* Files from a .pak are not decrypted and then decompressed in their entirety before they are parsed. Instead, the parser requests data from the decompressor as it goes along, which in turn requests data from the decrypter. Whenever the decompressor's buffer is empty and new data is requested, it requests blocks of 0x100 bytes from the decrypter and decompresses each block into its buffer of 0x400 bytes, until the buffer is full. Then, whenever the parser requests more data, it is simply copied over from the buffer – until the buffer is empty again.

* You ''must'' use the [http://www.zlib.net zlib library]. Don't use a different zlib-compatible implementation, and also don't skip over the zlib header and decompress the data with a deflate implementation. TrackMania very much depends on zlib's behaviour: how soon it starts returning decompressed data after compressed data has been put into it, and how much decompressed data it returns on every iteration.

* You ''must'' use the [http://www.zlib.net zlib library]. Don't use a different zlib-compatible implementation, and also don't skip over the zlib header and decompress the data with a deflate implementation. TrackMania very much depends on zlib's behaviour: how soon it starts returning decompressed data after compressed data has been put into it, and how much decompressed data it returns on every iteration.

== Tools ==

== Tools ==

* {{ArchiveOrg|http://forum.mania-creative.com/thread-2379.html|TMPakTool|https://web.archive.org/web/20121007000654/http://forum.mania-creative.com/thread-2379.html}} - an open source tool which can open and edit .pak files in an Explorer-like interface. Comes with a C# library which you can use in your own applications to work with .pak files. [http://www.mediafire.com/file/y4klvl2op6cpd4p/TMPakTool-v0.2.zip Download] & [http://www.mediafire.com/file/lq3j99k4tk6uav9/TMPakTool-v0.2+src.zip Source code]

* {{ArchiveOrg|http://forum.mania-creative.com/thread-2379.html|TMPakTool|https://web.archive.org/web/20121007000654/http://forum.mania-creative.com/thread-2379.html}} - an open source tool which can open and edit .pak files in an Explorer-like interface. Comes with a C# library which you can use in your own applications to work with .pak files. [http://www.mediafire.com/file/y4klvl2op6cpd4p/TMPakTool-v0.2.zip Download] & [http://www.mediafire.com/file/lq3j99k4tk6uav9/TMPakTool-v0.2+src.zip Source code]

* [[GbxDump]] - a Windows tool to dump and analyze the headers of .Challenge|Map.Gbx, .Replay.Gbx, .Pack.Gbx|.pak, .ObjectInfo.Gbx and .Item.Gbx files.

* [[GbxDump]] - a Windows tool to dump and analyze the headers of .Challenge|Map.Gbx, .Replay.Gbx, .Pack.Gbx|.pak, .ObjectInfo.Gbx and .Item.Gbx files.