Resource file offsets and structure

[Kavawuvi 7,026]

I spent a while on this. Lots of programs are being made, and I figured we should keep a lot of things fairly open, so I'm going to post information about the resource maps. In Halo PC, there are two resource maps: bitmaps.map and sounds.map. In Halo CE, there is an additional loc.map, which contains font, ustr, and hmt tag data. These maps are structured the same, though their purposes are different.
 
First, we've got the header. Unlike the header in a cache file, it doesn't have file size, meta offset, daeh, etc. It is only 0x10 in size. Data starts immediately after that. For those of you who don't know C or C++, uint32_t is an unsigned 32-bit integer.
 

typedef struct MapResourceHeader {
    uint32_t type;                  // 0x0; 1 = bitmaps, 2 = sounds, 3 = loc
    uint32_t names_offset;          // 0x4; File offset of names
    uint32_t resource_index_offset; // 0x8; File offset of resource array
    uint32_t resource_count;        // 0xC; Number of resources
} MapResourceHeader;

Similar to cache files, resource maps have headers of their own. They are much simpler. They also have a table of resources, much like how cache files have a table of tags. There isn't any "magic" number that you have to deal with to translate pointers into file offsets.

To get to the index in a cache file, you would go to the index offset in the header. Resource files are no different, though the index is just an array.
 

typedef struct MapResource {
    uint32_t resource_name;         // 0x0; name is relative to names_offset
    uint32_t resource_size;         // 0x4; size of the resource data in bytes
    uint32_t resource_data_offset;  // 0x8; file offset of resource data
} MapResource

Each resource has a name, a size, and a data offset. To get the name of the resource, you would add resource_name to names_offset. The file offset is just resource_data_offset.

Resource names are this format:

Sound (Halo PC) - tagname__range__permutation (ex: sound\sfx\ambience\multiplayer\10synthtones__0__3)
Bitmap (Halo PC) - tagname_bitmap (ex: effects\particles\air\bitmaps\smoke cloud_1)
Halo CE resource maps (bitmaps, sounds, loc) - tagname (ex: ui\large_ui)

Notice that sounds use double underscores, bitmaps use only one underscore.


If you got any questions or comments, let me know.

[NeX 1,756]

So what practical applications does this provide? Will we eventually be able to build our own custom resource files based on this info?

 

Not trying to sound like an ass, it's very interesting and I'd like to know where this is headed

[DeadHamster 954]

This will be very very ridiculously helpful, thank you. Only info I had was an off-hand post Modzy had made, was expecting to do a lot of figuring out myself.

Edited August 7, 2014 by DeadHamster

[Kavawuvi 7,026]

So what practical applications does this provide? Will we eventually be able to build our own custom resource files based on this info?

Efficient PC --> CE conversion to reduce size as much as possible.

Example:

• Bitmap offset in PC map is 34050348
• Search PC bitmaps for index with offset 34050348
• Get name of resource (character\cyborg\bitmaps\cyborg_0)
• Search CE bitmaps for resource (character\cyborg\bitmaps\cyborg_0)
• If exists, set offset to CE bitmap's offset. If it doesn't exist, internalize the bitmap.

---

Resource map building for efficient map set distribution.

Example:

• Get name of bitmap tag in PC or CE map (weapons\assault rifle\fp\bitmaps\compass_plate)
• Search for bitmaps in bitmaps.map for each bitmap for bitmap tag (weapons\assault rifle\fp\bitmaps\compass_plate_0, weapons\assault rifle\fp\bitmaps\compass_plate_1, etc.). If this bitmap exists, set bitmap offset to this bitmap.
• If the bitmap does not exist, add bitmap (weapons\assault rifle\fp\bitmaps\compass_plate_X) data to bitmaps.map. Set bitmap offset to new bitmap offset.

Edited August 7, 2014 by 002

[WaeV 5,049]

So the CE map format allows for this space-saving mechanism, and PC doesn't?

It sounds like you could make a map squasher tool which performs this space-saving mechanism for a given set of maps.

[Kavawuvi 7,026]

So the CE map format allows for this space-saving mechanism, and PC doesn't?

It sounds like you could make a map squasher tool which performs this space-saving mechanism for a given set of maps.

Such a tool would be useful for releasing an entire set of maps, which could probably save hundreds of megabytes from large projects.

[NeX 1,756]

Such a tool would be useful for releasing an entire set of maps, which could probably save hundreds of megabytes from large projects.

 

*ahem*