Welcome to Open Carnage

A resource for gamers and technology enthusiasts, with unique means of rewarding content creation and support. Have a wander to see why we're worth the time!

002

TiaraCE Map Parser

12 posts in this topic

I'm writing this map parser for TiaraCE, though if anyone wants to use it in their programs, I'm releasing it under the MIT license. This project is written in C++.

 

This map parser attempts to satisfy a few goals:

 

Speed - Everything should load quickly. Although it should load faster on faster PCs, it shouldn't take too long to load on any modern PC. Therefore, I've made it important to test it on the slowest PC I have:

  • CPU: Intel Core i3-4030U @ 1.9 GHz (Haswell Refresh)
  • RAM: 4 GB DDR3 @ 1600 MHz

 

Safety - When you provide a broken map to Halo, it segfaults. I think I can do better than this. In this map parser, maps should either:

  • Not load at all, generating an exception error that the program can handle
  • Load (if possible), but when something is wrong, generate an exception error that the program can handle
  • Load successfully

 

Cross platform - It must compile successfully on Windows (with Visual C++), macOS (with LLVM Clang), and Linux (with GCC). It must also function identically between platforms. Note that I am using the C++11 standard.

 

Tag mapping - I'm mapping out all of the tag classes using data from Guerilla and Sparky's Plugins as well as any data of my own. This will allow the programmer to easily access tag data without having to search for offsets (assuming that data is mapped out).

 

These maps are supported -

  • Halo PC (retail and Custom Edition)
  • OG Xbox (both compressed and decompressed)

 

I'm also planning on including a map deprotector later down the line to handle maps that are protected.

 

Source (GitHub): https://github.com/Halogen002/TiaraCEMapParser

 

Here are all of the tag classes that I currently plan on mapping out:

Spoiler
  • actor
  • actor_variant
  • antenna
  • biped
  • bitmap
  • camera_track
  • color_table
  • continuous_damage_effect
  • contrail
  • damage_effect
  • decal
  • detail_object_collection
  • device (subclass)
  • device_control
  • device_light_fixture
  • device_machine
  • effect
  • equipment
  • flag
  • fog
  • font
  • garbage
  • gbxmodel
  • globals
  • glow
  • grenade_hud_interface
  • hud_globals
  • hud_message_text
  • hud_number
  • input_device_defaults
  • item (subclass)
  • item_collection
  • lens_flare
  • light
  • light_volume
  • lightning
  • material_effects
  • meter
  • model
  • model_animations
  • model_collision_geometry
  • multiplayer_scenario_description
  • object (subclass)
  • particle
  • particle_system
  • physics
  • placeholder
  • point_physics
  • preferences_network_game
  • projectile
  • scenario
  • scenario_structure_bsp
  • scenery
  • shader (subclass)
  • shader_environment
  • shader_model
  • shader_transparent_chicago
  • shader_transparent_chicago_extended
  • shader_transparent_generic
  • shader_transparent_glass
  • shader_transparent_meter
  • shader_transparent_plasma
  • shader_transparent_water
  • sky
  • sound
  • sound_environment
  • sound_looping
  • sound_scenery
  • spheroid
  • string_list
  • tag_collection
  • ui_widget_collection
  • ui_widget_definition
  • unicode_string_list
  • unit (subclass)
  • unit_dialogue
  • unit_hud_interface
  • vehicle
  • virtual_keyboard
  • weapon
  • weapon_hud_interface
  • weather
  • wind

 

Tags in blue are verified completed.

 

Tags in green are completed up to Guerilla's tag data fields. There may be hidden values present that do things, but these values aren't in Guerilla.

 

Tags in yellow are partially completed.

 

Tags in in goldenrod are partially completed by extension of a subclass being completed.

 

Tags crossed out have little point to be completed.

 

Tucker933 and ST34MF0X like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

Members of Open Carnage never see off-site ads.

I've updated the post with a list of tag classes I plan on mapping out. Some of these tag classes likely won't get mapped out due to being useless. Because there are 83 tag classes, I've put them in spoiler tags.

 

Note that just because a tag is completed does not mean there aren't any hidden values not listed in Guerilla that I may go back and modify.

Sunstriker7, ST34MF0X and Tucker933 like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

I've added support for (decompressed) Xbox cache files. I can't guarantee that TiaraCE will actually support Xbox maps, but if you want to be able to parse these maps in your own tool, here you go.

Tucker933, ST34MF0X and Sunstriker7 like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

I've started .scenario_structure_bsp tags. Because the collision BSP structure is identical to the collision BSP structure used in collision tags, that part is already completed.

Tucker933 and ST34MF0X like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

Here are a few changes since the last time I posted in this topic:

  • .scenario_structure_bsp tags are now parsed (as much as what Guerilla's tag data reveals; I may go back and look for hidden values later)
  • Compressed Xbox maps are now supported. The map data will be decompressed (inflated) upon being opened.
ST34MF0X likes this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

New features:

  • To go along with being able to decompress Xbox maps, you can now compress cache files using the TiaraMap::compress() function, which then it returns a std::vector<char> with the compressed map. Because the Xbox version of Halo uses the DEFLATE algorithm, you can expect a similar filesize reduction as if you compressed it in a .gz or .zip file, and you can also specify a compression level (-1 [default (6)]; 0 [no compression; fastest] - 9 [best compression; slowest]).
    • This function will also work for Halo PC maps, but neither the retail nor Custom Edition versions of the game support compressed cache files and will result in an exception error should you try to load such a map. However, this map parser should load such maps.
  • I've added a function for calculating the CRC32 checksum of maps. Knowing the CRC32 is required for the Halo Custom Edition netcode in order to join servers. This function does not currently work with Xbox maps and will throw a TiaraMapUnimplementedException() if you use it with one.
     

More changes:

  • There is no longer a separate function for loading Xbox maps. The same constructor is used as the PC map.
  • TiaraMap::get_tag() now returns a reference instead of a pointer, and the tag parsing functions now use this instead of pointers.
  • I've made some minor changes to the functions. TiaraMap::cache_file_header() and the TiaraMap::tag_data_header_*() functions now return const pointers and are const functions, similar to TiaraTag::tag_header().
  • I've corrected some issues that were preventing the map parser from building on clang without warnings. Although I've always tested the repository regularly with these flags: "-Wall -Werror -Wextra -pedantic-errors", I've only been using gcc. It seems clang can be a bit more strict for some things, so I'm now using both toolchains.
  • Internally, the TiaraMap function now uses smart pointers instead of raw pointers. Before, I used raw pointers to allow the caller to manage the memory themselves, potentially improving performance. However, giving this level of control is highly unsafe, and some features won't even work (decompression, for instance, requires allocating a separate buffer).
Tucker933 and ST34MF0X like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

Here's what's gone on the past week.

 

New features:

  • valid_map() now checks model data (PC maps only, currently).
  • Model tags have been added. You can access the vertices with ::vertices() and ::vertex_count() (PC maps only, currently).
  • Some exception errors were added for more errors that may occur with invalid maps or programming.
  • Some exception errors can now tell you the tag path and class of the offending tag.

Changes:

  • zlib was removed from the repository. It's still required to compile this project, however.
    • GCC and Clang have -lz which can link their respective internal zlib library to your project.
    • You can also download it for free from https://zlib.net/
  • A few functions now return or take references instead of pointers.
  • A crc32() checksum function was added for Xbox maps, though this isn't going to give you the same result that's in the header, as I'm unsure how this is calculated.
  • Some documentation comments /// were made.

Bug fixes:

  • The wrong file size was given for wrong_map().

 

ST34MF0X and Tucker933 like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

Vertices and triangles in model/gbxmodel tags can now be accessed. There are three model tag parser classes you can use:

  • TiaraTagModel
    • Xbox models only
    • Can read AND write triangles and compressed vertices (vertices can be decompressed individually with TagModelCompressedVertex::decompress())
       
  • TiaraTagGbxmodel
    • PC/Custom Edition models only
    • Can read AND write triangles and uncompressed vertices
       
  • TiaraTagModelUniversal
    • PC/Custom Edition and Xbox models can be used interchangeably
    • Can read triangles and uncompressed vertices (Xbox models' vertices are automatically decompressed - vertices are stored in a buffer)

TiaraTagModelUniversal is best for manipulating tag data as well as viewing model data in programs that accept both PC and Xbox maps.

 

For editing model data, you will want to use the platform-specific TiaraTagModel or TiaraTagGbxmodel.

Tucker933, Sunstriker7 and ST34MF0X like this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites
1 hour ago, Skeezix the Cat said:

Why no collision model tags?

Those were mapped out a while ago.

 

On 11/17/2017 at 11:18 AM, 002 said:
  • model_collision_geometry

 

Here are the Halo CE structs: https://github.com/Halogen002/TiaraCEMapParser/blob/master/src/tag/hce/hce_tag_model_collision_geometry.h

 

Here is the TiaraCE abstraction: https://github.com/Halogen002/TiaraCEMapParser/blob/master/src/tag/tag_model_collision_geometry.h and https://github.com/Halogen002/TiaraCEMapParser/blob/master/src/tag/tag_model_collision_geometry.cpp

ST34MF0X likes this

status.png?customhost=ProtonNebula.com:1

status.png?customhost=ProtonNebula.com:4
competent.png

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.