Sign in to follow this  
Followers 0
Kavawuvi

Tritium

11 posts in this topic

Tritium is an open-source library written in Rust. This is not a tool by itself, but I or someone else may end up using it at a later date for a future Halo tool.

 

Features include (for programmers):

  • No segfaults - Rust guarantees that safe code does not result in segmentation fault.
  • Open source - You are free to download the source code. You can use it in your project and modify it as you please (aside from the MIT license), though you can also use it as a crate and simply depend on it with your project, if that's what you want.
  • Easy to use - You only need to provide the map data as a slice to make it usable. When you're done with the map data, you can turn it back into a cache file, if you want. If you're just reading the map file, that's okay, too.
  • Speed - While it's not the main goal of the project, it's notable that it's quite fast. As of the current build from July 16 on a computer with an Intel Core i3-6100 with DDR4 memory at 2666 MHz, Blood Gulch takes around 18 milliseconds to open and just under 44 milliseconds to rebuild.
  • Support for cache files and resource maps - Unprotected Halo PC and Custom Edition map files are supported. You could rebuild the entire maps folder if you wanted to.

 

Features include (for modders):

  • Recursive tag importation - You can import a tag, or recursively import a tag. Importing a tag non-recursively is a little faster, but it will fail if its dependencies cannot be resolved.
  • Tag deletion - You can delete a specific tag, as well as any unreferenced tags. Deleting a tag will automatically remove any references to it.
  • Tag data manipulation - You can insert/remove data into tags, which can be helpful for things such as reflexive addition and subtraction.

 

Takka likes this

Share this post


Link to post
Share on other sites

Tiddy-bits:

Here's a demonstration of Tritium: A BSP swapper

 

"Halo":

 

bkTiIbm.png

 

"343 Guilty Spark":

 

Kfq1H9I.png

 

It copies the BSP as well as scenery, decals, and detail objects, importing any necessary tags along the way.

Takka likes this

Share this post


Link to post
Share on other sites

Here's a demonstration of Tritium: a CE map translator for HaloMD

 

 

Now MD users can run most CE maps without any conversion being done to the map file. This plugin may see use in tandem with Samuco's netcode bridge.

aLTis and WaeV like this

Share this post


Link to post
Share on other sites

Tritium has been updated to 0.4. Here are the changes:

  • Strings in map names and tag paths now use latin1 instead of UTF-8. Halo only supports UTF-16 for unicode and uses ISO 8859-1 for map names and tag paths. This resulted in a new package dependency, encoding, though Cargo should automatically download it and compile it for you.
  • Effect tags' dependencies and addresses have been mapped out. This will lead to less error-prone and more efficient reading time and was necessary for a program I'm working on.
  • Tags now have an implicit property (true/false). Use this instead of checking if resource_index is Some, as sound tags do not utilize a resource index. This modification was important for many CE maps.
  • Cyclical references no longer result in a panic and are handled properly when importing, now.
Takka likes this

Share this post


Link to post
Share on other sites

Here's a demonstration of Tritium: A CE map converter which I have named "Amethyst" for now. This map converter does not require a maps folder to be specified and is much faster than Pearl. It's run in a command line interface, taking each argument as a path to a map to be converted. This allows for batch conversion of multiple maps at once. It's not immune to map protections (if a map is protected, use Deathstar) but shares some code from my Sapphire plugin with HaloMD and is essentially a copy of the same CE converter used in my Fire Dragon application for HaloMD users. It's slightly better than Pearl in some conversions and slightly worse in others.

 

There are two files required for this tool to work and must be placed in the same directory. They are all located in the amethyst.7z download.

 

Download (Windows - 32-bit): amethyst.7z

Takka likes this

Share this post


Link to post
Share on other sites

Here's a demonstration of Tritium: its speed compared to even the Mac version of Eschaton, which happens to be around 3x faster than the Windows version for an unknown reason. Note that the dependency swapping part does not use Tritium, so ignore it if you want. There's also some elevator-like music that YouTube provided.

 

 

If you want some numbers:

 

Expanding

 

Fire Dragon took 204 ms total. Retrieving the map from the SSD took 19 milliseconds and Tritium's tritium::map::Map::from_cache_file() function took 30 milliseconds parsing the opened map file. The remaining time was spent with the application processing Tritium's resulting struct into its user interface.


In comparison, Eschaton's H1.map_expander took 4.69 seconds, nearly 23 times longer than Fire Dragon. One of my assumptions was that Eschaton was being bottlenecked by my SSD, so I put bloodgulch.map onto a RAM disk. As a result, Eschaton took 4.69 seconds.

 

 

Importing

 

The tag used was b30.map's characters\marine_armored\marined_armored actor variant tag.

 

Fire Dragon took 3 milliseconds to extract, with Tritium's tritium::tag::tag_array::TagArray::insert_recursive taking 2 milliseconds when extracting and isolating the tag. It then took 191 milliseconds to import the extracted tag with tritium::tag::tag_array::TagArray::insert_recursive taking 9 milliseconds, with the rest of the time spent on the user interface.

 

In comparison, Eschaton's H1.Map.expandTag took 1.41 seconds to recursively extract and H1.Map.addTags took 1.74 seconds to import, taking 16 times as long.

 

 

Rebuilding

 

Unlike in the video, I reopened bloodgulch.map when recording these numbers, so this is done using a stock bloodgulch.map with no tags imported.

 

Fire Dragon took 117 ms total. Tritium's tritium::map::Map::to_cache_file() function took 60 milliseconds. Saving the result to the SSD took 55 milliseconds.

 

In comparison, Eschaton's H1.map_writer took 2.85 seconds, 24 times longer than Fire Dragon. Writing directly to a RAM disk took 2.67 seconds, an 8% improvement.

 

 

Specs: Intel Core i7-3667U. 8 GB of DDR3 RAM @ 1600 MHz. Mac OS X 10.11.5 "El Capitan"

Takka likes this

Share this post


Link to post
Share on other sites

Tritium 0.5.0 now has support for parsing and rebuilding resource map files (bitmaps.map, sounds.map, and loc.map) as well as creating them entirely from scratch. It won't parse the individual tag data - that's your job if you want to use this functionality. Basically, you can rebuild the entire maps folder if you wanted to.

Takka likes this

Share this post


Link to post
Share on other sites

I've released 0.6.0. This version contains one small improvement for now - BSPs from scenario tags other than the main one can be used when opening. This means that, with the exception of the having both the singleplayer and multiplayer globals tags, you could combine every single tag from every single map into one map file by simply importing them.

 

Lastly, I've been asked a few questions over the past eight months, so I'll answer some of them here.

 

What is Tritium?

Tritium is a library for parsing cache files written in Rust.

 

Does it parse tags?

No. There are a few tag classes it does have to map out to find information, but as for presenting tag data to the user, the tool implementing Tritium will need to do this. Parsing all of the tags would be far better as a separate library.

 

Does it work with protected maps?

It depends on the protection. Maps with only paths/names being obfuscated will work fine. Maps with tag classes being obfuscated may or may not. If a map is protected, then I recommend running a tool such as Deathstar on the map before opening.

 

Does this work with Halo Custom Edition or Halo PC?

It works on both. However, Halo Custom Edition has one main difference over the retail version of Halo - it supports implicit/indexed tags, that is, tags that are implied to be present in your maps folder through either bitmaps.map, sounds.map, or loc.map. Attempting to run a map with such a tag on the retail version of Halo will crash the game with the exception of HaloMD with the Sapphire plugin. Tritium does not actually check the version number when these tags are present, so it is up to you to implement such a check. Fortunately, Tritium makes this simple.

 

Does this work with the Xbox version of Halo?

No. I've considered that it may actually be possible, but it wouldn't be very useful, plus I have no method of verifying if the map works.

 

Why Rust?

Rust is a language that is both fast and safe. Most languages are either one or the other. Note that just because a language isn't "safe" or isn't "fast" does not mean that it is flawed or inferior compared to any other language. Another reason I chose Rust for its crate system and its documentation generator. If needed, you can statically or dynamically link a Rust library to your tool if your tool isn't written in Rust. Many GUI tools I write will be written in multiple languages. Lastly, in many cases, it's platform independent - I can take the same code which works on Windows on x86 and run it on my Raspberry Pi on ARM, and it will work exactly as well.

 

How fast is it?

Very. Even if it was on a Raspberry Pi Model B (Revision 2.0) which utilizes a 700 MHz ARM processor, it would still be faster than Eschaton on an i7-6700K.

 

Isn't a panic a crash? How is it safe if it can still crash?
A panic may crash, but depending on your implementation, you can also handle panics (you probably shouldn't, but you can with std::panic::catch_unwind). A panic is defined behavior, as in, you can take the same exact panicking code, run it 1000 times, and it will panic 1000 times. With a segfault or memory corruption, at best, it may crash 1000 times, but in some instances, it may also continue running, seemingly okay, but very likely not really. This is difficult to debug and also potentially dangerous.

WaeV and Takka like this

Share this post


Link to post
Share on other sites

I've made a few commits that map out tag references in tags like animations in the unstable branch. This gives a 50% speedup when expanding and a 375% speedup when rebuilding (all measured using stock Blood Gulch). I've also mapped out scenario tag pointers, so campaign maps should work properly.

Share this post


Link to post
Share on other sites
Sign in to follow this  
Followers 0
  • Recently Browsing   0 members

    No registered users viewing this page.