Replacing the Halo Editing Kit with free software

[Kavawuvi 7,026]

The Halo Editing Kit sucks and everyone knows it, and the Halo: CE Mod Tools aren't that much better. How do we replace it? I made a list of stuff that needs to be done and stuff that has already been done.

 

Why should the Halo Editing Kit be replaced?

The original Halo Editing Kit for Halo Custom Edition is unmaintained and unsupported. Like most Win32 applications, it can run on modern systems thanks to Windows's robust backwards compatibility. However, like many old applications, it hasn't exactly been unscathed by the passage of time. For a start, there are numerous user interface issues in both Sapien and Guerilla that have appeared in semi-recent versions of Windows. Not only that, but it also contains numerous limitations and restrictions that may have made sense with the limited 64 MiB memory of the original Xbox but not a PC game even from this time period. There are various other issues that have become well known by the modding community. None of these issues or concerns have or will ever be patched, officially, as the software is not supported.

 

The newer Halo: CE Mod Tools for Halo: Combat Evolved Anniversary is a better option, and while it has fewer limitations overall from the Halo Editing Kit, it isn't a perfect solution. For example, it is closed source and nonfree, requiring you to agree to an additional, restrictive license that doesn't grant you very much freedom to use their software. While it's great that we have an editing kit in the first place, this nonfreedom can be to the detriment in the community in the long run. Luckily 343 Industries have been updating these tools, but for any issues we experience, we are limited in our capacity to deal with them until 343 Industries decides to officially fix it, assuming they do.

 

This is an inherent issue with closed source, nonfree software, and it doesn't just apply to the tools but the games, too. Halo: Custom Edition is fundamentally broken on so many levels, and Halo: Combat Evolved Anniversary, while better, is still a ways away from anything I could recommend.

 

For any issues that do exist, there is no way to fix the software without resorting to actually making changes to the binary, itself (and doing this is quite limiting!). As a result, people end up being stuck with the limitations and bugs of the software. Unfortunately, another inherent issue is that, since the software at least "exists" despite its issues, few people are inclined to create free, open source alternatives. Even if the official software has issues, dealing with a single issue takes less effort than reimplementing the entire editing kit. Obviously, over the long run, it adds up, but most people are more concerned with making a map now than waiting for tools that better address the community's needs, and, of course, because the software is closed source and nonfree, you're restricted to whatever the developers intended to allow you to mod. With the original Halo Editing Kit, you can only modify Halo Custom Edition, and because it has an inferior renderer, people have resorted to dirty workarounds that sacrifice visuals (e.g. using model shaders instead of environment shaders due to fog). With the new Halo: CE Mod Tools, it is intended for use with only Halo: Combat Evolved Anniversary, and as I mentioned before, Halo: Combat Evolved Anniversary has its own issues, plus it doesn't actually even support custom maps. At least not yet.

 

Without modifying the tools or creating new tools, you're only allowed to do whatever the tools allow you to do, and this is not everything you could do or would want to do. And of course, the tools are restricted to Windows on 32-bit x86, and while most people have access to that, it's possible that it may not be an option for most people in the distant future.

 

In the end, it is a waste of time to try to fix something that is closed source and nonfree. While it's great that this software at least exists, we should focus our efforts on replacing the binaries, altogether, with better free, open source software. Usage of the Halo Editing Kit, Halo: CE Mod Tools, or any other "closed" modding tools should be discouraged if there is a better, free, open source replacement. And as you can see, for many use cases, there is.

 

What needs done?

What can we do with free, open source, community-made software?

• How do you edit your tags? Use invader-edit-qt.
• How do you make string list tags? Use invader-string.
• How do you make HUD message text tags? Use Mozzarilla.
• How do you make your bitmaps? Use invader-bitmap.
• How do you make your fonts? Use invader-font.
• How do you make sprites? Use invader-bitmap.
• How do you make cubemaps? Use invader-bitmap.
• How do you make your sounds? Use invader-sound.
• How do you compile your scripts? Use invader-script.
• How do you make physics tags? Use Blender with Halo-Asset-Blender-Development-Toolset (make JMS files) and Mozzarilla (JMS -> physics).
• How do you make model tags? Use Blender with Halo-Asset-Blender-Development-Toolset (make JMS files) and Invader (JMS -> gbxmodel/model) or Mozzarilla (JMS -> gbxmodel).
• How do you make animation tags? Use Blender with Halo-Asset-Blender-Development-Toolset (make JMA files) and Mozzarilla (JMA -> animations).
• How do you build your maps? Use invader-build.
• How do you place your objects? Technically you can use invader-edit-qt, but there's no free, open source 3D scenario editor yet.

What still requires using the closed source, official software?

• How do you make .scenario_structure_bsp tags?
• How do you make collision model tags?
• How do you bake lightmaps?

Milestones to achieve?

Making a HUD from scratch:

• Requires being able to edit tag data (use invader-edit-qt)
• Requires being able to create 2D texture bitmaps (use invader-bitmap)
• Requires being able to create sprites (use invader-bitmap)

Making a simple multiplayer map with stock assets:

• Requires being able to build a map (use invader-build)
• Requires being able to create scenario tags (use invader-edit-qt)
• Requires being able to edit tag data (use invader-edit-qt)
• Requires being able to place objects (technically you can use invader-edit-qt, but Sapien from the Halo: CE Mod Tools is the only way to do it with a 3D preview)
• Requires being able to compile BSPs (use Blender w/ Halo-Asset-Blender-Development-Toolset and tool.exe from the Halo: CE Mod Tools)
• Requires being able to bake lightmaps (use tool.exe from the Halo: CE Mod Tools)

Making a new object completely from scratch:

• Requires being able to edit tag data (use invader-edit-qt or Mozzarilla)
• Requires being able to make model tags (use Blender w/ Halo-Asset-Blender-Development-Toolset with invader-model or Mozzarilla)
• Optionally requires being able to make physics tags (use Blender w/ Halo-Asset-Blender-Development-Toolset with Mozzarilla)
• Optionally requires being able to make textures (use invader-bitmap)
• Optionally requires being able to make sprites (use invader-bitmap)
• Optionally requires being able to make cubemaps (use invader-bitmap)
• Optionally requires being able to make a HUD for weapons, units, and vehicles (see Making a HUD from scratch...)
• Requires being able to make animation tags (use Mozzarilla and Blender w/ Halo-Asset-Blender-Development-Toolset)
• Requires being able to make collision model tags (use Blender w/ Halo-Asset-Blender-Development-Toolset and tool.exe from the Halo: CE Mod Tools)

Making a new campaign map with stock assets:

• Requires being able to edit tag data (use invader-edit-qt)
• Requires being able to make string tags (use invader-string)
• Requires being able to create scenario tags (use invader-edit-qt)
• Requires being able to compile scripts (use invader-script)
• Optionally requires being able to create sounds for voice acting (using invader-sound)
• Requires being able to build singleplayer maps (use invader-build)
• Requires being able to place objects (technically you can use invader-edit-qt, but Sapien from the Halo: CE Mod Tools is the only way to do it with a 3D preview)
• Requires being able to compile BSPs (use tool.exe from the Halo: CE Mod Tools)
• Requires being able to bake lightmaps (use tool.exe from the Halo: CE Mod Tools)

Making a complete game:

• Requires being able to edit tag data (use invader-edit-qt)
• Requires being able to make fonts (use invader-font)
• Requires being able to make bitmaps for interfaces (use invader-bitmap)
• Requires being able to create sounds from scratch for various effects (use invader-sound)
• Requires being able to make multiplayer maps (incomplete)
• Requires being able to make campaign maps (incomplete)
• Requires being able to make objects from scratch (incomplete)

Where do I get this stuff?

• Invader (invader-*): https://opencarnage.net/index.php?/topic/7489-invader/
• Mozz Editing Kit (Mozzarilla): https://opencarnage.net/index.php?/topic/7877-mozz-editing-kit-mek/
• H2V-Blender-JMS-Exporter: https://github.com/General-101/Halo-Asset-Blender-Development-Toolset
• Halo: CE Mod Tools: https://store.steampowered.com/agecheck/app/1532190/

[Schulzy 44]

Pin?

[Kavawuvi 7,026]

Added a section, "Why should the Halo Editing Kit be replaced?", to the top of the post.

 

1 hour ago, Hiralis said:

Pin?

It's a good idea, but it's possible that pinning it right now may be premature. While I do think the topic is worth watching for updates, it isn't yet helpful enough to warrant pinning to the top of a section, plus it's in the Development section, not the Tools section.

 

At the time of writing this, not enough of the Halo Editing Kit has been replaced for anyone to even make a simple box map. If and when enough of the Halo Editing Kit has been replaced that you can start to make complete maps without using the Halo Editing Kit, then I think it will be worth pinning this. For now, I recommend following this topic if you want updates to it. You can do so by clicking the Follow button on the top right corner of the page.

 

I hope that answers your question.

[Kavawuvi 7,026]

As of Invader 0.4.0, font tags can now be created using only open source software (invader-font).

 

Due to the last update being posted sooner than 24 hours ago, the Invader 0.4 announcement will be posted in the Invader topic tomorrow.

[Kavawuvi 7,026]

As of Invader 0.8.0, invader-bitmap can now make sprites, cubemaps, 3D textures, and interface bitmaps. I've updated the original post with this.

 

We're a little over halfway there for a full open source HEK replacement.

[Kavawuvi 7,026]

I've gotten a few requests to add some other community-made tools to this list. Please note that, unless I can locate an up-to-date version of the source code, I will not add tools to this list.

 

While I am happy that there are people in the community who are willing to create and share useful software, the goal of this project is to have open source software. I did not title this list, "Replacing the Halo Editing Kit with community-made software".

 

Also, not only is closed source community software out of the scope of this project, but I personally consider using community-made closed source software to actually be worse than using the Halo Editing Kit regardless of how much better the actual software is. The reason for this is because the software exists and, if it's a replacement, it is often better than the Halo Editing Kit equivalent, thus potential contributors will be less motivated to write an open source replacement, and this effectively screws over the community when everyone has to rely on it.

[Kavawuvi 7,026]

I'm updating the list to reflect the latest community developments. Basically, we can now make sound tags!

• Mozzarilla now has a fully-functional sound tag generator. This tool supports generating sound tags using arbitrary sample rates (I haven't tested this myself - I'm just going by what they said in the Halo CE Reclaimers chat). It isn't in the master branch, though, so you'll need to poke around the MEK repos if you want to test this.
• Invader is getting a sound tag generator soon, too, but there are a few things left for me to work out (Xbox ADPCM encoding, mouth data, and resampling of sound data to Halo's supported 22.05 kHz and 44.1 kHz formats) before I start uploading builds of it. You can test it if you want, though, if you're willing to compile from source.

Both tools support .wav files with the common 16-bit and 24-bit signed PCM formats. Obviously, if you generate an uncompressed sound, you will only get 16-bit uncompressed samples, and Halo decompresses Vorbis audio to 16-bit samples, so 24-bit is going to mostly be useful for convenience.

[Kavawuvi 7,026]

I've just updated the list, since my new invader-sound has been pushed to the master branch of Invader. Right now, it does not do Xbox ADPCM. Then again, neither does tool.exe unless you obtain the codec from the Xbox SDK. So, your only option for lossy compression is Ogg Vorbis for now.

 

The MEK sound compiling branch is still in the process of being merged from what I have heard, so it may be a while before you can get that through the MEK easily. However, it can do Xbox ADPCM without requiring you to commit morally questionable acts or be an original, licensed developer for the Xbox system, so there ya go.

 

Update: I added Xbox ADPCM support to invader-sound. Merry Christmas!

[Kavawuvi 7,026]

Good news to MEK users: Mozzarilla now supports creating both PCM and Xbox ADPCM sound tags! It does not yet support Ogg Vorbis, though, so I'm not going to put it on the list until that's completed, but if you want a GUI-based program that can generate sound tags and you're okay with being limited to uncompressed PCM or low quality ADPCM, you now have an option for that.

[Kavawuvi 7,026]

Since invader-build now makes campaign maps quite well, I've updated the list. That said, if you're using child scenarios that have scripts in them, you will still need to use tool.exe as Invader lacks a script compiler at the moment.