Welcome to Open Carnage

A resource for Halo 1 modding and tech, with unique means of rewarding individual content creation and support. Have a wander to see why we're worth the time! EST. 2012

Kavawuvi

Invader

76 posts in this topic
Quote

Extra options that do not preserve themselves in the tag data leads to an issue where bitmaps could end up being changed inadvertently, for example if you used a script to run through the data directory and rebuild every bitmap it finds.

Honestly? Tough shit imo.

Share this post


Link to post
Share on other sites

Members of Open Carnage never see off-site ads.

On 9/26/2019 at 6:55 PM, gbMichelle said:

If you used a better compression scheme that looked better that would also not save in the tag. If you can recreate what you made with your tools using vanilla tool, what is the point of your tool?

 

To be clear, compatibility with legacy is good, but I don't think we should be limiting ourselves to that all the time. The whole draw for this project for me is to be able to do stuff that I've been manually purposefully avoiding tool to accomplish, but automated. Edit: Even if it means adding an extra switch to my command.

To answer your first question, here is what invader-bitmap does that tool.exe doesn't do:

  • invader-bitmap can use png, bmp, and tga files in addition to tif/tiff
  • invader-bitmap can let you create non-2D texture bitmaps as well as bitmaps with any available data format without you ever having to leave the command line interface
  • invader-bitmap can let you overwrite various other bitmap settings like bump height and mipmap count without you ever having to leave the command line interface
  • invader-bitmap can let you specify any data folder and any tags folder rather than using the current directory
  • invader-bitmap can create bitmaps that exceed 16 MiB

As for your other part of the comment, every single parameter used to generate a bitmap tag is stored in the bitmap tag. By doing this, you don't need to re-enter every single parameter every time you re-generate the bitmap tag. This is how tags fundamentally work, and deviating from this puts us in a situation where some settings get saved and some settings don't, negating the point of using tags in the first place.

 

The main goal is that anything made with tool.exe that is re-made in invader-bitmap should function identically. If it doesn't, I consider this a bug that needs fixed.

 

However, the opposite doesn't need to be true: something made with Invader does not necessarily have to work with tool.exe. For example, tool.exe will not process bitmaps that exceed 16 MiB. A 16 MiB limitation more-or-less limits you to these resolutions:

 

With mipmaps:

  • 4096x4096 - DXT1
  • 4096x2048 - DXT3/DXT5
  • 2048x2048 - 16-bit
  • 2048x1024 - 32-bit

Without mipmaps:

  • 8192x4096 - DXT1
  • 4096x4096 - DXT3/DXT5
  • 4096x2048 - 16-bit
  • 2048x2048 - 32-bit

 

However, invader-bitmap doesn't have such arbitrary limits. You can make 4096x4096 32-bit bitmaps, and Invader won't have any issue with them provided they are otherwise valid.

Share this post


Link to post
Share on other sites

Invader is now 0.13.0. It just so happens that this spooky version number happened on a spooky month. Here are the changes since 0.12.0:

 

Changed

  • invader-build: Changed how stubbed tags are created so tag extractors won't try to extract them
  • invader-bitmap: Errors if the tag contains uppercase characters in its path
  • invader-build: Errors if any non-stubbed tags contain uppercase characters in their paths
  • invader-font: Errors if the tag contains uppercase characters in its path
  • invader-string: Errors if the tag contains uppercase characters in its path

Fixed

  • invader-build: Fixed not fixing the render bounding radius if it was less than the bounding radius but non-zero.
  • invader-build: Fixed not setting the weight value for color change permutations in objects.
  • invader-bitmap: Fixed detail fade factor so it matches tool.exe's detail fade factor more closely if not exactly.
  • invader-build: Fixed certain sound permutation file offsets not being correctly marked as internal; this should fix some sounds that sounded fine when built with tool.exe but sounded corrupt when built with invader-build

Removed

  • invader-bitmap: Removed --filter-blur and --filter-sharpen. Tags that have these values set will still have the filter(s) applied. However, for newer tags, you should use an image editor, as you will get similar or better results.
  • invader-bitmap: Removed --sprite-spacing and used tool.exe's broken functionality, instead.
  • invader-bitmap: Removed the ability to set sprite budgets below 32 or above 512.

View the full changelog at https://github.com/Kavawuvi/Invader/blob/master/CHANGELOG.md

 

From what I've seen, people prefer these less frequent but longer update announcements, so I'll keep doing them like this since it's less work for me. The changelog and GitHub repo, however, will be updated as changes are made. So, if you need to read about the more recent changes, then be sure to check that from time to time.

 

As always, you can get the latest build at https://invader.opencarnage.net/builds/nightly/download-latest.html. These are automatically compiled and uploaded via a script on a local machine. These snapshots are built from the latest commit (at the time of building) on the Git repository which may include unreleased features, so these announcements can and probably will get outdated fairly quickly.

 

See you next month!

Share this post


Link to post
Share on other sites

Here's a monthly update for the progress made in October. I hope everyone had a good Halloween last night.

 

Invader is now 0.15.2, and there have been a number of changes, including the new invader-compress program. View this topic for more information.

 

Added

  • invader-archive: Now accepts filesystem paths for the tag if -P is passed
  • invader-bitmap: Now accepts filesystem paths for the image input if -P is passed
  • invader-build: Now accepts filesystem paths for the scenario tag if -P is passed
  • invader-dependency: Now accepts filesystem paths for the tag if -P is passed
  • invader-font: Now accepts filesystem paths for the the TTF file if -P is passed
  • invader-string: Now accepts filesystem paths for the text file if -P is passed
  • invader-crc: Now prints a warning to stderr if the CRC32 in the cache file header is wrong
  • invader-crc: Now errors with a useful error message if a resource map is loaded with it
  • invader-build: Halo Demo / Trial maps can now be built (-g demo)
  • invader-crc: Halo Demo / Trial maps can now be used
  • invader-build: --no-indexed-tags was renamed to --no-external-tags
  • invader-build: Building retail and demo maps now uses the resource maps
  • invader-build: The gain modifier for sound\sfx\impulse\ting\ting.sound is now changed to 0.2 if building for retail or demo and 1.0 otherwise
  • invader-build: Now shows the target engine of the map
  • invader-compress - Compresses cache files using the Zstandard algorithm
  • invader-resource: Added --retail / -R which will allow you to build retail maps
  • invader-build: Copying TagString data now checks for string overflow
  • invader-crc: Works with maps compressed with invader-compress
  • invader-indexer: Works with maps compressed with invader-compress

Changed

  • invader-build: The diagnostic messages now show tags using external raw data when building a retail or demo map
  • invader-dependency: Now uses -R instead of -r for reverse dependencies
  • invader-dependency: Now uses -r instead of -R for recursive dependencies
  • invader-build: Bitmaps and sound data are now shown separately next to their combined total in the diagnostic output
  • invader-build: The number of tags that could be cached/indexed is now shown
  • invader-archive: Now handles exceptions instead of calling abort() while either compiling or parsing cache files
  • invader-build: No longer errors when orphaned model nodes exist
  • invader-bitmap: Made detail fade factor closer to tool.exe's output - still needs more work but it's remarkably close

Fixed

  • invader-build: Fixed decals with null references crashing the game
  • invader-bitmap: Fully implemented and fixed 3D textures
  • invader-dependency: Fixed an issue where it didn't include a null terminator, causing some tags to fail to open
  • invader-archive: Fixed an issue where some stock HEK .gbxmodel tags tried (and failed) to be archived as .model tags
  • invader-bitmap: Fixed an issue where sprites spanning the entire width wouldn't be detected
  • invader-archive: Fixed an issue where the root tag would have its full system path included when using -s

As always, you can get the latest build at https://invader.opencarnage.net/builds/nightly/download-latest.html or view the changelog at https://github.com/Kavawuvi/invader/blob/master/CHANGELOG.md.

 

If anyone has any comments, suggestions, or bug reports, feel free to reply here. Otherwise, see you next month! Stay warm.

Share this post


Link to post
Share on other sites

A mini-update: I converted the tag definitions to the more parsable .json format. This won't significantly impact Invader's performance since these definitions will be transpiled back into C++ headers. However, it will allow me to tag parsers.

 

As a test, I converted them into Eschaton plugins. I use Eschaton for viewing tag data and finding offsets (which for building cache files is VERY useful), and I've found Sparky's plugins to be fairly inaccurate in some cases. Since my definitions are basically used to build cache files, I figured these would actually be more complete and more useful for my purposes. Here's a screenshot of Eschaton with them:

 

dMuvsCD.png

 

As usual, I will post an update to Invader next month. To give you an idea of the size of the update, this month is only half over and there are already a number of changes already logged in the changelog. These changes over 80 commits with about 40 more on the way in a separate branch, and there have been several version bumps.

Share this post


Link to post
Share on other sites

Here's a update on the month of November! Invader is now 0.20.2.

 

Not much has changed, besides adding a new tag extractor and a new meta data viewer as well as fixing a lot of issues, of course.

 

Added

  • New tool: invader-info - Displays meta data of a map file but can also show specific data (-T): build, compressed, compression-ratio, crc32, crc32-mismatched, dirty, engine, map-type-scenario-scenario-path, stub-count, tag-count, tags
  • New tool: invader-extract - Extracts tags from cache files
  • Added a new tag parser
  • invader-build: Added -N to rename scenarios when building
  • invader-build: Added -c for compress (forging CRC now uses -C)
  • invader-build: Strips default data from compressed animations now
  • invader-font: Added support for .otf files

Changed

  • Help menus have been rewritten
  • All tag definitions have been converted to .json format
  • invader-bitmap: Changed -m to -M to avoid confusion with --maps
  • invader-bitmap: The default bump height is now 0.026
  • invader-build: Shows map type when building
  • invader-build: Uses new indexer format (previous index files will not work)
  • invader-build: 32 byte strings are now zeroed out on copy
  • invader-build: The stub count is now shown
  • invader-build: Index parsing is now case insensitive
  • invader-compress: Strings are now zeroed out before copy
  • invader-compress: The default compresson level is now 19
  • invader-indexer: Changed index format to simply list tag paths with extensions

Fixed

  • invader-bitmap: Fixed -P not working with non-.tif files
  • invader-build: Fixed indexed sound data taking up more space than needed in the tag file; this will reduce tag space usage a little more
  • invader-build: Fixed -g not erroring if an invalid engine was given
  • invader-build: Fixed sound looping tags' gain being set to 0
  • invader-build: Fixed some values in shader_transparent_plasma being set to 0
  • invader-build: Fixed biped A In, B In, C In, D In being set to 0
  • invader-build: Fixed some objects' flags being set to 0
  • invader-build: Fixed an undefined behavior issue with firing positions
  • invader-build: Fixed light tag durations not being properly multiplied by 30
  • invader-build: Fixed some sound tags not working as intended
  • invader-build: Fixed lens flare rotation scale not being converted properly
  • invader-build: Fixed setting mouth data and subtitle data to 0 size
  • invader-build: Fixed incorrect footstep sounds being used if not using an MEK-extracted BSP tag

Removed

  • Removed invader-crc. invader-info now fulfills the same exact purpose
  • invader-compress: Levels above 19 were removed; they are pointless and needlessly use extra memory

Another feature added post-0.20.2 was colored diagnostic text output. I'm still figuring out how I'll do this for Windows. Basically, here are some screenshots:

 

XzbmTYO.png

 

12tEA5a.png

 

Lastly, I posted this earlier today on the Halo CE Reclaimers Discord, but basically, I'm considering removing dithering and nearest-neighbor mipmapping from invader-bitmap. Here's the whole spiel from the Discord I gave:

[12:18 PM] Kavawuvi: So, next version of Invader, I'm considering removing dithering as well as nearest-neighbor mipmapping from invader-bitmap. Any thoughts?
[12:20 PM] Kavawuvi: The problem is that the original point of having these bitmap tags having all of this data was so that it provided parameters for the bitmap tool (tool.exe) to generate the bitmap data.
[12:21 PM] Kavawuvi: Previously, in order to satisfy this, I added more flags to specify how to generate bitmap data, and this included dithering and mipmap settings.
[12:22 PM] Kavawuvi: The problem with doing this is that there is no way to know if a tag has these flags or not, since tags lack a versioning system once they're in the map file. For all I know, MCC uses the data where these flags are located for other purposes.
[12:25 PM] Kavawuvi: Considering hardly anyone actually uses Invader let alone invader-bitmap, these features probably won't be missed anyway, so having them in here is more a liability than an asset in my opinion.

Anyway, as always, you can download Windows builds from the Nightly Builds page. https://invader.opencarnage.net/builds/nightly/download-latest.html

 

Stay tuned until next month for the next update!

ST34MF0X, Sunstriker7 and Tucker933 like this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now
  • Recently Browsing   0 members

    No registered users viewing this page.