Welcome to Open Carnage

A resource for Halo Custom Edition and MCC modding, with unique means of rewarding content creation and support. Have a wander to see why we're worth the time! - EST. 2012

Zatarita

Regular
  • Content count

    153
  • Joined

  • Last visited

  • Raffle Tickets

    0

Everything posted by Zatarita

  1. Found out today that there are more types for the Xbox version. So it's not compete complete. Will update with Xbox specifics later this week
  2. edited for completeness Preface in order to see the contents of the file, you're going to have to decompress the file. I used CEAFlate for this: I believe the file is compressed using the zlib library. The endianess of the file is little. The s3dpak format .S3dpak is an archive style format, similar to a zip, or rar file; however, it is specialized for use in the saber3d engine. The header The header consists of only 4 bytes, and It tells us how many child files are in the parent. After that we have (What I call) the glossary which contains all the the files, and where to find them. Similar to what we’re used to in a book. The glossary The glossary consists of entry points for data. Each entry consists of: 4 bytes: offset 4 bytes: size 4 bytes: string length String (string length bytes): name 4 bytes: type of data stored 8 bytes: pad The offset is simply the pointer to the beginning of the data segment in the file. In combination With the size variable, we can know where the data is located, and how much data to look for. The string length is important, because the strings are NOT null terminate. This means we read the string length, and then directly after we read that many bytes to get the name of the data segment. This alone is enough to extract the data from the file; although, in order to put new data in we need to tell the engine how to interpret the data. This is done with “type”. The type is an enumeration linking a number, to a data type. The values are as follows: 0 : "Scene Data", 1 : "Data", 2 : "Single Player Lines", 3 : "Shader (.fx, .psh, .vsh)", 5 : "Textures Info", 8 : "Sound Data", 10 : "Memory", 11 : "Skull data?", 12 : "Template", 14 : "String List", 16 : "Game Logic (.lg)", 17 : "Breakable glass", 18 : "effects (.gfx)", 22 : "(.grs)", 25 : "rain", 26 : "(.cdt)", 27 : "(.sm)", 29 : "(.vis)" Some numbers aren’t used in CEA. Looking through all the s3dpaks provided with CEA these are the only ones seen throughout. Currently the definitions of these files are unknown to me. I will work towards figuring them out, but for now it goes beyond the scope of this specification.
  3. SeT Saber editing Toolkit is a set of python scripts dedicated towards editing saber files. While i was working on SuP, i realized that the path I was going down would limit the abilities I have to work with the data. I decided I would migrate to a more generalized toolkit, and use this as a backbone to program a more versatile set of tools. I intend to write a full SEK for editing saber files once I have fully mapped out the structures. hopefully, this may be useful to others who would like to create their own modding tools as well. It will be an ongoing project until everything is mapped out. The github has a readme that explains how to use it. Basically it allows programmatic access to the contents of saber files saving others the stress of having to reinvent the wheel every time they want to make a modding program. https://github.com/Zatarita/SeT Currently I have s3dpak, ipak, fmeta, imeta, TexturesInfo, and SceneData defined; however, ipak, and imeta are partially implimented. I am willing to accept any help if you wish to contribute to the project. join the discord for smaller updates. major updates will be brought to this post. You don't have to be a programmer to help! Anyone willing to poke around with a hex editor can be a big help c:
  4. I honestly don't feel this needs to be made; however, for sake of completeness: The fmeta format: The fmeta file format simply links dependent files together. It is as follows: 4 bytes : child count 4 byte : padding string (256 bytes) : name 8 bytes : padding 4 bytes : type (-1 for s3dpak, 1 for .map) 4 bytes : decompressed size (if file is a .map) the decompressed size is -1 (ffff ffff) if it's a s3dpak I guess I should also point out it's a fixed size file coming in at a whopping 18 kb
  5. OK, so I'm running into some stress dealing with importing ipak/imeta into SuP. I decided to take a break, and write this up. IPAK/IMETA First of all. Why are they included together? ipak, and imeta are BASICALLY the same thing, the only difference is the ipak contains the actual texture data. Preface To start, just like with s3dpak, you need to decompress the data with ceaflate. The file format is little endian, and this data is close to complete; however, not 100% The Header The header for is very simple, just 8 bytes. 4 bytes : Child count 4 bytes : padding The Imeta The data present in the Imeta is in both the ipak, and the imeta. I'm assuming the reason for this is so the game doesn't have to load every texture in the ipak, and instead can grab only what it needs. This will make more sense in a min. This makes modding the file a little more difficult, as you need to edit two different files. Think of the ipak as a giant gallery of paintings, and the imeta as the directory. You have a directory for each floor (the imeta for each level) but you can also have an over view of everything in the entire building (the ipak's imeta) The format is as follows: string (256 bytes): texture name with no extension 8 bytes : padding 4 bytes : constant = 1 4 bytes : width 4 bytes : height 4 bytes : constant = 1 4 bytes : mipmap count 4 bytes : face count (only 1 face, or 6 faces for cubemaps) 4 bytes : type (see below) 8 bytes : pad 4 bytes : size 4 bytes : pad 4 bytes : size 4 bytes : offset 4 bytes : pad 4 bytes : size 4 bytes : pad Why the size is there three times, I dunno. From what I've seen poking around, I haven't noticed any variation. This engine seems to be rather redundant in many different ways. The offsets must start at : 0x00290008. meaning there is only enough space for 8192 (0x2000) imetas in an ipak (with 8 bytes of padding). the "type" is a hex enumerated representation of the dxt format used. 0x0a : "AI88" - No block compression. 0x30 : "AI88" (for whatever reason "part plasma" uses this, and that's basically it) 0x46 : "AXT1/OXT1" - BC1 0x49 : "XT3" - BC2 0x4c : "XT5" - BC3 0x4f : "XT5A" - BC4 0x52 : "DXN" - BC5 0x5a : "ARGB8888/XRGB8888" - No block compression The Ipak The Ipak data starts at offset 0x00290008 as stated above. Each entry has it's own header, and the data contained in these blocks are simply just .dds textures stripped of their header. The header is as follows: signature : f0 00 ff ff ff ff 54 43 49 50 02 01 ff ff ff ff 4 bytes : Image Width 4 bytes : image height 4 bytes : unknown (usually 1, there are 5 exceptions, in all cases it's a power of two) 4 bytes : face count singature2 : f2 00 ff ff ff ff (4 bytes unknown) f9 00 ff ff ff ff 4 bytes : mip map count start of data: ff 00 ff ff ff ff .... .... footer : 01 00 ff ff ff ff There are a few unknowns in here I'm working to figure out, but for the most part the things labeled "signature" are fairly consistent (except the 4 bytes of unknown in the middle) Most of this is pretty self explanatory. (You can reconstruct the .dds header using the known information using : https://docs.microsoft.com/en-us/windows/win32/direct3ddds/dds-header) The Ipak also has padding tacked onto the end of each file. It's literally just 2,097,151 (0x1fffff) bytes of null data.
  6. Interesting, I actually haven't looked into the blam side of things yet; however, I'll look into it see what I can figure out. That could explain the other "size" variables
  7. I don't know what you mean, I haven't noticed any discrepancies. I ran into an issue originally with SuP, and I found out the issue was rawtex. It didn't extract mipmap, and face data so it would be cut off.
  8. It'll be his 10,000th post! Don't let me down Tucker, this is means to celebrate! Post memes
  9. What if the good thing IS being a doctor
  10. THE PROPHECY BECOMES REALITY!
  11. Let's add a section for modding "Sweet Baby Girl Summer Fun 2 - Sunny Makeover" CMOOONNNNNN DON'T TEASE ;-; He's making us work for it, WE NEED TO SH*TPOST HARDER
  12. You're right! We should talk to an administrator about that If that happens, I'll be a heritic, and expect to have my armor stripped -Pokemon battle theme plays in the distance-
  13. Is this irony .-.
  14. ipak, Imeta, and fmeta file formats coming soon
  15. H2A-inflate is a quick python script I threw together to decompress halo 2 anniversary .pck files cli style interface (time to embrace my inner kava): h2a-inflate <file> <output> Quick and simple; source might be available if I can compress back in the future. This might also mean I can add h2a support into SuP at some point c: for batch files, I recommend copying the compressed files out of the directory, and creating a text document named "inflate.bat" copy this into it. Make sure h2a-inflate is in the same directory as the compressed file. mkdir inflated FOR %%i IN (*.pck) DO h2a-inflate "%%i" "inflated\%%~ni.pck" h2a-inflate.rar
  16. See next post for v 0.2.0 information Sabre unPacker is a program that will extract the contents from a s3dpak file (and with future updates hopefully other sabre files) Icon is still being developed. Currently it's in beta version. If there are any features you want to see added, just put them in the comments. To use the program simply open a *compressed* s3dpak file ceaflate will decompress the file and add it to the TMP directory. Select the files you wish to extract, and select a location for them to go. profit. I hope this is a good step towards helping research the engine a bit more. future plans are to repack, and add the ability to read ipak Big thanks to Kavawuvi for letting me use ceaflate. Source License: GNU GPL v3 SuP 0.2.0.rar
  17. SuP v 0.2.0 prebuild v 2 is out! What's new? Added the ability to repack s3dpaks, as well as partial support for ipak. How does it work? Viewing, and exporting data is the same as before! just load up the s3dpak and click extract. Importing data has been added. To use the feature use the select drop down, or right click and go to import. You'll see the new import window open. You can import as many files as you like, just make sure you match the file type to the correct format. You can see the format in the new "types" column in the main window. When importing SceneData you MUST have the file be labeled "SceneData" exactly as shown, caps and everything. After you import you'll see the main window update the new contents of the file, adjust the offsets, and overwrite existing data. If you've changed your mind on importing a file, you can delete it from the file. You can also create a new file from scratch by importing on an empty window (litterally JUST realized I don't have a new file option, will add in v 0.3.0 for now restart the program) Once done, you can save the file with file>Save as.. and it will automatically create the file, and compress if again for you. Ipak Support This version also introduced limited Ipak support. It can read almost all of the data types (currently it just wont save it if it doesn't know how. I figured that was better than a popup every time you tried to extract in a batch) The header is added using Rawtex. (A program developed by Daemon1 and used with permission. Check the readme for details on the program.) To load a Ipak you need to change the open file type to Ipak. You need to make sure the Ipak is in the same location as the .csv file which contains all the texture information. Once ceaflate decompresses the file (Please be patient, the file is huge; it does take some time) It will populate in the main window, just like a s3dpak. From there you can extract one by one(if SuP knows how to handle the format) or you can extract all the files SuP knows how to. Do note you cannot save Ipaks currently, you cannot import into Ipaks, and there are no fail safes to prevent you from trying. the program will more than likely crash if you will. This is still beta release. Big thanks to kavawuvi for CEAFlate, and Daemon1 for rawtex. Source license: GNU GPL v3 Download: Updated in original post
  18. put ceaflate and desired files to decompress in the same folder, and you can use this bat file to decompress in a batch (remove /r to only do current directory, and not subdirectories) FOR /r %%i IN (*.*) DO ceaflate d "%%i" "%%i_decompressed"
  19. Radium now has a more up to date github repo. Just did a bunch of house keeping. Renamed a bunch of files, and reorganized. I wouldn't recommend downloading it, still not 100% functional; however, if you want to follow along. You can watch the repo: https://github.com/Zatarita/Radium
  20. Weclome! I've been working on a new tool to hopefully fill a gap. Radium is going to be a recorded animation creator, and edit. As well as a halo script IDE. I have a small bit of work done at this point, and I wanted to give a run down on what to expect. Starting off with the recorded animation editor. by the initial release I plan to be able to read and edit all recorded animations inside a compiled map, and a scenario file. I decided on working with a project space, instead of modifying individual scenarios at a time. So you can bind as many scenarios as you like to the project. The primary window will have a opengl context which can render actors, as well as vehicles. It will be able to load each of the units in a map, and you can chose which the ra will run on for preview. Each of the vectors can be modified with a rotation gizmo. Along side that is a timeline scrubber, where you can add key points and interpolate between the keys. The timeline will have a curve modifier so you can adjust the weights of the interpolation. Radium will save the recorded animations to the scenario. For convenience it will also be able to call tool (or tool variant) to compile the map(s). basically for beta release I plan to have the following for RAs done: Create a Radium project [x] Bind scenarios to that project[x] Load existing RAs from scenarios[ ] Load existing RAs from compiled map [x] Load RA from a exported snipping. Find any scripted references to the RA and list which units it's actually used on in game[ ] modify/create recorded animations[ ] Timeline scrubber to preview RA[ ] Keys that can be interpolated between (rotations) [ ] Keys that can change player control state (start crouch, start jump)[ ] Keys that can change the players walking direction[ ] Keys that can bind scripts to be called at specific times during RA (say, when character starts to run, call the script that destroys the warthog)[ ] Keys that call other recorded animations to play, while this one pauses and waits for return.[ ] Save the recorded animations to the scenario[ ] Call tool functions to compile map (Convenience) Call halo ce with devmode and console (if you don't already have a seperate shortcut for it already) This would be enough to get us started making custom RAs for halo. Now since halo script, and recorded animations are closely related, I decided I wanted to go ahead and include an IDE into Radium for halo script. This will also be able to gather the scripts in precompiled, and compiled maps and allow us to edit them in the window. I plan to have full syntax color styling, and auto fill. Since radium binds to the scenario, it will also be able to gather any references, and autofill the correct tag type into scripts for you based on the tags in the map. (say you call 'Recording_play <unit> <RA>' It will automatically fill in the available units in the scenario, and the available RAs in the scenario, and throw errors if you entered something not found.) The script editor will also be able to autofill function return types as well. Since Radium has access to the scenario information it will also be able to implement "compiler directive" style commands. It can dynamically fill in tag data from the scenario into the script when it saves/compiles the script. Say for example you want to check to see if a specific tag is available in a map, you can ask radium to check if it exists. It will fill in the true or false when it saves the script. This could also be used for anything really. Say...the name of a tag with local directory, which dependencies it has, how many dependencies its dependencies have, etc. I'm going to ATTEMPT to hook to sapien's .exe and try to find it I can get the function location for compiling scripts. If I can, with a reference to sapien, I could compile a scenarios scripts from inside the editor, and output sapiens output to the output box in radium. This would allow for you to work all in one place, while still getting the scripts compiled the same way they would be if radium hadn't been used. for beta release I plan to have the following: Create Radium Project [x] Bind scenario(s) to the project [x] Load existing scripts from map [x] Decompile script data + find string references. [ ] Load existing scripts from scenario [ ] Decompile script data + find string references. [ ] Modify existing, or create a new script Create script from exported snipping Dictionary for tags in scenario [ ] Dictionary for scripts (with return types) in the scenario [ ] Dictionary for globals in the scenario [ ] Save the scripts, and organize in the scenario folder. [ ] (Compile the scripts by calling sapiens script compile function [ ]) Gather tag information dynamically from scenario at compile. [ ] Beside that there will be some quality of life things like autoback up, and autosave. a few last notes: Everything with an x is what's been done already. After some discussion it will end up open source once it's all said and done. I do work two jobs, so it might take a bit to finish; however, projected time until a beta release could be a month or two. I plan to update this thread as I go so you can follow along with development. Here are some pictures of the progress. I'm not the best at UI design (this is actually my first attempt at a UI with c++ instead of .net); however, I plan to make all the colors customizable as well. (Sorry for the activate windows, I downloaded the windows 10 beta, then it crashed and fucked me out of my windows 8 legit key. I'm not giving them my money again -.-) If there are any features you think would be nice to implement, let me know and I'll see if I can make it happen c:
  21. I think this part alone should be enough for everyone tbh. If the original intention was to create an open source hek, supporting mcc falls outside that scope, and would cause bloat reducing the options for optimizing for its original intention. Besides, with the niche design of mcc, it requires a bit more finesse to keep everything working smoother. If someone REALLY wants it they can fork it and do it; however, I’m certain there will be someone who comes along and can dedicate the appropriate amount of time to their own project and do it right instead of just adding partial support to an existing project. Plus managing expectations is important for project management. When working with something as ambitious as an open source hek, from a programmers perspective it can become overwhelming to add things while you’re going. Almost managing two separate projects at once. i respect your decision. I appreciate the work you put in. Thank you for trying
  22. Welcome! Check out: https://docs.google.com/spreadsheets/d/e/2PACX-1vTjoZRnDje_T08EUo-gy3tBsMWIYrZfFFXbjXJQtewyukZL0Wqnyx_VtUadjBIufLM1kHy1wH0uap96/pubhtml For some tutorials! And horror :o? How do you feel about args? I have a secret love for discomforting videos like smile guide, or don't hug me I'm scared
  23. enchánté Welcome aboard
  24. Progress bar working, text out progress working, loads the scripts currently compiled in the scenario into the scripts panel. Double clicking now loads scripts. Next is globals. EDIT: if a picture is worth a thousand words, then a video must be worth a lot more EDIT TWO to avoid spamming too much in one day: globals loaded, and completely editable from this panel. Upon consolidation of the scripts, they'll be generated and placed at the top