PDA

View Full Version : MDTools GIT repository



Sik
07-04-2011, 02:17 AM
Eh, I needed a place to store all my MD development tools so I decided to open a GitHub repository to store them (in before complaints about how I'm not using it properly and such). So whatever, here we go:
https://github.com/sikthehedgehog/mdtools

Currently stored there:
SLZ (general purpose compression)
UFTC (fast graphics compression)
=P

Chilly Willy
07-04-2011, 04:04 PM
That UFTC is pretty damn nifty! I know it says "real-time" in the readme, but do you have some idea of how fast it decompresses (KB/sec)?

Sik
07-04-2011, 09:20 PM
The assembly version takes up a bit less than 10 scanlines to decompress 16 tiles (regardless of complexity). That's about 3.82% of a NTSC frame, to give you an idea. Originally I had the measurement in the README but then I realized it probably won't hold up for the C version...

Chilly Willy
07-04-2011, 09:32 PM
I've been thinking about compression on 32X games like Wolf3D... the current version uses the data decompressed in rom since you can't fit it all in ram, but a sufficiently fast enough decompression scheme may be worth trying. I can't have a big cache in memory, but if the decompression is fast enough, a smaller cache may be fine. I can probably even have the slave SH2 do the decompression in parallel. I'll have to do some tests...

Sik
07-05-2011, 01:32 AM
OK, I checked those textures and I have no idea what kind of compression can work there o_o; Stupid question, does any texture use more than 16 colors? Because for now you could try turning them into 4BPP (giving each texture its own set of colors, of course). 50% of original size, but yeah, not very useful...

Also just now I bothered to do a stress test of UFTC, I had forgotten to do it >_> Tested against Stephany sprites in Project MD, which seems like a good candidate for this thing:

Uncompressed size: 44,576 bytes (100%)
Compressed size: 19,482 bytes (43.7%)

Well, it isn't much, but it's still some important saving =| This also goes to prove that UFTC works well when used in the domain it was designed for.

Chilly Willy
07-05-2011, 02:35 AM
Wolf3D uses Carmac-format compression. ;) :D

It's basically huffman with RLE, or something like that. I would be decompressing everything, then recompressing it with a command line tool. I'd post the tool so others could do their own conversions. All Wolf32X graphics are 256 colors using the same palette, found at the bottom of the page here:
http://www.wolfenstein3d.co.uk/graphics-readthis.htm

Sik
07-05-2011, 03:08 AM
...if the graphics are already compressed then I don't think there's much that can be done. RLE, though? To be fair those rock textures don't let themselves well to RLE at all =/ I think you're screwed. The best that comes to my mind is try to pull off a quick LZ77 on individual columns and see what you can get out of that.

Chilly Willy
07-05-2011, 04:15 PM
...if the graphics are already compressed then I don't think there's much that can be done. RLE, though? To be fair those rock textures don't let themselves well to RLE at all =/ I think you're screwed. The best that comes to my mind is try to pull off a quick LZ77 on individual columns and see what you can get out of that.

I don't want to use the original W3D compression - I want to try something that is (hopefully) faster. Like I said, I'd decompress the old stuff and recompress it before it gets put in the rom. So instead of Carmac compression, the images would be slz (if I wound up using it). Given that W3D renders by columns, maybe it's worth making a column oriented compression... or perhaps rotating the images 90 degrees (cols to rows).

Archer
07-05-2011, 05:29 PM
I have just downloaded the set from github and compiled slz. It works but only when i specify -16 or -24. I have tested the compression on the first level data from supertux (the original), this is a text file with A LOT of the same text in it so it is easy to compress.

uncompressed: 106,083 (100%)
compressed: 14,019 (13.22%)

I will need to translate the assembly to compile under GCC and add a wrapper "function" that can be called from C, are you interested in adding those files when i have them?

Sik
07-05-2011, 11:22 PM
It works but only when i specify -16 or -24.
Oops, fixed.


I will need to translate the assembly to compile under GCC and add a wrapper "function" that can be called from C, are you interested in adding those files when i have them?
Did you check /slz/md/slz.c?

Archer
07-06-2011, 01:59 AM
Did you check /slz/md/slz.c?
Yes, i have seen it. From what i have heard, "handpicked assembly" runs faster than anything a C compiler can do. That's why i want time critical functions like UFTC and SLZ in assembly.

EDIT: The C version does make it more portable for other processors, like the SH2 in the 32X.

Sik
10-13-2011, 01:54 PM
Completely useless by now since there's a giant "DO NOT USE YET" warning in Echo's repository (https://github.com/sikthehedgehog/Echo), but whatever:

tfi2eif (https://github.com/sikthehedgehog/mdtools/tree/master/tfi2eif): converts TFM Maker instruments into Echo FM instruments. Documentation is provided for both formats (including fixing a mistake in TFM Maker's help file regarding the TFI format - had to look into vopmxtfi's source code to find it out >_>). Echo's format is quite easy to use in sound engines though, so maybe somebody else can find this useful.

EDIT: by the way, you need a C99 compiler for this program, don't bother me to make this compliant with C89 (you know who you are).

Sik
11-09-2011, 06:24 AM
pcm2ewf (https://github.com/sikthehedgehog/mdtools/tree/master/pcm2ewf): converts raw PCM data into Echo PCM instruments. Documentation is provided for Echo's PCM format. Not much else to say, other than this is the official tool for the job.