Gamecube GCM Lossless Encoder

[LedZeppelin68]

Hello, let me introduce the tool - GCM Lossless Encoder, now it is NASOS (Not Another Shrinker Or Scrubber)
the purpose is to "optimize" GameCube and Wii images for better archiving
it doesn't delete uncompressable "junk" data, but uses algorithm to reconstruct it
that's why I call it lossless

*this is not shrinker or scrubber*

beta 1:
added batch processing

alpha 4:
fixed wii9 decode (forgot to uncomment in alpha 3)
fixed 2nd GC disc issue without changing format, old .dec files are compatible

alpha 3:
changed the name of utility to "NASOS"
support for Wii dvd images
this is the last alpha, use it only for testing

alpha 2:
works much faster
some statistics added
batch encoding

alpha 1:
first release
support for gamecube dvd images

GameCube TESTs

Animal Crossing (GAFP)
1.34 Gb - RAW image in RAR
26 Mb - optimized image in 7z

Donkey Konga (GKGE)
1.34 Gb - RAW image in RAR
235 Mb - optimized image in 7z

The Legend of Zelda - Four Sword (G4SE)
1.3 Gb - RAW image in RAR
159 Mb - optimized image in 7z

Wii TESTs (All Wii images are around 4.3 GiB in rar)

Resident Evil Archives: Resident Evil Zero (RBHP)
optimized: 2.8 GiB in 7z

Angry Birds Trilogy (SAWE)
optimized: 929 MiB in 7z

Animal Crossing: City Folk (RUUE)
optimized: 370 MiB in 7z

the tool have drag and drop interface
just drop the image or optimized image on it and wait

.NET framework 4.5 needed

[Oddbrother]

This is pretty nice for saving space, but it leaves a lot to be desired if there isn't more you can do with an image after it's optimized.

When an image is optimized, it outputs the file with an iso.dec extension. Is this normal, and can it be reversed with anything so it's usable again?

[LedZeppelin68]

Is this normal, and can it be reversed with anything so it's usable again?

Yes, of course, drop the .DEC file back to utility and you'll get the original one


the utility produces RAW to DEC conversion and DEC to RAW

[do_0m]

No ****ing way. How did you determine the algorithm? Do you plan on releasing the source code?

My go-to for testing was going to be Animal Crossing, but it seems like I don't have any untouched images lying around. I've already used them for various tests.

So I dug up this and went to work.

Here's the SHA256 for good measure. fa4362802e1ea0b891ce90377e0d122dcb8feae22463409e37488cf72335f9a5

When in the "dec" format the program creates:
Baten Kaitos - Eternal Wings and the Lost Ocean - Special Experience Disc (Japan).iso.dec checksums:
MD5: 623de456a34c1ffeae743fe93d668502
SHA1: e6dd27a614e58bbe46a35d35f8b464c2d0f26858
SHA256: 13f0323a71ba3d4c580af277562b296603b825298e268a5dbc5c7d89675d868d

Now I turned off my PC and left it powered-off for 5 minutes, to ensure the padding / garbage data wasn't residing in memory somewhere. I verified the SHA256 hash once again with Hashcalc. It was still 13f0323a71ba3d4c580af277562b296603b825298e268a5dbc5c7d89675d868d.

Then I re-built the original ISO with the program. By the program's timer, it took 28 seconds. Sounds about right, but I wasn't keeping track.

Every hash matches. Even the SHA256. Unless you can generate SHA256 collisions, or the program is sneakily storing the padding data somewhere else on the PC, it seems like the real deal. (I'll of course test rebuilding the ISO on another PC.)

I'm dying to know how this works. One thing to know is that GPack (maybe) and WIT (definitely) have a neat trick to squeeze another ~1-5% out of a Wii disc. As I recall, there are built-in checksums on Wii discs (and maybe GameCube discs too). The checksum algorithm is known, so you can throw them out and simply re-calculate them when rebuilding the original ISO.

[Jackal]

Wow, so someone figured out the "random" padding structure? Maybe the same can be done for Xbox?

[reentrant]

The source code is already released. If you're curious how it's done download .net reflector and load the assemblies into it. The code is not obfuscated and it's clearly visible how it's implemented. Having looked at it I'm pretty sure this project is based on a different project. There are two similarly looking function names: a100026e0 and a10002710. Naming convention is probably taken from some disassembler like IDA and the code is probably taken from HexRays and adapted to .net.

So I believe something happened at Nintendo and there was a leak of some x86 binary. The algorithm there is impossible to reverse engineer.

Code from GCMLENCa1.exe:

private void a10002710(regs reg, BinaryReader br, BinaryWriter bw, uint[] stack)
{
    reg.EAX = stack[(int) ((IntPtr) ((reg.ESP + 4) / 4))];
    reg.ECX = 0;
    reg.ESI = 0xadb5c;
    do
    {
        reg.EDX = 0;
        while (reg.EDX < 0x20)
        {
            reg.EAX *= 0x5d588b65;
            reg.EAX++;
            reg.ECX = (reg.ECX >> 1) | (reg.EAX & 0x80000000);
            reg.EDX++;
        }
        bw.BaseStream.Position = reg.ESI;
        bw.Write(reg.ECX);
        reg.ESI += 4;
    }
    while (reg.ESI != 0xadba0);
    br.BaseStream.Position = 0xadb9cL;
    reg.EAX = br.ReadUInt32();
    br.BaseStream.Position = 0xadb5cL;
    reg.ECX = br.ReadUInt32();
    reg.EDX = reg.EAX;
    reg.ECX = reg.ECX >> 9;
    reg.EDX = reg.EDX << 0x17;
    reg.ECX ^= reg.EDX;
    reg.EAX ^= reg.ECX;
    bw.BaseStream.Position = 0xadb9cL;
    bw.Write(reg.EAX);
    reg.EAX = 0xadb60;
    do
    {
        br.BaseStream.Position = reg.EAX - 4;
        reg.ECX = br.ReadUInt32();
        br.BaseStream.Position = reg.EAX;
        reg.EDX = br.ReadUInt32();
        br.BaseStream.Position = reg.EAX + 60;
        reg.ESI = br.ReadUInt32();
        reg.ECX = reg.ECX << 0x17;
        reg.EDX = reg.EDX >> 9;
        reg.ECX ^= reg.EDX;
        reg.ECX ^= reg.ESI;
        bw.BaseStream.Position = reg.EAX + 0x40;
        bw.Write(reg.ECX);
        reg.EAX += 4;
    }
    while (reg.EAX != 0xae340);
    this.a100026e0(ref reg, br, bw);
    this.a100026e0(ref reg, br, bw);
    this.a100026e0(ref reg, br, bw);
    stack[(int) ((IntPtr) (reg.ESP / 4))] = reg.EDI;
    bw.BaseStream.Position = 0xae384L;
    bw.Write(520);
    reg.ESP -= 4;
    stack[(int) ((IntPtr) (reg.ESP / 4))] = reg.ESI;
    reg.ESP += 4;
}

Still it's very nice quality work

[tossEAC]

I look forward to running some tests Awesome

[tossEAC]

It would be nice if you could add an output directory so the isos can be on one drive and the output files end up on another drive, it would help speed up the converting, as reading from one drive and writing to another gives better speed than reading and writing to the same drive.

Thanks.

Had a re-think, what would be best is drag-n-drop, exactly as it is, nothing changed, but have an option that allows you to set an input and an output folder, for both dec and enc, were it processes all files in the input folder and outputs all files to the output folder, that way you could leave it processing hundreds of files without having to do each file separately, that would speed things up, and would mean you could leave it running while you got some sleep, or watched TV, etc... etc.

[LedZeppelin68]

there is a tool in gc sdk for mastering dvd images. I was surprised, when discovered that it produces this "garbage".
after that, it was only a matter of time to dissassemble it.

a100026e0 and a10002710 - yes, IDA and x64dbg helped

since sdk is available for ages, it's a pity, that there no such tool since

and scene produces so many shrinked, scrubbed, relocated crap.


about XBOX and XBOX360 images, there is also tools in their sdks, but they produces "standard" fatx isos, like those in original xbox scene releases. My thoughts, this iso-container then transfered to MS for final mastering (security sectors, "garbage" padding).

[Jackal]

about XBOX and XBOX360 images, there is also tools in their sdks, but they produces "standard" fatx isos, like those in original xbox scene releases. My thoughts, this iso-container then transfered to MS for final mastering (security sectors, "garbage" padding).

There is also a tool in the SDK for creating a double layer disc layout (it also shows the position of the security placeholders) and iirc it can also generate output images with padding? Although I'm not sure if this is the final padding.