redump.info Forum

Hi, I am rev-engineering the CDROM format and the various different CDROM-image formats used by various CDR-burning programs.
What I do currently know can be found here http://nocash.emubase.de/psx-spx.htm#cdromdrive (containing some PSX-specifc stuff, general CDROM specs, and specs for NRG, CUE, ISO images).

And, at the moment, I am working on rev-engineering specs for MDS and CCD formats. To make things more difficult, I don't have the Alcohol (CCD) and CloneCD (MDS) programs, and not even a CDROM drive in my current computer, so I am relying on 1 (one) MDS file, and a handful of CCD files that I've found in the internet.

For MDS, it would be very helpful to have more than one file. Could anybody help me there? Things that I am particulary looking for are:
- MDS file with more than 9 tracks (eg. Audio CD, for seeing if the use BCD or non-BCD numbers).
- MDS file with multiple sessions (eg. a recording of a multisession CDR).
- MDS file with EAN (aka CATALOG), ISRC (eg. found on my Audio CDs).
- MDS file with CD-TEXT (found on a few Audio CDs) (maybe more common on homebrew Audio CDRs).
If you can help and could send some MDS files per email, please PM me. I would need only the small MDS file (not the huge MDF file).

For CCD, I have mostly figured out how the format works. There are only two things puzzling me:
The sector size seems to be constant 930h bytes, if that's correct then it's easy to use, if it's wrong, then I've no idea how to obtain the sector size.
The "pregap" (the number of missing sectors at begin of first track) seems to be always 150. Could that be true? Ie. CCD doesn't allow record the first 150 sectors?
There isn't any "PreGapSize" entry in the CCD file, so I would assume that is fixed (unless one is supposed to randomly pick one of the [Entry N]'s, and then to calculate it as "PreGapSize=(Pmin*60+Psec)*75+Pfram-PLBA")

Aside from ISO, CCD, CUE, MDS, NRG - are there any other popular formats that I should be aware of, ie. formats that I should support in my PSX/PSone emulator?
Oh, and are there any existing specs on such formats? Aside from CUE, most of them seem to be left undocumented, so rev-engineering seems to be the only way to deal with them.

nocash wrote:For CCD, I have mostly figured out how the format works. There are only two things puzzling me:
The sector size seems to be constant 930h bytes, if that's correct then it's easy to use, if it's wrong, then I've no idea how to obtain the sector size.

Yes, it's constant.

nocash wrote:The "pregap" (the number of missing sectors at begin of first track) seems to be always 150. Could that be true? Ie. CCD doesn't allow record the first 150 sectors?

Only NRG allows to store the first 150 sectors and a very small amount of drives can dump these (and only via the tricks, not directly).

nocash wrote:There isn't any "PreGapSize" entry in the CCD file, so I would assume that is fixed (unless one is supposed to randomly pick one of the [Entry N]'s, and then to calculate it as "PreGapSize=(Pmin*60+Psec)*75+Pfram-PLBA")

CloneCD dumps always contain the .SUB file, .CCD file contains only the TOC, the index stuff is inside the .SUB.

nocash wrote:Aside from ISO, CCD, CUE, MDS, NRG - are there any other popular formats that I should be aware of, ie. formats that I should support in my PSX/PSone emulator?

CDI? CHD?

As for the MDS files, they are way more complex, than you think. Try to find some from StarForce-protected MDF-MDS images with DPM measured and examine them.

Hi nocash.. wait, are you THE nocash? who made the no$ emus?

Anyway, check out libmirage, it's an open source library for cd/dvd rom image reading, and supports many common and uncommon formats, including MDS.

Hope it helps.

Yes, the no$emus, that was me. Newest one is no$psx (which got me into dealing with cdrom formats).

MikuroK wrote:Anyway, check out libmirage

Yup, I've found that, too. It really contains some useful info. For MDS, it says that DPM is used only for DVDs... so, for the PSX/PSone, I hope that I can ignore that part.
Best would be comparing the libmirage source code with real MDS binaries. Anyone having some MDS files around? (I don't want to spend hours on downloading whole CDROM-images, specially if I don't even know if the ZIP contains an MDS or something else).

F1ReB4LL wrote:Yes, it's constant.
Only NRG allows to store the first 150 sectors
CDI? CHD?

Good to know that MDS [EDIT: oops, I meant CCD, not MDS] sector size and pregap are constant. Haven't yet come across CDI or CHD files (but I'll keep them in mind).

The one thing that appears to be also popular appears to be .BIN.ECM, which is a nice idea (to remove error correction info when it isn't needed), but the way how is implemented in the ECM format is, well... it's somehow inviting to write a book about "what is wrong this file format", the biggest mistake is that it doesn't allow random access without scanning ALL preceeding sectors.
I guess the only way to "support" in an emulator would be to offer an option to convert it back into a regular .BIN file.
Or, another solution would be to scan the ECM file once, and store the missing sector look-up table in a separate .BIN.ECM.TAB file. Or is there already such a workaround-solution? When using RLU-like compression, the table could be quite small (first N sectors with size X, next N sectors with size Y, and so on). Though that might make the mess of numerous file formats even bigger, and I think ECM isn't really worth support.

EDIT: The look-up table idea wouldn't work too well: The ECM type 0 chunks could be, for example, 940h bytes in size (one 930h-byte sector, plus 10h bytes from next sector), so the chunks aren't guaranteed to be located on sector boundaries, supporting that special cases in a look-up file would be almost equally bizarre as the ECM format as such :-/

One thing that I'd prefer would be a good compressed-cdrom format. Is there already such a thing?
Some years ago I came across .CDZ, which didn't compress too well, mostly because it didn't support removing error correction before compression. At least back then, some years it didn't support that.

Oh, and .SBI files. From what I can see, the format is as so (sorry if that was a well kept secret):
The things that I don't understand are:
What is the "Unknown 01h" byte for? Disc number? Subchannel number? Patch type/version? Number of following 10-byte pairs? I have no clue.
And the https://redump.info/guide/libcrypt/ page says that CRC-16 is also scattered on libcrypt'ed cdrom... but the SBI files don't support that?
And the tool for generating LOGs for generating SBIs does throw out something like:
Original sectors: 48
LC1 sectors:      0
LC2 sectors:      0
Other sectors:    16
What does that mean? Does it mean that there are 64 sectors being common candidates to contain protection info on PSX discs? And this case, 16 of them actually containing that info? And what is LC1, LC2, and is it PSX related, too? (Asking because I'd like to support SBI's in my emu.)

nocash wrote:Yes, the no$emus, that was me. Newest one is no$psx (which got me into dealing with cdrom formats).

That's awesome, the first time I ever saw an emulator was back in '99 when I saw someone playing pokemon yellow in no$gmb.

It's interesting that you would want to support so many formats, personally i just mount my images and have my emulators access the virtual cdrom drive, which avoids the need for them to support them all. Though I suppose you're also doing it to learn how they all work, just the same as making a psx emulator...

Any chance you'll make it open source/linux compatible?

nocash wrote:

F1ReB4LL wrote:Yes, it's constant.
Only NRG allows to store the first 150 sectors
CDI? CHD?

Good to know that MDS sector size and pregap are constant. Haven't yet come across CDI or CHD files (but I'll keep them in mind).

lol? MDF (MDS is a descriptor, not image) sector size is exactly NOT constant, can be 2048, 2048+96, 2352, 2352+96, dunno about 2336 and 2336+96, but I guess it's possible.

nocash wrote:For MDS, it says that DPM is used only for DVDs... so, for the PSX/PSone, I hope that I can ignore that part.

Wrong, it is used for CDs as well. And again: look for images of StarForce-protected CDs. IIRC there were even sites with MDS collections for such titles (in case your drive can't measure DPM).

CDZ and CHD are containers and aren't really image formats. CHD is from MAME/MESS, so it's easy to find out all the needed sources. CDZ is from pSX emulator.

Oops, yes, I meant that CCD has constant sectorsize+pregap, not MDS.

I wasn't succesful finding MDS collections. And no info on StarForce+DPM, does that combination really exist? Wikipedia does only mention SecuROM+DPM.

How well is CHD working? Does it allow random access (without needing to decompress the whole cdrom-image)? And does it reach a good compression ratio?

Is it possible that "LC1" and "LC2" and "Other" sectors just refer to some sort of "libcrypt version"?
The "Unknown 01h" byte seems to be always 01h though, no matter of the version.

nocash wrote:I wasn't succesful finding MDS collections. And no info on StarForce+DPM, does that combination really exist? Wikipedia does only mention SecuROM+DPM.

Of course it exists, for StarForce-protected CDs MDS stores all the angle measurements, also there's a ton of guides and tools for working with MDS files with DPM (Advanced MDS Editor, BWA Builder, etc.).

nocash wrote:How well is CHD working? Does it allow random access (without needing to decompress the whole cdrom-image)? And does it reach a good compression ratio?

Yes, it allows the random access (so you can decompress only the needed "chunks", not the whole image). Compression ratio varies, depends on the data and algorythms. Previously it used only the regular zip/zlib, now, IIRC, it has different algorythms (but can't say for sure, better check the sources and/or message boards).

hi nocash

nocash wrote:What is the "Unknown 01h" byte for? Disc number? Subchannel number? Patch type/version? Number of following 10-byte pairs?

this byte defines format of following bytes
specifications of sbi format are given in 'PSEmu Pro CDR plugin interface'

// 1 byte Format: the format can be 1, 2 or 3:
// Format 1: complete 10 bytes sub q data
// Format 2: 3 bytes wrong relative address
// Format 3: 3 bytes wrong absolute address

nocash wrote:And the https://redump.info/guide/libcrypt/ page says that CRC-16 is also scattered on libcrypt'ed cdrom... but the SBI files don't support that?

yes, it's a pity
for exact data can not be reproduced from .sbi files
though it's enought to fool protection algorithm
as console does not return erroneous subchannel data, in such cases it returns previous frame instead
so .sbi basically just stores information where error occured and that's enough to bypass LC

nocash wrote:What does that mean? Does it mean that there are 64 sectors being common candidates to contain protection info on PSX discs? And this case, 16 of them actually containing that info? And what is LC1, LC2, and is it PSX related, too?

LC1 and LC2 are LibCrypt patterns from a viewpoint of data not code
i.e. if you'll look for documents on LibCrypt, they'll reference changes that took place in code
and variations noted in those guides will not correspond to LCx generations in this DB

as i remember it this program was updated long ago for a last time
and could be it does not identify every LC pattern as such
might be some would just be reported as odd sectors
also not everything reported as odd means it's LC

generally LC always has those pairs
and backup copy of pattern further on CD, if data track is large enough
this backup though seems to never get actually used by algorithm
...many strange things with LC, it's as if it was rushed