Relation to tosec.org

[Eidolon]

Hi y'all, I'd have about 50ish SegaCD games, 50ish Saturn games, 50ish Dreamcast games and a couple of PS1 games to contribute to the database.

While doing some research on the net, I also found tosec.org with a similar approach to the matter and also a large database. How's this site's relation with them? Wouldn't it be a good idea to merge the databases?

Anyway, the processing of doing the "best possible rips" is very cumbersome. Even for only my few games it seems a little like "overkill", at least from a time consumption point of view. Any way that there might be programs released to automate the process?

Best regards,
Eidolon
http://www.eidolons-inn.net

[Jackal]

Welcome

We've tried working together with them in the past, because we too felt it would be better to have one database for all dumps. TOSEC however didn't want to give up on their dumping method, because redumping all their discs using our new method (raw data tracks, write offset correction on audio, no riff headers added to audio) wouldn't be possible.. (we were told that a lot of the discs that were dumped could not be redumped anymore because they were already sold again etc..) Also there have been some differences in opinions on what's the best way to dump discs, which lead to some arguments between certain members.

With that said, a decent amount of people who dumped for TOSEC have also dumped PSX games for our project. We've added new systems over the past months not to try and replicate what TOSEC is doing, but because there were demands of people who wanted to dump the discs with what they believe is the best method. This is because CD dumps from TOSEC (except the data-only ones like 3DO, of which the error correct data can be generated) sometimes have missing samples in the first and the last audio tracks (that couldn't be dumped due to EAC limitations and lack of write offset correction.. TOSEC doesn't think of this as a big deal but we do, because we want perfect 1:1 dumps, even if it's just a few bytes).

I'd still recommend people with Dreamcast discs to team up with TOSEC for that, because they're better organized at that (we have 4 dumps and they have hundreds). The only different for that system is the write offset correction, which in the end only comes down to some shifted bytes.

As for automated tools, the only one we have so far is PerfectRip. The version that we are testing was abandoned and the new version won't be out for aprox. 6 months. Anyway the version that we are testing works pretty good on Plextor drives, so if you have a plextor drive I could send you a test version (unfortunately other brands don't seem to work that well).

It's not much use comparing the two dumping methods (Redump vs TOSEC), because (as harsh as it may sound) we consider their method to be obsolete. We started out with a method that was almost identical to TOSEC's. Then soon after we discovered how to detect the write offset of a disc and how to correct is. The current method is final, even though there's still room for improvement in the speed area. Once you get used to it it will only take a couple of minutes for each disc to dump (unlike EAC, PerfectRip can automatically rip data and audio tracks with the correct gaps).

ps. for SegaCD dumps we work together with the no-intro project - http://gbadat.altervista.org/

ps.2 A google search for redump.org brought up this topic: http://forums.segaxtreme.net/showthread … amp;page=2 so I assume that's where you came from if you you have any more questions let us know

[Eidolon]

Hi, thanks for your warm welcome.

However, after further discussing the issue with people over at SegaXtreme (your link) and my site (Eidolon's Inn - relevant discussion thread) I have decided that making perfect dumps is pretty pointless, at least for the time being when it's such a hassle to produce.

Thanks anyway and good luck with your project - at a free weekend I will at least try to reproduce the results some of your users had - I own some of the same SegaCD and Saturn games which are already in your DB.

Regards,
Eidolon.

[F1ReB4LL]

I have decided that making perfect dumps is pretty pointless, at least for the time being when it's such a hassle to produce.

Perfect dumps for history, like MAME. If you just want to play, bin-cue is enough. But if you want to save a perfect digital copy of your CD - you should waste some time. Remember, that CDs aren't eternal...

[Macarro]

Nice to see you here, Eidolon, I've visited your site for a long time.

As F1ReB4LL has said, it doesn't matter how long takes to make perfect dumps since they are forever. Also, perfect means unique images, wich makes possible to start other projects like No-Intro Screenshot Archive ( http://no-intro.dlgsoftware.net/main.php?lang=1 , sorry for the little spam ). Without a valid dat, you won't be able to start any massive project ever.

Best regards.

[pepsidrinker]

Anyway, the processing of doing the "best possible rips" is very cumbersome. Even for only my few games it seems a little like "overkill", at least from a time consumption point of view. Any way that there might be programs released to automate the process?

I have decided that making perfect dumps is pretty pointless.

*sigh* This is nothing, it takes a couple of minutes of your time (Not including the actual programs doing the dumping but setting up, which you could be doing other things during that time). A real time consuming project but extremely worthy never the less is the [Software Preservation Society] project of preserving magnetic media games.

Lucky for them they don't have a forum so people can come on and say how pointless the work they are trying to accomplish is.

[Jackal]

Also, the replies I got from the guys at redump.org leads me to the conclusion that it is simply too much hassle to go after "perfect" dumps. There is no such thing as a perfect dump.

Taking Eidolon's quote from his forums, I think he misinterpreted my post. We DO believe that our method creates perfect dumps. Perfect in the sense that every single byte of data on the cd is secured (except the lead-in, lead-out and subchannel data which are irrelevant).

To sum up the differences again between our method and the TOSEC one:

- TOSEC extracts the cooked 2048 data track from a raw image, dropping 304 bytes of information in each data sector that are on the cd (many systems like PSX require the complete main channel, so all 2352 bytes/sector to be ripped, there's no real point in having a 2048 bytes/sector data track for preservation, regardless of what other people say.. audio is 2352 bytes/sector and so is data).
- TOSEC leaves out the track02 pregap, because older EAC versions were not capable of dumping it. This pregap can contain relevant data which will be left out in the resulting dump.
- TOSEC adds a 44-byte RIFF (wav) header to each audio track that isn't on the cd.
- TOSEC corrects the read offset only. Redump corrects both the read and the write offset, allowing audio tracks to be put back into the position they were BEFORE manufacturing. This has proven to be a more senseful way of dealing with audio than just correcting the read offset, because after read+write offset correction we now have audio tracks that start exactly at the intended position (often right after the pregap at byte 352800 for instance) and that have matching checksums across different regions for discs with different write offsets (these tracks would have different checksums using the TOSEC method). Some examples: https://redump.info/disc/1777/ https://redump.info/disc/447/ , all Mortal Kombat Trilogy versions, and lots more.

If you still think it's too much work for you (even with PerfectRip) then that's alright. I just wanted to explain why we believe that our method IS perfect. Perfect in the sense that the full contents of the cd are preserved the best possible way. There is still room for improvement for the speed and difficulty of the dumping process, but the output that is achieved is final.

[F1ReB4LL]

BIN/CUE *IS* the best format to use for this stuff. CDRWIN all the way baby.

CDRWIN doesn't have any error correction/verification for audiotracks. You can loose a half of the track and it won't tell you about that (it's a rare case, but anyway). You can also get a dump full of clicks in audio. Audiotracks should be ripped with EAC only, and it should show "OK" for every track. As for Alcohol 120%, there's a long-standing bug, it produces wrong cue-sheets, so, never rip anything into bin/cue with Alcohol.

[Eidolon]

Let's first focus on Sega CD and Saturn games. The structure of those CDs is pretty simple:
- mode 1 data track, only 2048 bytes of data
- 0 to many audio tracks

The conclusion of the discussion on the Inn was the following.

There definitely IS the problem that every drive has a slight offset in reading the audio tracks. However, this offset is less than 1 sector of the CD, i.e. less than 1/75th of a second. So, practically, it does not matter at all!

So, we have concluded that the drive offset introduces a problem in checksumming methodology, not a problem in ripping methodology.

So you need an INTELLIGENT checksumming tool which calculates the CRC32 for the data track and the audio tracks within the BIN file seperately, adjusting dynamically for the drive offset.

Practically, that means that there might still be floating around several BIN/CUE images of a game, with different checksums for the BIN as a whole. But the idea is that for all those BIN files, the "intelligent checksum" is always the same, thus enabling comparability! The BIN/CUE files may have an audio offset of a few 1/75th of a second. That in itself is irrelevant again because you cannot predict the offsetting which happens when burning back a dump to a real CD, or playing it in a real system.

Concerning hard errors in reading the audio tracks, that can simply be checked by ripping the CD twice. If the resulting BIN file is the same, it means that the CD dump is ok. If the resulting BIN files differ in the audio track data, it means the CD is badly scratched, and the CD drive's error correction kicks in, producing different results with each rip. Meaning, the disc is unsuitable for a "perfect" rip anyway.

Consquently, I've begun working on the GoodSegaCD and GoodSaturn projects on the Inn, hoping that this slightly easier method will be adapted by the Sega retrogaming community as a new defacto standard (similar to the GoodGen stuff).

Looking forward to hearing your thoughts on this!

[Jackal]

There is no such thing as "intelligent checksums". There has to be a standard of how reference checksums are calculated before you can disregard the data offset. We prefer to take both the read and the write offset into account when determining the reference, allowing audio tracks (when saved using the standard) to have identical checksums across different regions/games and not just to look at the data integrity the way you are planning to do. It makes me wonder if the benefit of speed will really be that great, because even a minor small scratch on any of your cd's will give you problems dumping and verifying them the 'fast' way. Sooner or later you will propably end up using EAC after all. Anyway, good luck with your projects.

As for GoodGen, I like No-Intro's dat better, because it's more accurate. Then of course I'm not even talking about MAME and how they want to preserve the Sega Genesis roms (splitting the data into separate files exactly like they are stored on the actual rom chips). Most people also consider this 'pointless' while others don't (see the resemblance?).