Tip: you might be interested in my DVD ripping guide ^_^

OS X gives you a pretty wide choice of video players and enconders, most of which are base on the ffmpeg project. Quicktime sucks. The last version improved a lot in terms of usability, exporting videos and other features, but it's all Apple centric. You can't play 90% of the video formats unless you install external plugins that integrate the ffmpeg library into it.

Now, let's focus on the real video players, those that support a LOT of formats. The most popular is by far VLC. It's easy to use, it has many features integrated and it's basically ubiquitous, though it's not the best one. The best choice is undoubtedly MPlayer. I won't even spend my time debating on how MPlayer is better, it' just better.