Wait so the Chinese reading in Japan is different from the Chinese reading in China and most parts of the Chinese speaking world (ignoring the influences of accents)?

Can you provide an example please?

Here's some that are different:

    方法 (method)
    In Chinese:  fāngfǎ "FONG-fah"
    In Japanese: houhou "HOE-hoe"

    日常 (everyday, ordinary)
    In Chinese:  rìcháng "ZI-tchong"
    In Japanese: nichijou "NEE-chee-joe"

    七月 (July)
    In Chinese:  qīyuè "CHEE-yue"
    In Japanese: shichigatsu "SHEE-chee-gah-tsoo"

(Yes, 'shichigatsu' is seriously the Chinese reading of July; the Japanese reading would be 'nanatsuki'.)

And some that are similar:

    開始 (start)
    In Chinese:  kāishǐ "KAI-tsi"
    In Japanese: kaishi "kai-shee"

    第三 (third)
    In Chinese:  dìsān "dee-SAN"
    In Japanese: daisan "dai-san"

(Note that the pronunciation guide is somewhat approximate since certain sounds don't map well and English vowel pronunciation is a mess.)

I've heard that the Japanese on'yomi (Chinese) readings are generally closer to Old Chinese than modern Chinese is. Also, notice that Chinese is tonal and Japanese isn't.

Wow. So what are the times when one would choose one reading over another?

The general rule is: if there's a kun'yomi that fits, use the kun'yomi. Otherwise, use the on'yomi.

Most commonly, multiple kanji next to each other are on'yomi and single kanji are kun'yomi.