Simon is easy - he's INTJ, (unemotional, shy, logical) like Spock. Just as ESTP (Mal) is the classic type for the captain, INTJ is the classic type for the tormented repressed science officer. And the INTJ has a crush on the ESTP, always.
River I thought was probably INFP (shy, sensitive, intuitive), though Calle makes a case for INTP (more tough minded and intellectual) in the previous comments. I take INFP to be a person who feels a great deal, but protects that feeling with a strong intuitive competence. I am also tempted by INFJ, because this is the typical personality type for token psychic crew member. Difficult one. I want her to be NF because otherwise we don't have any on the ship.
Because I think Book is ENTJ. That is, he is an intellectual strategist who is playing a big game that nobody else sees. 'I don't care whether you are innocent or not': that's a very ENTJ comment. Think Servalan, think Bill from Kill Bill.
Just must add - what about my personality type (not that I'm egotistical or anything)? Hurrah that there is an important character who is ENTP. Sadly, he is a psychotic bounty hunter who threatens to rape Kaylee to death. Yes, it's Jubal Early. So many ENTP strengths - communicative, perceptive, independent. So many ENTP weaknesses - psychotic, exploitative, violent, and way way too eccentric for his own good.