Start from the question, not from the test. Are you comparing groups, looking for a relationship, or predicting an outcome?
Then ask how the outcome is measured. A continuous outcome, a count, a proportion and an ordered category each lead somewhere different.
Finally, ask whether the observations are independent. Repeated measures on the same person, or people clustered inside the same household or school, rule out the simple tests immediately.
