Recently, there emerged several fMRI studies investigating the reliability of long-term BOLD responses (e.g., Brennan et al., 2010; Hasson et al., 2010; Kauppi et al., 2010; Lerner et al., 2011). Experimental findings of these studies indicated that reliability introduced scientific phenomena not accounted for by the conventional analysis of response amplitudes with predefined HRF models. In this study, we propose a model-free approach using the intra-class correlation (ICC) to assess the within- and between-subject reliability of blood oxygen level-dependent (BOLD) responses in functional magnetic resonance imaging (fMRI) experiments. The method is sensitive to experimental effects that are reproducible within individual subjects or among a group of subjects, and that would be otherwise missed by using inappropriate models of the data in a model-dependent analysis. In the empirical example, we illustrate the proposed method in a Sternberg verbal working memory task, involving 10 patients with schizophrenia and 10 healthy controls (gender-matched). The empirical results indicate that the between-subject analysis is sensitive to the global BOLD response changes between levels of working memory load (low load versus high load), and the within-subject analysis is particularly informative about a variety of brain activities induced by individual properties independent of working memory load. The patient group has relatively less reproducible activity in the limbic and subcortical regions than the control group. The proposed method provides potential contributions of assessments using both within- and between-subject reliability analyses to cognitive and medical fMRI experiments.