Feature extraction from neuroimaging data and selection of features play an important role in prediction studies such as conversion from mild cognitive impairment (MCI) to Alzheimer's disease (AD). We have reported that brain atlases affect the performance for classification with MRI-based features (Ota et al., 2014). The primary goal of the present study was to evaluate the effects of image modalities as well as brain atlases on the performance for MCI prediction. We also investigated the influence of a feature selection method on the performance. Eighty patients with amnestic MCI (of these, 40 patients (50%) converted to AD within three years) underwent structural MRI and FDG-PET scans at baseline. Using two brain atlases, the Automated Anatomical Labeling (AAL) and the LONI Probabilistic Brain Atlas (LPBA40), we extracted features from the baseline MRI and FDG-PET data, representing gray matter density and relative cerebral metabolic rate for glucose in each region of interest, respectively. We applied linear support vector machines (SVMs) to the classification of the patients. We used a bootstrap aggregating (bagging) method within a leave-one-out cross-validation (LOOCV) loop, and computed the area under the receiver operating characteristic (ROC) curve (AUC) as a measure of classification performance. We performed feature selection using a multiple SVM recursive feature elimination (SVM-RFE) method to compute feature ranking. We statistically analyzed average AUC values across 20 LOOCV tests for the groups of datasets. Multimodal features generally yielded significantly good AUCs as compared with single modality features: MRI-LPBA40 + PET-AAL (0.748) > MRI-AAL + PET-AAL (0.746) > MRI-LPBA40 + PET-LPBA40 (0.725) > MRI-AAL + PET-LPBA40 (0.680). For MRI data, LPBA40 (0.706) was better than AAL (0.655). In contrast, AAL (0.708) was superior to LPBA40 (0.660) for PET data. The SVM-RFE feature selection method showed no significant improvement in the classification performance. Our results suggest that data-specific feature extraction methods can improve the performance for MCI prediction.