One can flexibly switch their behavior based on multiple rules. Although neural activity in the lateral intraparietal area (LIP) of monkeys resembles accumulation of sensory information for perceptual decision making, it remains unknown if and how the LIP accumulates relevant information from identical stimuli depending on task demand. To address this issue, we investigated neural activity in LIP while monkeys performed a reaction time task version of a task switching paradigm. Two Japanese macaques were trained to discriminate either direction or depth in a moving random dot stereogram. On each trial, the color of the fixation point indicated whether the monkey had to discriminate motion direction (Up or Down) or stereoscopic depth (Near or Far). The monkeys responded with a saccadic eye movement as soon as a decision was made. To control the difficulty of the tasks, we varied the percentage of coherently moving and binocularly correlated dots of the visual stimulus across trials. While the monkey performed this task, we recorded extracellular activity from isolated LIP neurons. We focused on neurons that showed saccade direction-selective delay activity in a memory-guided saccade task, and measured their build-up activity during the course of a trial. To assess switching performance, we examined switch ratio (SR). Mean SR for the direction and the depth discrimination task was 0.81 and 0.76, indicating that the monkeys proficiently switched between the two tasks. On the other hand, both psychometric and chronometric functions were affected depending on the strength of the irrelevant stimulus feature. The build-up activity of individual LIP neurons depended on both motion coherence and binocular correlation. More importantly, the coherence- or correlation-dependent change in slope of the build-up activity was larger for the relevant compared to the irrelevant stimulus feature. These activities built-up to a fixed firing rate just prior to the saccade regardless of the stimulus and the task. Our results suggest that LIP neurons flexibly accumulate relevant information depending on context to decide where to move the eyes.