A study found that Dutch-speaking children who prefer egocentric (left/right) reference frame when describing spatial relationships, and Hai||om-speaking children who use a geocentric (north/south) frame had difficulty recreating small-scale spatial arrays using their language-incongruent system (Haun et al., 2011). In five experiments, we reconciled these results with another study showing that English (egocentric) and Tseltal Mayan (geocentric) speakers can flexibly use both systems (Li et al., 2011; Abarbanell, 2010). In replicating and extending Haun et al. (Experiment 1), English- but not Tseltal-speaking children could use their language-incongruent system when the instructions used their non-preferred frame of reference. Perseveration due to task order may explain the discrepancies between present English- and previous Dutch-speaking children, while not understanding task instructions using left/right language may explain why present Tseltal- and previous Hai||om-speaking children had difficulty with their language-incongruent systems. In support, Tseltal-speaking children could use an egocentric system when the instructions were conveyed without left/right language (Experiments 2-4), and many did not know left/right language (Experiment 5). These findings help reconcile seemingly conflicting sets of results and suggest that task constraints, rather than language, determine which system is easier (Experiment 2 vs. 3).