In the course of their development, many Indo-European languages modified the system of three grammatical genders − masculine, feminine and neuter − inherited from Proto-Indo-European, either by eliminating neuter or merging feminine with masculine. The reasons and processes of such modifications are often poorly understood. Baltic languages are no exception to this tendency: they lost neuter gender either before the time of historical record (Latvian and Lithuanian) or shortly after it started (Old Prussian). Phonological factors are usually assumed to have played the crucial role in the Baltic neuter loss, causing minimally different masculine and neuter paradigms to merge. However, this account is unsatisfactory, since it can be shown that paradigm merge alone is insufficient for gender merge. In this paper, I propose that contact with genderless Finnic languages shares the responsibility for elimination of neuter in Baltic. Evidence for this comes from the distribution of remnants of old neuter across Baltic languages (the further north the less remnants), as well as the loss of the remaining gender categories in Baltic dialects (Tamian) spoken on the former Finnic territory.