We study the local structure and vibrational dynamics of the brownmillerite-based proton conductors Ba2In2O5(H2O)x, with x = 0.30, 0.76, and 0.92, using infrared spectroscopy, inelastic neutron scattering and ab initio molecular dynamics simulations. Ba2In2O5(H2O)x is found to exhibit two main types of proton sites, H(1) and H(2). The H(1) site is characterised by the coexistence of two intra-octahedral hydrogen-bond geometries, whereas the H(2) site is characterised by inter-octahedral hydrogen bonding. While the strength of the hydrogen bonding is similar for the majority of protons in the two proton sites, ≈10% of the H(2) protons forms unusually strong hydrogen bonds due to local proton environments characterised by an unusually short oxygen–oxygen separation distance of ≈2.6 A. These local proton environments are manifested as two O–H stretch bands in the infrared absorbance spectra, at 255 and 290 meV, respectively. These O–H stretch bands are as well observed in the related class of In-doped perovskite-type oxides, BaInyZr1−yO3−y/2 (0.25 ≤ y ≤ 0.75), suggesting that these perovskites may display brownmillerite-like distortions on a local length scale. In effect, these results point towards a clustering of the In atoms in these perovskite materials. Further, the infrared spectra of Ba2In2O5(H2O)x show a minor evolution as a function of x, because the protons tend to segregate into oxygen-rich hydrogen-rich domains upon dehydration. This points towards a highly anisotropic proton conduction mechanism in partially hydrated phases. This insight motivates efforts to identify ways to avoid phase separation, perhaps by suitable cation substitutions, as a route to accommodate high proton conductivity.