Purpose Several radionuclides with high (60Co, 75Se) and intermediate (169Yb, 153Gd) energies have been investigated as alternatives to 192Ir for high-dose-rate brachytherapy. The purpose of this study was to evaluate the impact of tissue heterogeneities for these five high- to intermediate-energy sources in prostate and head & neck brachytherapy. Methods and Materials Treatment plans were generated for a cohort of prostate (n = 10) and oral tongue (n = 10) patients. Dose calculations were performed using RapidBrachyMCTPS, an in-house Geant4-based Monte Carlo treatment planning system. Treatment plans were simulated using 60Co, 192Ir, 75Se, 169Yb, and 153Gd as the active core of the microSelectron v2 source. Two dose calculation scenarios were presented: (1) dose to water in water (Dw,w), and (2) dose to medium in medium (Dm,m). Results Dw,w overestimates planning target volume coverage compared with Dm,m, regardless of photon energy. The average planning target volume D90 reduction was ∼1% for high-energy sources, whereas larger differences were observed for intermediate-energy sources (1%–2% for prostate and 4%–7% for oral tongue). Dose differences were not clinically relevant ( Conclusions Dw,w underestimates the dose to bones and, to a lesser extent, overestimates the dose to soft tissues for radionuclides with average energies lower than 192Ir. Further studies regarding bone toxicities are needed before intermediate-energy sources can be adopted in cases where bones are in close vicinity to the tumor.