Proton pump inhibitors (PPIs) are among the most prescribed medications. Previous epidemiological studies have presented contradictory results about PPIs and the risk of dementia. Our objective was to investigate the association between the use of PPIs and an increasing risk of incident AD or non-AD dementias. A community-based retrospective cohort study was conducted based on the data available from 1st January 2002 to 31st December 2015 in the Catalan health service (CatSalut) system. This cohort included all PPI users (N = 36,360) and non-users (N = 99,362). A lag window of 5 years was considered between the beginning of the PPI treatment and the diagnosis of dementia. PPI use was not associated with the risk of AD (adjusted odds ratio (OR) 1.06) (95% CI 0.93–1.21; p = 0.408). A weakly but significantly increased risk of non-AD dementias was observed among PPI users (adjusted OR 1.20, 95% CI 1.05–1.37; p = 0.007). A higher dose of PPIs was not associated with an increased risk of either AD or non-AD dementias (OR 1.20; 95% CI 0.91–1.61 and OR 0.95; 95% CI 0.74–1.22, respectively). Regarding the number of PPIs used, we observed an increased risk of AD (OR 1.47; 95% CI 1.18–1.83) and non-AD dementias (OR 1.38; 95% CI 1.12–1.70) in users of two types of PPIs compared with those who used only one type. We did not find a higher incidence of AD among PPI users, but a weak increase in the risk of non-AD dementias among PPI users was observed.