Many transcription factors (TFs) localize in nuclear clusters of locally increased concentrations, but how TF clustering is regulated and how it influences gene expression is not well understood. Here, we use quantitative microscopy in living cells to study the regulation and function of clustering of the budding yeast TF Gal4 in its endogenous context. Our results show that Gal4 forms clusters that overlap with the GAL loci. Cluster number, density and size are regulated in different growth conditions by the Gal4-inhibitor Gal80 and Gal4 concentration. Gal4 truncation mutants reveal that Gal4 clustering is facilitated by, but does not completely depend on DNA binding and intrinsically disordered regions. Moreover, we discover that clustering acts as a double-edged sword: self-interactions aid TF recruitment to target genes, but recruited Gal4 molecules that are not DNA-bound do not contribute to, and may even inhibit, transcription activation. We propose that cells need to balance the different effects of TF clustering on target search and transcription activation to facilitate proper gene expression.