This paper presents a vision-based virtual eyeglasses fitting system, which transforms the user's computer into virtual mirror using a webcam. With the system, users can virtually try on glasses to see how a particular glasses style looks and fits their face before they make a purchase. This really saves a lot of time for the selection of eyeglasses in the store. The proposed system first detects the face of a user, and then locates the corners of the user eyes. Based on the two corners, an isosceles right triangle is defined and then used to estimate the pose and scale of the face at any time using an affine transformation. The system thus automatically adjusts glasses images to fit the user's face pose and scale. In addition, a fingertip detection scheme is designed to make another benefit that the users can take virtual wearing easily by themselves with some simple hand gestures.