Optical activity has been demonstrated by twisting cascading multilayer anisotropic metamaterials with cross-polarization conversions, for which the x-(y-) polarized light is significantly transformed to a y-(x-) polarized one with enhanced transmittance in a unidirectional manner. Here, we design a twisted tri-layer anisotropic metamaterial with the twisted angle θ = 22.5° to present the unidirectional optical activity by linear-to-circular polarization conversions. When the structure is illuminated by x-polarized incident, the x-polarized light is converted to a right-circularly polarization for forward transmission at 175.5 THz. But for backward transmission, the x-polarized light is converted to a left-circularly polarization at 350THz. This phenomenon is attributed to the chiral structural arrangement and anisotropic resonance of nanobars. We verify our design at optical frequencies and show potential application in unidirectional circular polarizer and other nanodevices.