Experiment planning algorithms are a required component of autonomous platforms for scientific discovery. Selecting a suitable optimization algorithm for a novel application is an important yet difficult choice a researcher has to make based on past empirical performance on similar tasks. To facilitate the evaluation of various algorithms on chemistry and materials science optimization tasks, we previously introduced OLYMPUS (Mach. Learn.: Sci. Technol. 2, 035021, 2021), a Python package providing a consistent and easy-to-use interface to numerous optimization algorithms and benchmark datasets. While the original package was limited to continuous parameters and single objectives, in this work we expand OLYMPUS' capabilities to include mixed (continuous, discrete, and categorical) parameter types and multiple objectives. Several experiment planning algorithms already contained in OLYMPUS are extended to handle categorical and discrete parameter types, and five additional planners are implemented (23 in total). We also provide 23 additional benchmark datasets taken from the chemistry and materials science literature (33 in total), covering a wide range of research areas, from chemical reaction optimization to materials manufacturing. Finally, the visualization capabilities of OLYMPUS are enhanced to allow for easy inspection of the results, and the core functionality of the package is embedded in a Streamlit web application for code-free usage. We demonstrate how OLYMPUS enables researchers to rapidly benchmark different optimization strategies and gain insight into their behavior by focusing on two case studies: the optimization of a Suzuki-Miyaura cross-coupling reaction with categorical reaction conditions, and the multi-objective optimization of redox-active materials. The updated OLYMPUS package provides practitioners with a large suite of tools to efficiently benchmark and analyze experiment planning algorithms on mixed-parameter and multi-objective optimization tasks.