Report
scida: scalable analysis for scientific big data
| العنوان: | scida: scalable analysis for scientific big data |
|---|---|
| المؤلفون: | Byrohl, Chris, Nelson, Dylan |
| سنة النشر: | 2024 |
| المجموعة: | Astrophysics |
| مصطلحات موضوعية: | Astrophysics - Instrumentation and Methods for Astrophysics, Astrophysics - Cosmology and Nongalactic Astrophysics, Astrophysics - Astrophysics of Galaxies |
| الوصف: | scida is a Python package for reading and analyzing large scientific data sets with support for various cosmological and galaxy formation simulations out-of-the-box. Data access is provided through a hierarchical dictionary-like data structure after a simple load() function. Using the dask library for scalable, parallel and out-of-core computation, all computation requests from a user session are first collected in a task graph. Arbitrary custom analysis, as well as all available dask (array) operations, can be performed. The subsequent computation is executed only upon request, on a target resource (e.g. a HPC cluster). Comment: recommended for acceptance in the Journal of Open Source Software; open-source development at https://github.com/cbyrohl/scida |
| نوع الوثيقة: | Working Paper |
| URL الوصول: | http://arxiv.org/abs/2402.17818 |
| رقم الانضمام: | edsarx.2402.17818 |
| قاعدة البيانات: | arXiv |
| الوصف غير متاح. |