Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,22 +35,34 @@ one might choose one installation option over the other.
```
pip install xbitinfo
```
or
We also maintain other dependency sets for different subsets of functionality:
```
pip install "xbitinfo[viz]" # Optional dependencies for visualisation
pip install "xbitinfo[io]" # Optional dependencies for IO
```

For conda
```
conda install -c conda-forge xbitinfo-python
```
the IO and visualisation dependencies are included by default.

### Installation including optional Julia backend
```
conda install -c conda-forge xbitinfo
```
or
```
pip install xbitinfo # julia needs to be installed manually
pip install "xbitinfo[julia]" # julia needs to be installed manually
```

## How to use

To install all dependencies needed to run this example,
```
pip install "xbitinfo[example]"
```
is recommended
```python
import xarray as xr
import xbitinfo as xb
Expand Down
8 changes: 7 additions & 1 deletion docs/ArtificialInformation_Filter.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,13 @@
"\n",
"In simple terms the bitinformation is retrieved by checking how variable a bit pattern is. However, this approach cannot distinguish between actual information content and artifical information content. By studying the distribution of the information content the user can often identify clear cut-offs of real information content and artificial information content.\n",
"\n",
"The following example shows how such a separation of real information and artificial information can look like. To do so, artificial information is artificially added to an example dataset by applying linear quantization. Linear quantization is often applied to climate datasets (e.g. ERA5) and needs to be accounted for in order to retrieve meaningful bitinformation content. An algorithm that aims at detecting this artificial information itself is introduced."
"The following example shows how such a separation of real information and artificial information can look like. To do so, artificial information is artificially added to an example dataset by applying linear quantization. Linear quantization is often applied to climate datasets (e.g. ERA5) and needs to be accounted for in order to retrieve meaningful bitinformation content. An algorithm that aims at detecting this artificial information itself is introduced.\n",
"\n",
"To install all dependencies needed to run this example,\n",
"```\n",
"pip install \"xbitinfo[example]\"\n",
"```\n",
"is recommended "
]
},
{
Expand Down
8 changes: 7 additions & 1 deletion docs/chunking.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,13 @@
"\n",
"By default the number of bits that need to be kept (`keepbits`) to preserve the requested amount of information is determined based on the entire dataset. This approach doesn't always result in the best compression rates as it preserves too many keepbits in regions with anomalously low information density. The following steps show how the `keepbits` can be retrieved and applied on subsets. In this case, subsets are defined as dataset chunks.\n",
"\n",
"This work is a result of the ECMWF Code4Earth 2023. Please have a look at the [presentation of this project](https://youtu.be/IOi4XvECpsQ?si=hwZkppNRa-J2XVZ9) for additional details."
"This work is a result of the ECMWF Code4Earth 2023. Please have a look at the [presentation of this project](https://youtu.be/IOi4XvECpsQ?si=hwZkppNRa-J2XVZ9) for additional details.\n",
"\n",
"To install all dependencies needed to run this example,\n",
"```\n",
"pip install \"xbitinfo[example]\"\n",
"```\n",
"is recommended "
]
},
{
Expand Down
14 changes: 10 additions & 4 deletions docs/quick-start.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,13 @@
"1. Analysis of the bitwise information content of a dataset\n",
"2. Decision on a threshold of real information to preserve (e.g. 99%)\n",
"3. Bitround dataset accordingly (bitrounding)\n",
"4. Apply lossless compression of choice (e.g. zlib, blosc, zstd) and store the dataset"
"4. Apply lossless compression of choice (e.g. zlib, blosc, zstd) and store the dataset\n",
"\n",
"To install all dependencies needed to run this example,\n",
"```\n",
"pip install \"xbitinfo[example]\"\n",
"```\n",
"is recommended "
]
},
{
Expand Down Expand Up @@ -110,7 +116,7 @@
"id": "9a431133",
"metadata": {},
"source": [
"Without installing pooch, you will not be able to download and load the example datasets, which may result in errors or unexpected behavior."
"Without installing pooch, you will not be able to download and load the example datasets, which may result in errors or unexpected behavior. Alternatively, follow the example installation instructions above."
]
},
{
Expand Down Expand Up @@ -323,7 +329,7 @@
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"display_name": "xbitinfo (3.13.5)",
"language": "python",
"name": "python3"
},
Expand All @@ -337,7 +343,7 @@
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.4"
"version": "3.13.5"
},
"toc-autonumbering": true
},
Expand Down
8 changes: 6 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,13 +15,17 @@ dependencies = [
"matplotlib>=3.9.0",
"prefect>=3.3.0",
"zarr",
"julia>=0.6.2",
]

[dependency-groups]
[project.optional-dependencies]
viz = ["matplotlib<3.9.1", "cmcrameri"]
prefect = ["prefect>=3.3.0"]
io = ["netcdf4", "zarr"]
julia = ["julia"]
example = ["xbitinfo[viz,io]","pooch","jupyter>=1.1","s3fs"]

[dependency-groups]
prefect = ["prefect>=3.3.0"]
test = [
"pytest",
"pooch",
Expand Down
Loading