CHIP-AWS v1.0.0 2025 archive

The Chalmers Hydrometeor Inversion Product from the Arctic Weather Satellite (CHIP-AWS) provides
estimates of ice hydrometeor masses in the atmosphere across the globe within the +/-60° latitude
region. This dataset represents a level 2 processing of AWS level 1B instrument data.

This archival dataset contains retrievals for all available 2025 level 1B data.
For more recent retrievals see: <https://clouds-and-precip.group/datasets/chip-aws/>.

The dataset contains retrievals as NetCDF files for each orbit with the following quantities:

NAME                        UNIT        DESCRIPTION
Frozen Water Path (FWP)     kg/m^2      Mass of ice hydrometeors in a vertical column through the
                                        atmosphere.
D_m                         m           Mass-weighted mean volume equivalent diameter of ice
                                        hydrometeors inside the column.
Z_m                         m           Mass-weighted mean altitude of ice hydrometeors inside the
                                        column.

The uncertainty is quantified by presenting the quantiles of the retrieved CDF for each variable.

The dimensions and variables in the NetCDF files are illustrated with the example output from 'ncdump -h'
in the documentation file ncdump.txt

=== Data files ===

The folder structure and the number of nc files in each tar file are as follows:

chip-aws-v1.0.0
└── level2
├── 2025
│   ├── 01.tar 436 files
│   ├── 02.tar 301 files
│   ├── 03.tar 407 files
│   ├── 04.tar 416 files
│   ├── 05.tar 336 files
│   ├── 06.tar 351 files
│   ├── 07.tar 440 files
│   ├── 08.tar 458 files
│   ├── 09.tar 427 files
│   ├── 10.tar 443 files
│   ├── 11.tar 434 files
│   └── 12.tar 450 files
└── index.json

=== Usage ===
Following usage example loads the dataset with python and xarray.

```python
import xarray as xr

ds = xr.open_mfdataset(
    "level2/2025/**/*.nc",
    combine="nested",
    concat_dim="scan",
    data_vars="minimal",
    coords="minimal",
    compat="override",
)

# Each file is generated from a single source level 1 file for self-contained processing.
# Consecutive source files have short overlaps. To remove duplicate data,
# we remove overlapping scans by keeping only the last occurrence of each duplicate.
# Sorting ensures scans are in chronological order, allowing selection by time slices.
ds = ds.sel(scan=~ds.get_index('scan').duplicated(keep='last'))
ds = ds.sortby('scan')

print(ds)
```

=== Virtual dataset ===
When performing large analysis, the overhead from parsing file metadata from all netcdf files can become significant.
For such cases, consider using the provided `virtual.icechunk` dataset that provides instant access to the dataset metadata and references the individual netcdf files. 

Usage:
```python
import icechunk  # Tested on icechunk v2.3.0
import xarray as xr

config = icechunk.RepositoryConfig.default()
config.set_virtual_chunk_container(
    icechunk.VirtualChunkContainer(
        url_prefix='file:///mylocalpath/chip-aws-v1.0.0/level2/',
        store=icechunk.local_filesystem_store('/mylocalpath/chip-aws-v1.0.0/level2/'),
        name="level2"
    ),
)
repo = icechunk.Repository.open(
    storage=icechunk.local_filesystem_storage('/mylocalpath/chip-aws-v1.0.0/level2/virtual.icechunk'),
    authorize_virtual_chunk_access={"file:///mylocalpath/chip-aws-v1.0.0/level2/": icechunk.Credentials.LocalFileSystemAccess()},
    config=config
)
session = repo.readonly_session("main")
ds = xr.open_zarr(session.store, zarr_format=3, consolidated=False)
ds = ds.sel(scan=~ds.flag_overlap.values)  # filter out non-overlapping scans