Metadata-Version: 2.4
Name: HARD-lib
Version: 1.0.5
Summary: Harmony Analysis Ready Dataset
Author: Yan Yuan
Author-email: Yan.Yuan@tudelft.nl
Project-URL: Homepage, https://gitlab.tudelft.nl/yyuan6/hard
Classifier: Operating System :: OS Independent
Description-Content-Type: text/markdown
Requires-Dist: numpy
Requires-Dist: scipy
Requires-Dist: matplotlib
Requires-Dist: xarray<=2025.10.1
Requires-Dist: asf_search>=10.1.0
Requires-Dist: cdsapi
Requires-Dist: dask
Requires-Dist: ipykernel
Requires-Dist: cartopy==0.24.1
Requires-Dist: opencv-python-headless
Requires-Dist: xsar[S1]
Requires-Dist: dask==2025.7.0
Requires-Dist: zarr
Requires-Dist: tqdm
Requires-Dist: global-land-mask
Requires-Dist: h5netcdf
Requires-Dist: jupyterlab
Requires-Dist: watermark
Requires-Dist: joblib
Requires-Dist: tomli
Requires-Dist: build
Requires-Dist: twine
Provides-Extra: dev
Requires-Dist: pytest; extra == "dev"
Dynamic: author-email

## Pre-requisites
1. Create an ECMWF account to access the ERA5 CDS API (for setup go to [cds.climate.copernicus.eu](https://cds.climate.copernicus.eu/how-to-api)). This is needed for wind-field information.

2. Create an Alaskan Satellite Fascility (ASF) account so that you may querry their API using `asf_search`. Store your ASK credentials in a `.netrc` file. This will be needed to download data from ASF.

An example `.netrc`:
```bash
machine asf
login YOURUSERNAME
password YOURPASSWORD
```

## Installation 
Create a new environment and activate
```bash
conda create -n ENV_NAME python==3.12

conda activate ENV_NAME
```
Conda install GDAL (not enabled from pip)
```bash
conda install GDAL
```

### For development
Clone HARD repository

```bash
git clone https://gitlab.tudelft.nl/yyuan6/hard.git
```

Navigate to cloned folder and install an editibale version
```bash
cd HARD

pip install -e .
```

### For easy usage
Download and install the (likely outdated) package from pip 

```bash
pip install HARD-lib
```

## Usage
### Python
For Python tutorials refer to the notebook tutorials in `hard/Tutorial/`

### Command Line Interface (CLI)
To run an automated download, processing and saving exercise using a bash script go to either:

- `hard/HARD/CLI/download_process_save_local_example.sh`
- `hard/HARD/CLI/download_process_save_DelftBlue_example.sh`

Please copy the files before editing and do not track them in this git repository.

#### NOTE 
When submitting the bash script on Delft Blue, please make sure a `logs` subdirectory exists in the same location as the `.sh` file, e.g.:

```bash
mkdir -p logs
sbatch /home/.../hard/HARD/CLI/download_process_save_DelftBlue_example.sh
```


<!-- ## To Do
### ERA5
- check if ERA5 file exists before downloading
- give downlaoded era5 files descriptive names relating to aoi (so that they may be reused)
- enable using pre-downlaoded era5 files

### Processing
- return the xsar object instead of arbitrary roughness field
- land masking on radar data 
- add attributes to file for all processing steps and 
- get land mask from xsar variable
- streamline output types so that `.values` and `.item()` can be kept to a minimum
- remove unnecessary variables after preprocessing  -->


## NRCS Re-calibration

### Sentinel-1 WV
The Sentinel-1 WaVe mode calibration experienced significant changes throughout the mission lifetime, e.g. check figure 6/7 on page 72 in [Sentinel-1 annual performance report 2024](https://sentiwiki.copernicus.eu/__attachments/1681272/DI-MPC-APR-0715%20-%20Sentinel-1%20Annual%20Performance%20Report%202024%20-%202.1.pdf?inst-v=1dcdf0ad-f6da-4386-86fa-d8e5c1a6cc36). These calibration changes are **NOT** retroactively applied, such that new and old scenes may experience radiometric offsets purely due to the used calibration scheme. 

To Counteract this, a re-calibration needs to be performed. Here we set out to perfom a re-calibration using the most up-to-date Sentinel-1 calibration and processing files. The re-calition is an optional step, making using of `xsar`'s existing template. Currently `xsar` can already re-calibrate certain Sentinel-1 modes, but not yet WaVe mode (and IW mode anecdotally yields peculiar results). Thus, updates to `xsar` may depreciate `HARD`'s recalibration.

**NOTE**: The re-calibration is relative, and new calibration files may become available for past scenes, necessitating yet another re-calibration. Nonetheless, the follow steps should take care of the most blatant calibration-induced artefacts which primarily occur prior to 12-05-2020.

**NOTE**: **WV-mode NRCS re-calibration is only useful between 13-03-2018 - 12-05-2020**. Prior to 13-03-2018 no valid radiometric normalization was included in the Sentinel-1 processor, which, as you can see here [sar-mpc.eu/processor/ipf](https://sar-mpc.eu/processor/ipf/) was in version 2.82 for TOPS mode, but only fully from 2.90 for WV/SM modes. Beyond 12-05-2020 there is negligeable change after re-calibration as the system introduced then is still being used now (May 2026). For new updates check [sar-mpc.eu](https://sar-mpc.eu/).

#### Step 1.
Download the individual calibration and processing files from Sentinel-1's Mission Performance Center [sar-mpc.eu](https://sar-mpc.eu/). This can be done automatically by running 

```bash
python src/HARD/CLI/download_S1_calibration.py --path_storage /location/containing/the/recalibration/files/
```

#### Step 2.
Configure `xsar`'s config file, which should be located at `~/.xsar/config.yml`, to direct to necessary files and directory such as:

```bash
data_dir: /tmp
auxiliary_dir: /project/fluxsar/Data/Sentinel-1/calibration/Auxiliary/
path_dataframe_aux: /project/fluxsar/Data/Sentinel-1/calibration/Auxiliary/active_aux.csv
```

Where `auxiliary_dir` refers to the parent directory where the following subfolders live:
`/S1A_AUX_CAL/`, `/S1A_AUX_PP1/`,`/S1B_AUX_CAL/`, `/S1B_AUX_PP1/`, `/S1C_AUX_CAL/`, `/S1C_AUX_PP1/`, `/S1D_AUX_CAL/`, `/S1D_AUX_PP1/`,

#### Part 3.
When running `ProcessWV` inside the `download_preprocess_save_WV.py` file, make sure to supply 
```python
ProcessWV(
    ...
    custom_recalibration=True,
    ...
)
```

Re-calibrated processed imagettes will now contain new attributes `aux_cal_new` and `aux_pp1_new` in addition to the already exisiting `aux_cal` and `aux_pp1`, to clarify which fiels were used to perform the re-calibration (and, perhaps, whetehr re-calirbation should be performed again if the "new" files are now themselves outdated)
