kkpy.io¶

Functions to read and write files

`kkpy.io.get_fname`(indir, pattern, dt[, …])	Get filename corresponding to the given datetime(s) and format.
`kkpy.io.read_aws`(time[, date_range, …])	Read AWS (AWS_MIN).
`kkpy.io.read_2dvd_rho`(time[, date_range, …])	Read 2DVD density files into dataframe.
`kkpy.io.read_mxpol_rhi_with_hc`(rhifile_nc, …)	Read MXPOL RHI with hydrometeor classification into py-ART radar object.
`kkpy.io.read_dem`([file, area])	Read NASA SRTM 3-arcsec (90 meters) digital elevation model in South Korea.
`kkpy.io.read_wissdom`(fnames[, kind, degree])	Read WISSDOM wind field.
`kkpy.io.read_vertix`(fnames)	Read VertiX (nc).
`kkpy.io.read_vet`(fnames)	Read WRC VET motion vector.
`kkpy.io.read_hsr`(fnames)	Read WRC HSR reflectivity.
`kkpy.io.read_d3d`(fnames)	Read WRC D3D product.
`kkpy.io.read_r3d`(fnames[, kind])	Read WRC R3D product.
`kkpy.io.read_sounding`(fnames)	Read sounding.
`kkpy.io.read_lidar_wind`(fnames, ftimes[, dropna])	Read lidar wind profile.
`kkpy.io.read_wpr_kma`(fnames, ftimes[, dropna])	Read KMA wind profiler.
`kkpy.io.read_pluvio_raw`(fnames[, dropna])	Read pluvio raw data.
`kkpy.io.read_2dvd`(fnames[, verbose])	Read 2DVD ascii file.
`kkpy.io.read_wxt520`(fnames[, verbose])	Read WXT520 ascii file.

kkpy.io.read_aws(time, date_range=True, datadir='/disk/STORAGE/OBS/AWS/', stnid=None, dask=True)[source]¶

Read AWS (AWS_MIN).

Examples

>>> import datetime
>>> df_aws = kkpy.io.read_aws(time=datetime.datetime(2018,2,28,6,0))

>>> df_aws = kkpy.io.read_aws(time=[datetime.datetime(2018,2,28,6,0),datetime.datetime(2018,3,1,12,0)], datadir='/path/to/aws/files/')

Parameters

time (datetime or array_like of datetime) – Datetime of the data you want to read. If this is array of two elements, it will read all data within two datetimes by default. If this is array of elements and keyword date_range is False, it will read the data of specific time of each element.
date_range (bool, optional) – False if argument time contains element of specific time you want to read.
datadir (str, optional) – Directory of data.
stnid (list, optional) – List of station id you want to read. Read all site if None.
dask (boolean, optional) – Return a dask dataframe if True, otherwise return a pandas dataframe.

Returns

df_aws – Return dataframe of aws data.

Return type

dataframe

Warning

This routine will be updated to receive argument for only filenames. The syntax will be likely changed in the near future.

kkpy.io.read_2dvd_rho(time, date_range=True, datadir='/disk/common/kwonil_rainy/RHO_2DVD/', filename='2DVD_Dapp_v_rho_201*Deq.txt')[source]¶

Read 2DVD density files into dataframe.

Examples

>>> import datetime
>>> df_2dvd_drop = kkpy.io.read_2dvd_rho(time=datetime.datetime(2018,2,28)) # automatically date_range=False

>>> df_2dvd_drop = kkpy.io.read_2dvd_rho(time=[datetime.datetime(2018,2,28,6),datetime.datetime(2018,3,1,12)], datadir='/path/to/2dvd/files/')

>>> df_2dvd_drop = kkpy.io.read_2dvd_rho(time=list_of_many_datetimes, date_range=False)

>>> df_2dvd_drop = kkpy.io.read_2dvd_rho(time=datetime.datetime(2018,2,28), filename='2DVD_rho_test_*.txt')

Parameters

time (datetime or array_like of datetime) – Datetime of the data you want to read. If this is array of two elements, it will read all data within two datetimes by default. If this is array of elements and keyword date_range is False, it will read the data of specific time of each element.
date_range (bool, optional) – False if argument time contains element of specific time you want to read.
datadir (str, optional) – Directory of data.
filename (str, optional) – File naming of data.

Returns

df_2dvd_drop – Return dataframe of 2dvd data.

Return type

dataframe

kkpy.io.read_mxpol_rhi_with_hc(rhifile_nc, hcfile_mat)[source]¶

Read MXPOL RHI with hydrometeor classification into py-ART radar object.

Examples

>>> rhifile = '/disk/WORKSPACE/kwonil/MXPOL/RAW/2018/02/28/MXPol-polar-20180228-065130-RHI-225_8.nc'
>>> hidfile = '/disk/WORKSPACE/kwonil/MXPOL/HID/2018/02/28/MXPol-polar-20180228-065130-RHI-225_8_zdrcorr_demix.mat'
>>> radar_mxp = kkpy.io.read_mxpol_rhi_with_hc(rhifile, hcfile)

Parameters

rhifile_nc (str or array_like of str) – Filepath of RHI data to read. The number and the order of elements should match with hcfile_mat.
hcfile_mat (str or array_like of str) – Filepath of hydrometeor classification file to read. The number and the order of elements should match with rhifile_nc.

Returns

radar – Return py-ART radar object.

Return type

py-ART radar object

kkpy.io.read_dem(file=None, area='pyeongchang')[source]¶

Read NASA SRTM 3-arcsec (90 meters) digital elevation model in South Korea.

Examples

>>> dem, lon_dem, lat_dem, proj_dem = kkpy.io.read_dem(area='pyeongchang')
>>> ax = plt.subplot(projection=ccrs.PlateCarree())
>>> pm = ax.pcolormesh(lon_dem, lat_dem, dem, cmap=cmap, vmin=0, transform=ccrs.PlateCarree())

>>> dem, lon_dem, lat_dem, proj_dem = kkpy.io.read_dem(area='korea')

>>> dem, lon_dem, lat_dem, proj_dem = kkpy.io.read_dem(file='./pyeongchang_90m.tif')

Parameters

file (str, optional) – Filepath of .tif DEM file to read.
area (str, optional) – Region of interest. Possible options are ‘pyeongchang’ and ‘korea’. Default is ‘pyeongchang’.

Returns

dem (float 2D array) – Return DEM elevation.
lon_dem (float 2D array) – Return longitude of each DEM pixel.
lat_dem (float 2D array) – Return latitude of each DEM pixel.
proj_dem (osr object) – Spatial reference system of the used coordinates.

kkpy.io.get_fname(indir, pattern, dt, date_range=True, verbose=True)[source]¶

Get filename corresponding to the given datetime(s) and format.

Examples

>>> # Get radar filename
>>> fname, fdatetime = get_fname('/disk/STORAGE/OBS/Radar/ICE-POP/PPI/NOQC/KST/',
>>>                              '%Y%m/%d/RDR_GNG_%Y%m%d%H%M.uf',
>>>                              [datetime.datetime(2018,2,28,15,0), datetime.datetime(2018,3,2,16,0)])

>>> # Get AWS filename (no extension)
>>> fname, fdatetime = get_fname('/disk/STORAGE/OBS/AWS/',
>>>                              '%Y%m/%d/AWS_MIN_%Y%m%d%H%M',
>>>                              [datetime.datetime(2018,1,22,5,30), datetime.datetime(2018,1,23,4,28)])

>>> # Get MRR filename (duplicate format - %m)
>>> fname, fdatetime = get_fname('/disk/STORAGE/OBS/MRR/AveData/',
>>>                               '%Y%m/%m%d.ave',
>>>                               [datetime.datetime(2015,8,15,5,30), datetime.datetime(2015,8,17,4,28)])

>>> # Get 2DVD filename (one datetime, the use of DOY - %j)
>>> fname, fdatetime = get_fname('/disk/STORAGE/OBS/2DVD/2dvddata/hyd/',
>>>                              'V%y%j_1.txt',
>>>                              datetime.datetime(2012,2,3))

>>> # Get MRR-PRO filename (multiple datetimes with pandas)
>>> import pandas as pd
>>> fname, fdatetime = get_fname('/disk/STORAGE/OBS/MRR-PRO/',
>>>                              '%Y%m/%Y%m%d/%Y%m%d_%H%M%S.nc',
>>>                              pd.date_range(start='2020-06-01', end='2020-08-31', freq='1D'),
>>>                              date_range=False,
>>>                              verbose=True)

Parameters

indir (str) – Path of the root directory. This should not have any format string.
pattern (str) – Datetime pattern to match. The directory can be formatted here (eg. %Y%m/%d/sitename/data_%Y%m%d%H%M%S.csv). See format code description: https://docs.python.org/3/library/datetime.html#strftime-and-strptime-format-codes.
dt (datetime or list of datetime) – Datetime of interest to match. If a datetime object, find one matched file. If list of datetime objects, find matched files for specific datetimes if `date_range` is False. If list of two datetime objects and `date_range` is True, find all matched files within two datetimes.
date_range (boolean, optional) – True if find all matched files within two datetimes. The number of dt should be two if date_range is True. Return matched files for specific datetimes of dt if False. Default is True.
verbose (boolean, optional) – If True, print warnings ‘File does not exist’ if `date_range` is False.

Returns

fname (str or list of str) – Return filename of matched files.
fdt (datetime or list of datetime) – Return datetime of matched files.

kkpy.io.read_wissdom(fnames, kind='KNUv2', degree='essential')[source]¶

Read WISSDOM wind field.

Examples

>>> ds_wissdom = kkpy.io.read_wissdom('WISSDOM_VAR_201802280600.nc')

>>> ds_wissdom = kkpy.io.read_wissdom('RDR_R3D_KMA_WD_201802280600.bin.gz', kind='KMAbin')

>>> ds_wissdom = kkpy.io.read_wissdom('RDR_R3D_KMA_WD_201802280600.nc', kind='KMAnc', degree='extensive')

Parameters

fnames (str or array_like) – Filename(s) of WISSDOM to read.
kind (str, optional) – Data format. Possible options are ‘KNUv2’, ‘KMAnc’, and ‘KMAbin’. Default is ‘KNUv2’.
degree (str, optional) – Degree of variable type to read. Possible options are ‘essential’, ‘extensive’, and ‘debug’. Default is ‘essential’. ‘essential’ includes u, v, and w, while ‘extensive’ further includes divergence and vorticity. ‘debug’ returns all available variables. Note that the time efficiency for ‘extensive’ is low when kind=’KMAbin’.

Returns

ds – Return WISSDOM wind field.

Return type

xarray dataset object

kkpy.io.read_vertix(fnames)[source]¶

Read VertiX (nc).

Examples

>>> ds_vtx = kkpy.io.read_vertix(fnames)

Parameters: fnames (str or array_like) – Filename(s) of VertiX to read.
Returns: ds – Return VertiX dataset.
Return type: xarray dataset object

kkpy.io.read_vet(fnames)[source]¶

Read WRC VET motion vector.

Examples

>>> ds_vet = kkpy.io.read_vet(fnames)

Parameters: fnames (str or array_like) – Filename(s) of VET motion vector to read.
Returns: ds – Return WRC VET motion vector dataset.
Return type: xarray dataset object

kkpy.io.read_hsr(fnames)[source]¶

Read WRC HSR reflectivity.

Examples

>>> ds_hsr = kkpy.io.read_hsr(fnames)

Parameters: fnames (str or array_like) – Filename(s) of HSR reflectivity to read.
Returns: ds – Return WRC HSR reflectivity dataset.
Return type: xarray dataset object

kkpy.io.read_d3d(fnames)[source]¶

Read WRC D3D product.

Examples

>>> ds_d3d = kkpy.io.read_d3d(fnames)

Parameters: fnames (str or array_like) – Filename(s) of D3D product to read. Variables in all files must be of the same type.
Returns: ds – Return WRC D3D product dataset.
Return type: xarray dataset object

kkpy.io.read_r3d(fnames, kind='nc')[source]¶

Read WRC R3D product.

Examples

>>> ds_r3d = kkpy.io.read_r3d(fnames)

Parameters

fnames (str or array_like) – Filename(s) of R3D product to read. Variables in all files must be of the same type.
kind (str, optional) – Data format. Possible options are ‘nc’ and ‘bin’. Default is ‘nc’.
--------- –
ds_r3d = kkpy.io.read_r3d('RDR_R3D_EXT_KD_202301011740.nc') (>>>) –
ds_r3d = kkpy.io.read_r3d('RDR_R3D_EXT_HCI_202007010950.bin.gz' (>>>) –
kind='bin') –
ds_r3d = kkpy.io.read_r3d(['RDR_R3D_EXT_CZ_202007010740.bin.gz' (>>>) –
'RDR_R3D_EXT_CZ_202007010745.bin.gz'] –
kind='bin') –
fnames – Filename(s) of WISSDOM to read.

Returns

ds – Return WRC R3D product dataset.

Return type

xarray dataset object

kkpy.io.read_sounding(fnames)[source]¶

Read sounding.

Examples

>>> df = kkpy.io.read_sounding(fnames)

Parameters: fnames (str or array_like) – Filename(s) of sounding to read.
Returns: df – Return sounding dataframe.
Return type: pandas dataframe object

Notes

Columns:

‘P’: pressure [hPa]
‘T’: temperature [degC]
‘RH’: relative humidity [%]
‘WS’: wind speed [m s-1]
‘WD’: wind direction [deg]
‘Lon’: longitude [oE]
‘Lat’: latitude [oN]
‘Alt’: altitude from mean sea level [m]
‘Geo’: geopotential height [gpm]
‘Dew’: dew-point temperature [degC]
‘U’: u-wind [m s-1]
‘V’: v-wind [m s-1]
‘Uknot’: u-wind [knot]
‘Vknot’: v-wind [knot]
‘time’: launch time [file-dependent, generally UTC]

kkpy.io.read_lidar_wind(fnames, ftimes, dropna=True)[source]¶

Read lidar wind profile.

Examples

>>> df = kkpy.io.read_lidar_wind(fnames)

Parameters

fnames (str or array_like) – Filename(s) of lidar wind profile to read.
ftimes (str or array_like) – Datetime(s) of lidar wind profile to read (see: kkpy.io.get_fname).
dropna (boolean (optional)) – True if drop rows containing NaN values.

Returns

df – Return lidar wind profile dataframe.

Return type

pandas dataframe object

Notes

Columns:

‘Alt’: altitude from lidar [m]
‘U’: u-wind [m s-1]
‘V’: v-wind [m s-1]
‘W’: w-wind [m s-1]
‘WS’: wind speed [m s-1]
‘WD’: wind direction [deg]
‘Valid’: Proportion of the number of data available for wind calculation along the azimuth direction [%]
‘Uknot’: u-wind [knot]
‘Vknot’: v-wind [knot]
‘time’: measurement time [file-dependent, generally KST]

kkpy.io.read_wpr_kma(fnames, ftimes, dropna=True)[source]¶

Read KMA wind profiler.

Examples

>>> df = kkpy.io.read_wpr_kma(fnames)

Parameters

fnames (str or array_like) – Filename(s) of KMA wind profiler data to read.
ftimes (str or array_like) – Datetime(s) of KMA wind profiler data to read (see: kkpy.io.get_fname).
dropna (boolean (optional)) – True if drop rows containing NaN values.

Returns

df – Return KMA wind profiler data dataframe.

Return type

pandas dataframe object

Notes

Columns:

‘Alt’: altitude from wind profiler [m]
‘WD’: wind direction [deg]
‘WS’: wind speed [m s-1]
‘U’: u-wind [m s-1]
‘V’: v-wind [m s-1]
‘Uknot’: u-wind [knot]
‘Vknot’: v-wind [knot]
‘time’: measurement time [file-dependent, generally KST]

kkpy.io.read_pluvio_raw(fnames, dropna=True)[source]¶

Read pluvio raw data.

Examples

>>> df = kkpy.io.read_pluvio(fnames)

Parameters

fnames (str or array_like) – Filename(s) of pluvio raw data to read.
dropna (boolean (optional)) – True if drop rows containing NaN values.

Returns

df – Return pluvio raw data dataframe.

Return type

pandas dataframe object

Notes

Columns:

‘PRT’: Intensity RT [mm hr-1]
‘PA’: Accu RT-NRT [mm]
‘PNRT’: Accu NRT [mm]
‘PATNRT’: Accu total NRT [mm]
‘BRT’: Bucket RT [mm]
‘BNRT’: Bucket NRT [mm]
‘T’: Temperature of load cell [degC]
‘time’: measurement time [file-dependent, generally UTC]

See details at OTT manual (pluvio operating instructions). The variable names are based on the document number 70.020.000.B.E 04-0515.

kkpy.io.read_2dvd(fnames, verbose=False)[source]¶

Read 2DVD ascii file.

Examples

>>> df_2dvd = kkpy.io.read_2dvd(fnames)

Parameters

fnames (str or array_like) – Filename(s) of 2DVD to read.
verbose (bool, optional) – True if print reading status

Returns

df – Return 2dvd dataframe.

Return type

pandas dataframe object

kkpy.io.read_wxt520(fnames, verbose=False)[source]¶

Read WXT520 ascii file.

Examples

>>> df_wxt = kkpy.io.read_wxt520(fnames)

Parameters: fnames (str or array_like) – Filename(s) of WXT520 to read.
Returns: df – Return wxt520 dataframe.
Return type: pandas dataframe object