from libpysal.weights import Queen, raster
from libpysal import weights
import xarray
import esda
import warnings
pop = xarray.open_rasterio(
'https://geographicdata.science/book/_downloads/5263090bd0bdbd7d1635505ff7d36d04/ghsl_sao_paulo.tif'
).sel(
x=slice(-4436000, -4427000), y=slice(-2875000, -2886000), band=1
)
pop.dtype
dtype('float32')
w = Queen.from_xarray(pop, sparse=False)
w.sparse.dtype
dtype('int8')
w.transform = 'r'
w.sparse.dtype
dtype('float64')
# Fails
lisa = esda.Moran_Local(pop.to_series(), w)
def aligner(y, w, how='y'):
'''
Align `dtype` between `y` and `w`
...
Arguments
---------
y
w
how : str/dtype
[Optional. Default='y'] Alignment policy:
- 'y': use `y.dtype`
- 'w': use `dtype` in `w`
- `dtype`: different `dtype` target
Returns
-------
ya : ndarray/pandas.Series
wa : W
'''
if type(how) is not str:
ya = y.astype(how)
wa = _convert_w_dtype(w, how)
return ya, wa
elif how == 'y':
wa = _convert_w_dtype(w, y.dtype)
return y, wa
elif how == 'w':
ya = y.astype(w.sparse.dtype)
return ya, w
else:
warnings.warn("Please pass 'y', 'w', or an explicit dtype for `how`")
def _convert_w_dtype(w, dtype):
nw = w.sparse.astype(dtype)
nw = weights.WSP(nw, id_order=w.id_order)
nw = weights.WSP2W(nw)
return nw
ya1, wa1 = aligner(pop.to_series(), w, float)
lisa1 = esda.Moran_Local(ya1, wa1)
ya2, wa2 = aligner(pop.to_series(), w, 'w')
lisa2 = esda.Moran_Local(ya2, wa2)
(lisa1.Is != lisa2.Is).sum()
0
ya3, wa3 = aligner(pop.to_series(), w, 'y')
#lisa3 = esda.Moran_Local(ya3, wa3)
The above still fails because, after w
is converted into wa3
, expressed as float32
, when transforming it for row-standardisation (default in Moran_Local
), it is converted into float64
.
wa3.transform
'O'
wa3.sparse.dtype
dtype('float32')
wa3.transform = 'r'
wa3.sparse.dtype
dtype('float64')
Ways forward/thoughts:
aligner
could be embedded in Moran_Local
and all methods that rely on numba
, but this would need to be added after any transformations and on each of the classes that currently rely on accelerated randomisationy.dtype
or float64
) and leave flexibility as an option if folks need itw
is a large matrix, the approach taken in _convert_w_dtype
might not be feasible/desirable. Do we have other ways?aligner
was called within something like Moran_Local
, warnings would need to be raised so the user is aware. In fact, one option in Moran_local
could select what to do, with the option of raising an error if they're different, or converting dtype
s automatically.