Developer libraries for array computing.

## Motivation

NumPy has been very impactful in the Python ecosystem, providing efficient array representations and computations. It is still heavily used today and many recent Python libraries either use it internally or were influenced by its API.

However, with the explosion of machine learning and data analysis work happening in Python, many new libraries have implemented their own typed arrays and computation layers, because the NumPy model is not flexible enough. It is hard to extend NumPy with new types and get efficient computation on a variety of hardware devices. It can also be difficult to use just the type system of NumPy or just the computational infrastructure of NumPy.

We are building XND to recreate the foundations of NumPy as a number of smaller libraries, combining the lessons learned in the past twenty years of array computing in Python with the needs of newer applications. This is not a replacement of NumPy. Eventually, NumPy could use XND as could Pandas, Dask, and other libraries.

In fact, we are actively working on using XND in Numba and are also very interested in integrating it with a variety of libraries including Dask, xarray, Numba, Chainer, PyTorch, Tensorflow, PyMC4, TVM/NNVM, Plasma Store, Apache Arrow, and Tensor Comprehensions.

Please reach out if you have ideas or want to help, on either of the Gitter channels below:

XND XND integrations
We also have **weekly public XND meetings** that you are welcome to attend:

If you want to understand more about XND and Plures projects, we invite you to check out this introductory notebook:

Introduction Notebook (Git repo) XND Notebooks (Binder)## Packages

## gumath is a library for function dispatch.

**
libgumath
** is a C library supporting a general dispatch mechanism for general memory containers as well as a
specific implementation for NumPy-like containers of a composable, generalized function concept.

**
gumath
** is a Python wrapper for

`libgumath`

.```
$ conda install -c xnd/label/dev gumath
# or
$ python3 -m pip install gumath
```

```
>>> from gumath import functions as fn
>>> from xnd import xnd
>>> fn.sin(xnd([[1, 2], [3, 4]], type='2 * 2 * float64'))
xnd([[0.8414709848078965, 0.9092974268256817], [0.1411200080598672, -0.7568024953079282]], type='2 * 2 * float64'))
```

## xnd is a library for typed memory blocks.

**
libxnd
** implements support for typed memory blocks using the

`libndtypes`

type library.The
**
xnd
** module implements a container type that maps most Python values relevant for scientific computing
directly to typed memory.

```
>>> from xnd import xnd
>>> xnd([{'b': [10.2, 232.3]}, {'b': [0.2, 0.23]}])
xnd([{'b': [10.2, 232.3]}, {'b': [0.2, 0.23]}], type='2 * {b : 2 * float64}')
```

```
>>> from xnd import xnd
>>> import numpy as np
>>> xnd.from_buffer(np.arange(6).reshape(3, 2))
xnd([[0, 1], [2, 3], [4, 5]], type='3 * 2 * int64')
```

## ndtypes is a library for typing memory blocks.

**
libndtypes
** implements the type part of a compiler frontend. It can describe C types needed for array computing
and additionally includes symbolic types for the purposes of dynamic type checking.

The
**
ndtypes
** Python module implements bindings for libndtypes. Its purpose is to support the type part required
for the xnd container module.

```
>>> from ndtypes import ndt
>>> # fixed dimension type
>>> ndt("fixed(shape=10) * uint64")
ndt("10 * uint64")
```

```
>>> from ndtypes import ndt
>>> # variable dimension type
>>> ndt("var * float32")
ndt("var * float32")
```

## xndtools is a tool to help make gumath kernels.

** xndtools**
provides a script that builds gumath kernels from C prototypes of the computational functions.

```
# Generate kernels configuration file (mkl_blas-kernels.cfg)
$ xnd_tools config $CONDA_PREFIX/include/mkl_blas.h
```

```
# Modify mkl_blas-kernels.cfg
[MODULE mkl_blas]
typemaps:
MKL_INT: int64
includes:
mkl_blas.h
kinds: Xnd, C
ellipses: none, ...
[KERNEL dot]
prototypes:
double ddot(MKL_INT *n, double *x, MKL_INT *incx, double *y, MKL_INT *incy);
float sdot(MKL_INT *n, float *x, MKL_INT *incx, float *y, MKL_INT *incy);
description: Compute dot product of two arrays
input_arguments: x(n), y(n)
hide_arguments: n = len(x), incx = 1, incy = 1
```

```
# Generate kernels (mkl_blas-kernels.c)
# and Python extension module (mkl_blas-python.c)
$ xnd_tools module mkl_blas-kernels.cfg
```

```
>>> # Build Python extension and use:
>>> import mkl_blas
>>> from xnd import xnd
>>> x = xnd([[1,2], [3,4]], dtype = 'float64')
>>> mkl_blas.dot(x, x)
xnd([5.0, 25.0], type='2 * float64')
```