NumericArray – Compact Representation of “Numeric” Arrays

Piotr Wendykier, Wolfram Research

Join the Conversation #WolframTechConf

ABSTRACT

Coming in version 12, NumericArray is the most compact representations of numeric arrays of various types. In this talk, I will introduce NumericArray object, describe its features and compare it to other types of arrays. I will also demonstrate how NumericArray can be used with LibraryLink and discuss future plans for the integration of this object with the Wolfram Language functions.

Outline

Why do we need yet another array object?

Construction and conversion of NumericArrays.

NumericArrays in LibraryLink.

Integration of NumericArrays with existing Wolfram Language functions.

Why do we need yet another array object?

SparseArray and StructuredArray

Store structured and unstructured sparse data.

sa=SparseArray[Table[{2^i,3^i+i}->1,{i,8}]]

ByteCount[sa]

list=Normal[sa];

ByteCount[list]

ByteArray

Store raw bytes (integers from 0 to 255).

data= RandomInteger[{0,255}, 10^6];
ByteCount[data]

ba=ByteArray[data]

ByteCount[ba]

Packed array

Efficient container for dense arrays of numeric values.

Supported types are:

64-bit signed integer (Integer)

64-bit float (Real)

128-bit complex (Complex)

pa=RandomInteger[10,10^6];

ByteCount[pa]

Developer`PackedArrayQ[pa]

list=Developer`FromPackedArray[pa];

ByteCount[list]

What is missing?

Container for all C++ fixed width integer types

C++ fixed width integer types (int8_t, uint8_t, etc.)

Store raw data from various file formats: HDF5, audio, image, video files.

Container for single precision floating point numbers

Float and complex<float>.

Faster and more memory efficient numerical computations.

Container for half-precision floating point format (binary16 in IEEE 754-2008)

Half-precision floating point format.

GPU computations (CUDA, Neural Networks).

Construction and conversion of NumericArray

?NumericArray

NumericArray[array, type]

12 types are supported in version 12.0 (some names may change):

“Integer8”

“UnsignedInteger8”

“Integer16”

“UnsignedInteger16”

“Integer32”

“UnsignedInteger32”

“Integer64”

“UnsignedInteger64”

“Real32”

“Real64”

“Complex64”

“Complex128”

Create a NumericArray of a specified type:

NumericArray[{1,2,3,4,5}, "Real32"]

Convert a NumericArray to a different type:

NumericArray[NumericArray[], "Integer32"]

NumericArray[array]

Create the most compact integer NumericArray:

NumericArray[{1,2,3,4,5}]

NumericArrays require less memory than packed arrays:

pa=RandomInteger[10,10^6];

ByteCount[pa]

na=NumericArray[pa]

ByteCount[na]

Convert SparseArray to NumericArray:

NumericArray[SparseArray[]]

Convert ByteArray to NumericArray:

NumericArray[ByteArray[]]

ByteArray and  NumericArray use the same internal data structure, so it is possible to convert between these objects by replacing the head of an expression. A memory is not copied when the input expression has a reference count equal to 1:

NumericArray[ByteArray[{1,2,3}]]

The same is true for conversion from NumericArray to ByteArray:

ByteArray[NumericArray[{1,2,3}]]

NumericArray[array, type, method]

The coercion method specifies how a NumericArray is constructed. The default “ClipAndRound” method clips to the specified range and rounds reals to integers:

NumericArray[{-1,2.2,3}, "UnsignedInteger8","ClipAndRound"]//InputForm

The “Clip” method clips to the specified range:

NumericArray[{-1,2,3}, "UnsignedInteger8","Clip"]//InputForm

NumericArray[{-1,2.2,3}, "UnsignedInteger8","Clip"]//InputForm

The “Round” method rounds reals to integers:

NumericArray[{0,2.2,3}, "UnsignedInteger8","Round"]//InputForm

NumericArray[{-1,2.2,3}, "UnsignedInteger8","Round"]//InputForm

The “Check” method checks if all specified values are compatible with the type:

NumericArray[{0,2.0,3}, "UnsignedInteger8","Check"]//InputForm

NumericArray[{0,2.2,3}, "UnsignedInteger8","Check"]//InputForm

NumericArray in LibraryLink

A NumericArray  and ByteArray can be passed to or from a LibraryFunction where it appears as an argument or result of type MNumericArray. MNumericArray is a pointer to a data structure that is passed by reference.

Type specification

Arguments or results of type MNumericArray are specified using LibraryDataType

Grid[{{LibraryDataType[NumericArray],"numeric array of any type"},{LibraryDataType[ByteArray],"byte array"}},Alignment->{Left,Left},Spacings->{4,Automatic},Background->LightGray,ItemStyle->{Automatic,Automatic,{{{1,2},{1,1}}->Directive[FontWeight->Bold]}}]

Input argument memory management

The memory management for MNumericArray is very similar to memory management for MTensor.

Grid[{{LibraryDataType[NumericArray], "pass in a copy of the MNumericArray and automatically clean"},
{{LibraryDataType[NumericArray],Automatic},"pass in a copy of the MNumericArray and automatically clean"},
{{LibraryDataType[NumericArray],"Constant"}, "pass in a reference to the MNumericArray that should not be modified"},
{{LibraryDataType[NumericArray],"Manual"},"pass in a copy of the MNumericArray and do not automatically clean"},
{{LibraryDataType[NumericArray],"Shared"},"pass in a reference to the MNumericArray shared between a library and the Wolfram Language"}},Alignment->{Left,Left},Spacings->{4,Automatic},Background->LightGray,ItemStyle->{Automatic,Automatic,{{{1,5},{1,1}}->Directive[FontWeight->Bold]}}]

The same options can be used with ByteArray object.

Output argument memory management

When you return an MNumericArray from a library function, you can also control how its memory is managed.

Grid[{{LibraryDataType[NumericArray], "return a reference to the MNumericArray as a NumericArray in the Wolfram Language"},
{{LibraryDataType[NumericArray],Automatic},"return a reference to the MNumericArray as a NumericArray in the Wolfram Language"},
{{LibraryDataType[NumericArray],"Shared"}, "return a reference shared between the library and the Wolfram Language to the MNumericArray as a NumericArray in the Wolfram Language"}},Alignment->{Left,Left},Spacings->{4,Automatic},Background->LightGray,ItemStyle->{Automatic,Automatic,{{{1,3},{1,1}}->Directive[FontWeight->Bold]}}]

The same options can be used with ByteArray object.

Available functions

FilePrint[FileNameJoin[{$InstallationDirectory , "SystemFiles", "IncludeFiles", "C","WolframNumericArrayLibrary.h"}]]

NumericArray demo

sourceNumericArray = "
#include <cstdint>
#include <complex>
#include <cstring>
#include <fstream>
#include <vector>

#include \"WolframLibrary.h\"
#include \"WolframNumericArrayLibrary.h\"


/* Return the version of Library Link */
DLLEXPORT mint WolframLibrary_getVersion() { return WolframLibraryVersion; }

/* Initialize Library */
DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) {
	return LIBRARY_NO_ERROR;
}

/* Uninitialize Library */
DLLEXPORT void WolframLibrary_uninitialize(WolframLibraryData libData) {
	return;
}

template <typename T>
static void tplNumericArrayReverse(void *out0, const void *in0, mint length) {
	T *out = static_cast<T *>(out0);
	const T *in = static_cast<const T *>(in0);

	for (mint i = 0; i < length; i++) {
		out[length - i - 1] = in[i];
	}
}

/* Reverses elements in a one-dimensional NumericArray */
EXTERN_C DLLEXPORT int numericArrayReverse(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument res) {
	mint length;
	MNumericArray na_in = NULL, na_out = NULL;
	void *data_in = NULL, *data_out = NULL;
	int err = LIBRARY_FUNCTION_ERROR;
	int rank = 0;
	numericarray_data_t type = MNumericArray_Type_Undef;
	WolframNumericArrayLibrary_Functions naFuns = libData->numericarrayLibraryFunctions;

	na_in = MArgument_getMNumericArray(Args[0]);

	err = naFuns->MNumericArray_clone(na_in, &na_out);
	if (err) {
		return err;
	}
	rank = naFuns->MNumericArray_getRank(na_in);
    if(rank != 1) {
		return err;
	}	
	type = naFuns->MNumericArray_getType(na_in);
	length = naFuns->MNumericArray_getFlattenedLength(na_in);
	data_in = naFuns->MNumericArray_getData(na_in);
	data_out = naFuns->MNumericArray_getData(na_out);
	if (data_in == NULL || data_out == NULL) {
		goto cleanup;
	}

	switch (type) {
	case MNumericArray_Type_Bit8:
		tplNumericArrayReverse<std::int8_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_UBit8:
		tplNumericArrayReverse<std::uint8_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Bit16:
		tplNumericArrayReverse<std::int16_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_UBit16:
		tplNumericArrayReverse<std::uint16_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Bit32:
		tplNumericArrayReverse<std::int32_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_UBit32:
		tplNumericArrayReverse<std::uint32_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Bit64:
		tplNumericArrayReverse<std::int64_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_UBit64:
		tplNumericArrayReverse<std::uint64_t>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Real32:
		tplNumericArrayReverse<float>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Real64:
		tplNumericArrayReverse<double>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Complex_Real32:
		tplNumericArrayReverse<std::complex<float>>(data_out, data_in, length);
		break;
	case MNumericArray_Type_Complex_Real64:
		tplNumericArrayReverse<std::complex<double>>(data_out, data_in, length);
		break;
	default:
		goto cleanup;
	}

	MArgument_setMNumericArray(res, na_out);
	return LIBRARY_NO_ERROR;

cleanup:
	naFuns->MNumericArray_free(na_out);
	return err;
}";

Needs["CCompilerDriver`"]

numericarrayLib=CreateLibrary[sourceNumericArray,"numericarrayLib", "Language"->"C++", "CompileOptions"->If[$OperatingSystem=!="Windows","-std=c++11",""]];

numericArrayReverse=LibraryFunctionLoad[numericarrayLib,"numericArrayReverse",{{LibraryDataType[NumericArray],"Constant"}},{LibraryDataType[NumericArray]}]

na=NumericArray[Range[5]]

numericArrayReverse[na]//InputForm

ByteArray demo

sourceByteArray = "/* Include required headers */
#include <cstdint>
#include <complex>
#include <cstring>
#include <fstream>
#include <vector>

#include \"WolframLibrary.h\"
#include \"WolframNumericArrayLibrary.h\"


/* Return the version of Library Link */
DLLEXPORT mint WolframLibrary_getVersion() { return WolframLibraryVersion; }

/* Initialize Library */
DLLEXPORT int WolframLibrary_initialize(WolframLibraryData libData) {
	return LIBRARY_NO_ERROR;
}

/* Uninitialize Library */
DLLEXPORT void WolframLibrary_uninitialize(WolframLibraryData libData) {
	return;
}

static mint readBytes(char const* filename, char** bytes) {
	std::ifstream ifs(filename, std::ios::binary | std::ios::ate);
	if (ifs) {
		std::ifstream::pos_type pos = ifs.tellg();
		*bytes = new char[pos];
		ifs.seekg(0, std::ios::beg);
		ifs.read(*bytes, pos);
		return pos;
	}
	return 0;
}

/* Reads data from a file and returns it as a ByteArray */
EXTERN_C DLLEXPORT int readBytesFromFile(WolframLibraryData libData, mint Argc, MArgument *Args, MArgument res) {
	mint length = 0;
	char *filename = NULL;
	MNumericArray barray = NULL;
	char *barray_data = NULL;
	char *bytes = NULL;
	int err = LIBRARY_FUNCTION_ERROR;
	WolframNumericArrayLibrary_Functions naFuns = libData->numericarrayLibraryFunctions;

	filename = MArgument_getUTF8String(Args[0]);
	length = readBytes(filename, &bytes);
	if (length > 0) {
		err = naFuns->MNumericArray_new(MNumericArray_Type_Bit8, 1, &length, &barray);
		if (err != 0) {
			goto cleanup;
		}
		barray_data = static_cast<char*>(naFuns->MNumericArray_getData(barray));
		std::memcpy(barray_data, bytes, length);
	}
	else {
		goto cleanup;
	}

cleanup:
	if (err == LIBRARY_NO_ERROR) {
		MArgument_setMNumericArray(res, barray);
	}
	delete[] bytes;
	libData->UTF8String_disown(filename);
	return err;
}";

Needs["CCompilerDriver`"]

bytearrayLib=CreateLibrary[sourceByteArray,"bytearrayLib", "Language"->"C++"];

readBytesFromFile=LibraryFunctionLoad[bytearrayLib,"readBytesFromFile",{"UTF8String"},{LibraryDataType[ByteArray]}];

ba=readBytesFromFile[FindFile["ExampleData/rose.gif"]]

ImportByteArray[ba]

Integration of NumericArray with existing Wolfram Language functions

Version 12.0

Functions that accept NumericArrays

NumericArray is treated as a raw object by functions like AtomQ and for purposes of pattern matching.

AtomQ[NumericArray[]]

ArrayQ[NumericArray[]]

Normal[NumericArray[...]] returns the list of values in the numeric array.

arr=Normal[NumericArray[]]

Developer`PackedArrayQ[arr]

arr=Normal[NumericArray[{2^64-1}]]

Developer`PackedArrayQ[arr]

Functions such as Length, Equal, First and Min work with NumericArray objects.

First[NumericArray[]]

Most[NumericArray[]]

Functions such as Part, Take and Drop can be used with NumericArray objects.

na=NumericArray[];

na[[2;;3]]

Reassigning a part does not work in version 12.0:

na[[2;;3]]={4,5}

Import/Export

FITS

Import["ExampleData/sample.fits", {"RawData", 1}]

Export["out.fits",NumericArray[Range[5]]]

HDF5

Import["ExampleData/structured.h5", "RawData"]

Export["out.h5",NumericArray[Range[5]]]

Neural Networks

NetDecoder will accept NumericArrays.

NetEncoder will return NumericArrays.

You will see up to a 2x speedup for some kinds of NetEncoders.

New compiler

The new compiler runtime in version 12.0 already supports all additional data types needed for NumericArray.

Future versions (12.x)

Add half-precision floating-point format (binary16 in IEEE 754-2008).

Arithmetic and statistic operations on NumericArrays (Plus, Sin, etc.).

Functional programming constructs (Map, Scan, etc.).

Import / Export.

Future versions (13.x)

All Wolfram Language functions that operate on lists.