394 lines
18 KiB
ReStructuredText
394 lines
18 KiB
ReStructuredText
==========================
|
|
NumPy 1.11.0 Release Notes
|
|
==========================
|
|
|
|
This release supports Python 2.6 - 2.7 and 3.2 - 3.5 and contains a number
|
|
of enhancements and improvements. Note also the build system changes listed
|
|
below as they may have subtle effects.
|
|
|
|
No Windows (TM) binaries are provided for this release due to a broken
|
|
toolchain. One of the providers of Python packages for Windows (TM) is your
|
|
best bet.
|
|
|
|
|
|
Highlights
|
|
==========
|
|
|
|
Details of these improvements can be found below.
|
|
|
|
* The datetime64 type is now timezone naive.
|
|
* A dtype parameter has been added to ``randint``.
|
|
* Improved detection of two arrays possibly sharing memory.
|
|
* Automatic bin size estimation for ``np.histogram``.
|
|
* Speed optimization of A @ A.T and dot(A, A.T).
|
|
* New function ``np.moveaxis`` for reordering array axes.
|
|
|
|
|
|
Build System Changes
|
|
====================
|
|
|
|
* Numpy now uses ``setuptools`` for its builds instead of plain distutils.
|
|
This fixes usage of ``install_requires='numpy'`` in the ``setup.py`` files of
|
|
projects that depend on Numpy (see gh-6551). It potentially affects the way
|
|
that build/install methods for Numpy itself behave though. Please report any
|
|
unexpected behavior on the Numpy issue tracker.
|
|
* Bento build support and related files have been removed.
|
|
* Single file build support and related files have been removed.
|
|
|
|
|
|
Future Changes
|
|
==============
|
|
|
|
The following changes are scheduled for Numpy 1.12.0.
|
|
|
|
* Support for Python 2.6, 3.2, and 3.3 will be dropped.
|
|
* Relaxed stride checking will become the default. See the 1.8.0 release
|
|
notes for a more extended discussion of what this change implies.
|
|
* The behavior of the datetime64 "not a time" (NaT) value will be changed
|
|
to match that of floating point "not a number" (NaN) values: all
|
|
comparisons involving NaT will return False, except for NaT != NaT which
|
|
will return True.
|
|
* Indexing with floats will raise IndexError,
|
|
e.g., a[0, 0.0].
|
|
* Indexing with non-integer array_like will raise ``IndexError``,
|
|
e.g., ``a['1', '2']``
|
|
* Indexing with multiple ellipsis will raise ``IndexError``,
|
|
e.g., ``a[..., ...]``.
|
|
* Non-integers used as index values will raise ``TypeError``,
|
|
e.g., in ``reshape``, ``take``, and specifying reduce axis.
|
|
|
|
|
|
In a future release the following changes will be made.
|
|
|
|
* The ``rand`` function exposed in ``numpy.testing`` will be removed. That
|
|
function is left over from early Numpy and was implemented using the
|
|
Python random module. The random number generators from ``numpy.random``
|
|
should be used instead.
|
|
* The ``ndarray.view`` method will only allow c_contiguous arrays to be
|
|
viewed using a dtype of different size causing the last dimension to
|
|
change. That differs from the current behavior where arrays that are
|
|
f_contiguous but not c_contiguous can be viewed as a dtype type of
|
|
different size causing the first dimension to change.
|
|
* Slicing a ``MaskedArray`` will return views of both data **and** mask.
|
|
Currently the mask is copy-on-write and changes to the mask in the slice do
|
|
not propagate to the original mask. See the FutureWarnings section below for
|
|
details.
|
|
|
|
|
|
Compatibility notes
|
|
===================
|
|
|
|
datetime64 changes
|
|
------------------
|
|
In prior versions of NumPy the experimental datetime64 type always stored
|
|
times in UTC. By default, creating a datetime64 object from a string or
|
|
printing it would convert from or to local time::
|
|
|
|
# old behavior
|
|
>>> np.datetime64('2000-01-01T00:00:00')
|
|
numpy.datetime64('2000-01-01T00:00:00-0800') # note the timezone offset -08:00
|
|
|
|
|
|
A consensus of datetime64 users agreed that this behavior is undesirable
|
|
and at odds with how datetime64 is usually used (e.g., by `pandas
|
|
<http://pandas.pydata.org>`__). For most use cases, a timezone naive datetime
|
|
type is preferred, similar to the ``datetime.datetime`` type in the Python
|
|
standard library. Accordingly, datetime64 no longer assumes that input is in
|
|
local time, nor does it print local times::
|
|
|
|
>>> np.datetime64('2000-01-01T00:00:00')
|
|
numpy.datetime64('2000-01-01T00:00:00')
|
|
|
|
For backwards compatibility, datetime64 still parses timezone offsets, which
|
|
it handles by converting to UTC. However, the resulting datetime is timezone
|
|
naive::
|
|
|
|
>>> np.datetime64('2000-01-01T00:00:00-08')
|
|
DeprecationWarning: parsing timezone aware datetimes is deprecated;
|
|
this will raise an error in the future
|
|
numpy.datetime64('2000-01-01T08:00:00')
|
|
|
|
As a corollary to this change, we no longer prohibit casting between datetimes
|
|
with date units and datetimes with time units. With timezone naive datetimes,
|
|
the rule for casting from dates to times is no longer ambiguous.
|
|
|
|
``linalg.norm`` return type changes
|
|
-----------------------------------
|
|
The return type of the ``linalg.norm`` function is now floating point without
|
|
exception. Some of the norm types previously returned integers.
|
|
|
|
polynomial fit changes
|
|
----------------------
|
|
The various fit functions in the numpy polynomial package no longer accept
|
|
non-integers for degree specification.
|
|
|
|
*np.dot* now raises ``TypeError`` instead of ``ValueError``
|
|
-----------------------------------------------------------
|
|
This behaviour mimics that of other functions such as ``np.inner``. If the two
|
|
arguments cannot be cast to a common type, it could have raised a ``TypeError``
|
|
or ``ValueError`` depending on their order. Now, ``np.dot`` will now always
|
|
raise a ``TypeError``.
|
|
|
|
FutureWarning to changed behavior
|
|
---------------------------------
|
|
|
|
* In ``np.lib.split`` an empty array in the result always had dimension
|
|
``(0,)`` no matter the dimensions of the array being split. This
|
|
has been changed so that the dimensions will be preserved. A
|
|
``FutureWarning`` for this change has been in place since Numpy 1.9 but,
|
|
due to a bug, sometimes no warning was raised and the dimensions were
|
|
already preserved.
|
|
|
|
``%`` and ``//`` operators
|
|
--------------------------
|
|
These operators are implemented with the ``remainder`` and ``floor_divide``
|
|
functions respectively. Those functions are now based around ``fmod`` and are
|
|
computed together so as to be compatible with each other and with the Python
|
|
versions for float types. The results should be marginally more accurate or
|
|
outright bug fixes compared to the previous results, but they may
|
|
differ significantly in cases where roundoff makes a difference in the integer
|
|
returned by ``floor_divide``. Some corner cases also change, for instance, NaN
|
|
is always returned for both functions when the divisor is zero,
|
|
``divmod(1.0, inf)`` returns ``(0.0, 1.0)`` except on MSVC 2008, and
|
|
``divmod(-1.0, inf)`` returns ``(-1.0, inf)``.
|
|
|
|
C API
|
|
-----
|
|
|
|
Removed the ``check_return`` and ``inner_loop_selector`` members of
|
|
the ``PyUFuncObject`` struct (replacing them with ``reserved`` slots
|
|
to preserve struct layout). These were never used for anything, so
|
|
it's unlikely that any third-party code is using them either, but we
|
|
mention it here for completeness.
|
|
|
|
|
|
object dtype detection for old-style classes
|
|
--------------------------------------------
|
|
|
|
In python 2, objects which are instances of old-style user-defined classes no
|
|
longer automatically count as 'object' type in the dtype-detection handler.
|
|
Instead, as in python 3, they may potentially count as sequences, but only if
|
|
they define both a `__len__` and a `__getitem__` method. This fixes a segfault
|
|
and inconsistency between python 2 and 3.
|
|
|
|
New Features
|
|
============
|
|
|
|
* ``np.histogram`` now provides plugin estimators for automatically
|
|
estimating the optimal number of bins. Passing one of ['auto', 'fd',
|
|
'scott', 'rice', 'sturges'] as the argument to 'bins' results in the
|
|
corresponding estimator being used.
|
|
|
|
* A benchmark suite using `Airspeed Velocity
|
|
<https://asv.readthedocs.io/>`__ has been added, converting the
|
|
previous vbench-based one. You can run the suite locally via ``python
|
|
runtests.py --bench``. For more details, see ``benchmarks/README.rst``.
|
|
|
|
* A new function ``np.shares_memory`` that can check exactly whether two
|
|
arrays have memory overlap is added. ``np.may_share_memory`` also now has
|
|
an option to spend more effort to reduce false positives.
|
|
|
|
* ``SkipTest`` and ``KnownFailureException`` exception classes are exposed
|
|
in the ``numpy.testing`` namespace. Raise them in a test function to mark
|
|
the test to be skipped or mark it as a known failure, respectively.
|
|
|
|
* ``f2py.compile`` has a new ``extension`` keyword parameter that allows the
|
|
fortran extension to be specified for generated temp files. For instance,
|
|
the files can be specifies to be ``*.f90``. The ``verbose`` argument is
|
|
also activated, it was previously ignored.
|
|
|
|
* A ``dtype`` parameter has been added to ``np.random.randint``
|
|
Random ndarrays of the following types can now be generated:
|
|
|
|
- ``np.bool_``,
|
|
- ``np.int8``, ``np.uint8``,
|
|
- ``np.int16``, ``np.uint16``,
|
|
- ``np.int32``, ``np.uint32``,
|
|
- ``np.int64``, ``np.uint64``,
|
|
- ``np.int_ ``, ``np.intp``
|
|
|
|
The specification is by precision rather than by C type. Hence, on some
|
|
platforms ``np.int64`` may be a ``long`` instead of ``long long`` even if
|
|
the specified dtype is ``long long`` because the two may have the same
|
|
precision. The resulting type depends on which C type numpy uses for the
|
|
given precision. The byteorder specification is also ignored, the
|
|
generated arrays are always in native byte order.
|
|
|
|
* A new ``np.moveaxis`` function allows for moving one or more array axes
|
|
to a new position by explicitly providing source and destination axes.
|
|
This function should be easier to use than the current ``rollaxis``
|
|
function as well as providing more functionality.
|
|
|
|
* The ``deg`` parameter of the various ``numpy.polynomial`` fits has been
|
|
extended to accept a list of the degrees of the terms to be included in
|
|
the fit, the coefficients of all other terms being constrained to zero.
|
|
The change is backward compatible, passing a scalar ``deg`` will behave
|
|
as before.
|
|
|
|
* A divmod function for float types modeled after the Python version has
|
|
been added to the npy_math library.
|
|
|
|
|
|
Improvements
|
|
============
|
|
|
|
``np.gradient`` now supports an ``axis`` argument
|
|
-------------------------------------------------
|
|
The ``axis`` parameter was added to ``np.gradient`` for consistency. It
|
|
allows to specify over which axes the gradient is calculated.
|
|
|
|
``np.lexsort`` now supports arrays with object data-type
|
|
--------------------------------------------------------
|
|
The function now internally calls the generic ``npy_amergesort`` when the
|
|
type does not implement a merge-sort kind of ``argsort`` method.
|
|
|
|
``np.ma.core.MaskedArray`` now supports an ``order`` argument
|
|
-------------------------------------------------------------
|
|
When constructing a new ``MaskedArray`` instance, it can be configured with
|
|
an ``order`` argument analogous to the one when calling ``np.ndarray``. The
|
|
addition of this argument allows for the proper processing of an ``order``
|
|
argument in several MaskedArray-related utility functions such as
|
|
``np.ma.core.array`` and ``np.ma.core.asarray``.
|
|
|
|
Memory and speed improvements for masked arrays
|
|
-----------------------------------------------
|
|
Creating a masked array with ``mask=True`` (resp. ``mask=False``) now uses
|
|
``np.ones`` (resp. ``np.zeros``) to create the mask, which is faster and
|
|
avoid a big memory peak. Another optimization was done to avoid a memory
|
|
peak and useless computations when printing a masked array.
|
|
|
|
``ndarray.tofile`` now uses fallocate on linux
|
|
----------------------------------------------
|
|
The function now uses the fallocate system call to reserve sufficient
|
|
disk space on file systems that support it.
|
|
|
|
Optimizations for operations of the form ``A.T @ A`` and ``A @ A.T``
|
|
--------------------------------------------------------------------
|
|
Previously, ``gemm`` BLAS operations were used for all matrix products. Now,
|
|
if the matrix product is between a matrix and its transpose, it will use
|
|
``syrk`` BLAS operations for a performance boost. This optimization has been
|
|
extended to ``@``, ``numpy.dot``, ``numpy.inner``, and ``numpy.matmul``.
|
|
|
|
**Note:** Requires the transposed and non-transposed matrices to share data.
|
|
|
|
``np.testing.assert_warns`` can now be used as a context manager
|
|
----------------------------------------------------------------
|
|
This matches the behavior of ``assert_raises``.
|
|
|
|
Speed improvement for np.random.shuffle
|
|
---------------------------------------
|
|
``np.random.shuffle`` is now much faster for 1d ndarrays.
|
|
|
|
|
|
Changes
|
|
=======
|
|
|
|
Pyrex support was removed from ``numpy.distutils``
|
|
--------------------------------------------------
|
|
The method ``build_src.generate_a_pyrex_source`` will remain available; it
|
|
has been monkeypatched by users to support Cython instead of Pyrex. It's
|
|
recommended to switch to a better supported method of build Cython
|
|
extensions though.
|
|
|
|
``np.broadcast`` can now be called with a single argument
|
|
---------------------------------------------------------
|
|
The resulting object in that case will simply mimic iteration over
|
|
a single array. This change obsoletes distinctions like
|
|
|
|
if len(x) == 1:
|
|
shape = x[0].shape
|
|
else:
|
|
shape = np.broadcast(\*x).shape
|
|
|
|
Instead, ``np.broadcast`` can be used in all cases.
|
|
|
|
``np.trace`` now respects array subclasses
|
|
------------------------------------------
|
|
This behaviour mimics that of other functions such as ``np.diagonal`` and
|
|
ensures, e.g., that for masked arrays ``np.trace(ma)`` and ``ma.trace()`` give
|
|
the same result.
|
|
|
|
``np.dot`` now raises ``TypeError`` instead of ``ValueError``
|
|
-------------------------------------------------------------
|
|
This behaviour mimics that of other functions such as ``np.inner``. If the two
|
|
arguments cannot be cast to a common type, it could have raised a ``TypeError``
|
|
or ``ValueError`` depending on their order. Now, ``np.dot`` will now always
|
|
raise a ``TypeError``.
|
|
|
|
``linalg.norm`` return type changes
|
|
-----------------------------------
|
|
The ``linalg.norm`` function now does all its computations in floating point
|
|
and returns floating results. This change fixes bugs due to integer overflow
|
|
and the failure of abs with signed integers of minimum value, e.g., int8(-128).
|
|
For consistency, floats are used even where an integer might work.
|
|
|
|
|
|
Deprecations
|
|
============
|
|
|
|
Views of arrays in Fortran order
|
|
--------------------------------
|
|
The F_CONTIGUOUS flag was used to signal that views using a dtype that
|
|
changed the element size would change the first index. This was always
|
|
problematical for arrays that were both F_CONTIGUOUS and C_CONTIGUOUS
|
|
because C_CONTIGUOUS took precedence. Relaxed stride checking results in
|
|
more such dual contiguous arrays and breaks some existing code as a result.
|
|
Note that this also affects changing the dtype by assigning to the dtype
|
|
attribute of an array. The aim of this deprecation is to restrict views to
|
|
C_CONTIGUOUS arrays at some future time. A work around that is backward
|
|
compatible is to use ``a.T.view(...).T`` instead. A parameter may also be
|
|
added to the view method to explicitly ask for Fortran order views, but
|
|
that will not be backward compatible.
|
|
|
|
Invalid arguments for array ordering
|
|
------------------------------------
|
|
It is currently possible to pass in arguments for the ``order``
|
|
parameter in methods like ``array.flatten`` or ``array.ravel``
|
|
that were not one of the following: 'C', 'F', 'A', 'K' (note that
|
|
all of these possible values are both unicode and case insensitive).
|
|
Such behavior will not be allowed in future releases.
|
|
|
|
Random number generator in the ``testing`` namespace
|
|
----------------------------------------------------
|
|
The Python standard library random number generator was previously exposed
|
|
in the ``testing`` namespace as ``testing.rand``. Using this generator is
|
|
not recommended and it will be removed in a future release. Use generators
|
|
from ``numpy.random`` namespace instead.
|
|
|
|
Random integer generation on a closed interval
|
|
----------------------------------------------
|
|
In accordance with the Python C API, which gives preference to the half-open
|
|
interval over the closed one, ``np.random.random_integers`` is being
|
|
deprecated in favor of calling ``np.random.randint``, which has been
|
|
enhanced with the ``dtype`` parameter as described under "New Features".
|
|
However, ``np.random.random_integers`` will not be removed anytime soon.
|
|
|
|
|
|
FutureWarnings
|
|
==============
|
|
|
|
Assigning to slices/views of ``MaskedArray``
|
|
--------------------------------------------
|
|
Currently a slice of a masked array contains a view of the original data and a
|
|
copy-on-write view of the mask. Consequently, any changes to the slice's mask
|
|
will result in a copy of the original mask being made and that new mask being
|
|
changed rather than the original. For example, if we make a slice of the
|
|
original like so, ``view = original[:]``, then modifications to the data in one
|
|
array will affect the data of the other but, because the mask will be copied
|
|
during assignment operations, changes to the mask will remain local. A similar
|
|
situation occurs when explicitly constructing a masked array using
|
|
``MaskedArray(data, mask)``, the returned array will contain a view of ``data``
|
|
but the mask will be a copy-on-write view of ``mask``.
|
|
|
|
In the future, these cases will be normalized so that the data and mask arrays
|
|
are treated the same way and modifications to either will propagate between
|
|
views. In 1.11, numpy will issue a ``MaskedArrayFutureWarning`` warning
|
|
whenever user code modifies the mask of a view that in the future may cause
|
|
values to propagate back to the original. To silence these warnings and make
|
|
your code robust against the upcoming changes, you have two options: if you
|
|
want to keep the current behavior, call ``masked_view.unshare_mask()`` before
|
|
modifying the mask. If you want to get the future behavior early, use
|
|
``masked_view._sharedmask = False``. However, note that setting the
|
|
``_sharedmask`` attribute will break following explicit calls to
|
|
``masked_view.unshare_mask()``.
|