python,arrays,numpy,input-sanitization , input/output validation/casting of a numpy calculation

## Question:

Tag: python,arrays,numpy,input-sanitization

This is a situation that happens quite often in my codes. Say I have a function `do_sth(a,b)`, that, only for the sake of this example, simply calculates `a+b`, with `a,b` either 1D `numpy` arrays or scalars. In many occasions, I need the function to broadcast the operation, so that if both `a,b` are 1D arrays, the result will be a 2D array. An example of what I mean follows:

``````do_sth(1,2) -> 3
do_sth([1,2],0) -> array([1, 2])
do_sth(0,[3,4]) -> array([3, 4])
do_sth([1,2],[3,4]) -> array([[4, 5], [5, 6]])
``````

This is a bit similar to how a numpy `ufunc` behaves. A possible implementation follows:

``````from numpy import newaxis, atleast_1d

def do_sth(a, b):
"a,b should be either 1d numpy arrays or scalars"
a, b = map(atleast_1d, [a, b])
# the line below mocks a more complicated calculation
res = a[:, newaxis] + b[newaxis]

conds = [a.size == 1, b.size == 1]

if all(conds):
return res[0, 0]
elif any(conds):
return res.ravel()
else:
return res
``````

As you can see, there's quite a lot of boilerplate. The first question is: is this the right way to do this input/output casting? Is there any reason to not use a decorator to deal with a situation like this? Is there any guideline on the matter?

Moreover, the more complicated calculation, here mocked by the addition, often fails badly if `a` or `b` are numpy arrays with 2D,3D shape for example. I say badly in the sense that the point where the calculation fails is not obvious, or may change with time in different revisions of the code, and it is hard to see the connection between the error and the wrong input shape. I think it is then NOT advisable to put the complicated calculation in a `try/except` block (following python EAFP). In this case, is it correct to check the shape of the 2 arrays at the beginning of the function? Is there any alternative? Is there a numpy function that allows at the same time to convert the input to a numpy array, and also check that the input is compatible with a certain number of dimensions, something like `asarray_withdim(arr,ndim=5)`?

Regarding the use of decorators - I haven't seen much use of decorators in `numpy` code, but I think that's because most of the functionality was developed before decorators become common in Python. If you can make it work, there shouldn't be a any downside (but I'm not an expert with either decorators or `ufunc`).

Non complied numpy functions often have a lot of code that massages the inputs into convenient dimensions. Then they do the core action, followed by final reshaping and type wrapping. They might use functions like np.atleast_2d to ensure there are enough dimensions, and .reshape(-1,1,1) to compress excess dimensions.

`np.tensordot` is an example of one that performs axes transpose plus reshape on the inputs so it can apply the compiled `np.dot`. `np.insert` starts with a number of `ndim` and `isinstance` tests. Special cases are handled early, while the general one is left to the end. `np.einsum` is compiled, but there's a lot of preprocessing being done in C code, before it finally creates an `nditer` object and does the calculation.

# Related:

## Pandas Dataframe Complex Calculation

python,python-2.7,pandas,dataframes
I have the following dataframe,df: Year totalPubs ActualCitations 0 1994 71 191.002034 1 1995 77 2763.911781 2 1996 69 2022.374474 3 1997 78 3393.094951 I want to write code that would do the following: Citations of currentyear / Sum of totalPubs of the two previous years I want something to...

## Blank screen on GridView

android,arrays,gridview
I'm trying to create a GridView with an array of strings. These are XML, and MainActivity and Adapter, but what I get is a blank screen. I'm change the background, but the result is the same, but clicking on a point on the screen appears to me the toast stating...

## Sort when values are None or empty strings python

python,list,sorting,null
I have a list with dictionaries in which I sort them on different values. I'm doing it with these lines of code: def orderBy(self, col, dir, objlist): if dir == 'asc': sorted_objects = sorted(objlist, key=lambda k: k[col]) else: sorted_objects = sorted(objlist, key=lambda k: k[col], reverse=True) return sorted_objects Now the problem...

## How to pivot array into another array in Ruby

arrays,ruby,csv
I have a multidimensional array like this one : myArray = [["Alaska","Rain","3"],["Alaska","Snow","4"],["Alabama","Snow","2"],["Alabama","Hail","1"]] I would like to end up with CSV output like this. State,Snow,Rain,Hail Alaska,4,3,nil Alabama,2,nil,1 I know that to get this outputted to CSV the way I want it I have to have output array like this: outputArray =[["State","Snow","Rain","Hail"],["Alaska",4,3,nil],["Alabama",2,nil,1]]...

## How to pass array in rails 4 strong parameters

ruby-on-rails,arrays
I have to pass a array of food_item_ids in my order_controller. Every order will have many food_items. How can I pass these food_items_id as an array in strong parameters. orders_controller.rb def create @order = Order.new(order_params) if @order.save render :json, @order, status:201, location: [:api, @order] else render :json, { errors: @order.errors...

## SQLAlchemy. 2 different relationships for 1 column

python,sqlalchemy
I have a simple many-to-many relationship with associated table: with following data: matches: users: users_mathces: ONE user can play MANY matches and ONE match can involve up to TWO users I want to realize proper relationships in both "Match" and "User" classes users_matches_table = Table('users_matches', Base.metadata, Column('match_id', Integer, ForeignKey('matches.id', onupdate="CASCADE",...