PyArray_SimpleNewFromData example

Question

I know that this thing has been answered a lot of times and I have also read the documentation as well but still I am not able to clearly understand how is this working. As in, I am not able to understand how the values are populated in its arguments. The examples are not very clearly explaining it(or may be I am not able to). Can anyone please help me understand how are the arguments of this function populated? What should be their values? I have to pass a vector from C++ to Python without reallocating the memory. Any help is much appreciated. I am stuck on this from a lot of days.

My code which I am implementing:

int main(int argc, char *argv[])
{
PyObject *pName, *pModule, *pDict, *pFunc, *pValue, *pArgs,*pXVec,*c, *xarr1;
int i;
float fArray[5] = {0,1,2,3,4};
//float *p = &fArray[0] ;
npy_intp m = 5;
//void* PyArray_GetPtr(PyArrayObject* aobj, npy_intp* ind)¶


// Initialize the Python Interpreter
Py_Initialize();
PySys_SetArgv(argc, argv); 
// Build the name object
pName = PyString_FromString(argv[1]);

// Load the module object
pModule = PyImport_Import(pName);
printf("check0\n");
// pDict is a borrowed reference 
pDict = PyModule_GetDict(pModule);
printf("check1\n");
// pFunc is also a borrowed reference 
pFunc = PyDict_GetItemString(pDict, argv[2]);
printf("check2\n");
//    if (PyCallable_Check(pFunc)) 
//    {
// Prepare the argument list for the call
//xarr1 = PyFloat_FromDouble(xarr[1]);
    printf("check3\n");
c = PyArray_SimpleNewFromData(1,&m,NPY_FLOAT,(void *)fArray);
printf("check3\n");
    pArgs = PyTuple_New(1);
    PyTuple_SetItem(pArgs,0, c);    

    pValue = PyObject_CallObject(pFunc, pArgs);

    if (pArgs != NULL)
    {
        Py_DECREF(pArgs);
    }

//}
//   else 
//    {
//        PyErr_Print();
//    }

// Clean up
Py_DECREF(pModule);
Py_DECREF(pName);

// Finish the Python Interpreter
Py_Finalize();

return 0;
}

Solution

The function is:

 PyObject *
    PyArray_SimpleNewFromData(
        int nd, 
        npy_intp* dims, 
        int typenum, 
         void* data)

• The last argument (data) is a buffer to the data. Let's dispense with that.

• The second argument (dims) is a buffer, whose each entry is a dimension; so for a 1d array, it could be a length-1 buffer (or even an integer, since each integer is a length-1 buffer)

• Since the second argument is a buffer, the first argument (nd) tells its length

• The third argument (typenum) indicates the type.

---

For example, say you have 4 64-bit ints at x:

To create an array, use

int dims[1];
dims[0] = 4;
PyArray_SimpleNewFromData(1, dims, NPY_INT64, x)

To create a 2X2 matrix, use

int dims[2];
dims[0] = dims[1] = 2;
PyArray_SimpleNewFromData(2, dims, NPY_INT64, x)

Comments

[Sajal Jain]

have something like this : float fArray[5] = {0,1,2,3,4}; npy_intp m = 14; PyObject *c ; c = PyArray_SimpleNewFromData(1,&m,NPY_FLOAT,(void *)fArray); Can you please let me know what is the error in this as I am getting segmentation fault.

[Ami Tavory]

Eee, sorry - it looks fine. Are you absolutely sure the segfault is from this? Could you printf immediately before and after this line?

[Sajal Jain]

I am stuck on how x is evaluated here. First 3 arguments are clear to me.

[Sajal Jain]

Yes I did and this is the line where seg fault is there.

[Ami Tavory]

Bummer. Your use of x seems fine to me. Sorry, I'm out of ideas.

[Sajal Jain]

So, I am again stuck on this :(

[Ami Tavory]

Looks so. I suggest you go a different way. The problem, I think, is not with this function, but about dealing with segfaults in numpy. See if you can use valgrind on it. If not, start a new SO question whose focus is on valgrind/numpy, and refers to this as an instance of the problem.

[Jaime]

You have provided a buffer with room for 5 items, fArray, but are telling it that it has room for 14 (m), so that's a segmentation fault waiting to happen as soon as you try accessing it. That said, your problem is more likely due to not calling import_array() at module initialization, see this.

[Ami Tavory]

Oh, very nice - I missed that 14. Great job.

[Sajal Jain]

Even after changing it to 5 the error is still there. And I have just posted the relevant strings for the function, I am using all the functions like import_array() and initialize.

[Sajal Jain]

I just posted my code. Please see if you find any error in this.

[musbur]

@Ami, your "dims" argument is of type int* whereas PyArray_SimpleNewFromData expects an int** type (and the compiler complains accordingly). Yet your code works. Why?

[Tony]

@AmiTavory I am having a problem using this example. When I create the array in python the numbers in numpy array are not the ones set, instead some extremely big numbers are in the array. I am converting doubles to NPY_DOUBLE. Any idea what could be the issue?

[Ami Tavory]

@Tony Are you using NPY_INT64?

[Tony]

@AmiTavory No, but I am using doubles, the array is double x[2] = {1.2,1.5}.

[Ami Tavory]

@Tony I'd need to see the code to understand, and it's difficult to do via a comments dialog. Could you open a new question, with minimal code showing your problem? Feel free to paste here a link to it, and I'll have a look.

[Tony]

@AmiTavory I will consider opening a new question. However, I found the problem, the variable is deleted on function return. When using the new command to create the array, it works fine.