You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
align is not working when using only strings #5293
I'm not even sure what alignment is supposed to mean for strings -
characters are single bytes, so do they just need single byte alignment?
What are you hoping to learn by checking the alignment flag? Do you have
some code that requires a specific alignment?
On 19 Nov 2014 02:44, "pduch" [email protected] wrote:
import numpy as np
a = np.zeros(4, dtype=np.dtype('|S4', align=True))
print a.flags['ALIGNED'] # -- should be True
Works with 1.8 but not with 1.9
—
Reply to this email directly or view it on GitHub #5293.
I need to call c code with an array of character. It has been working for
years but with 1.9 it is not.
The issue is that it is impossible to get the align flag to true, therefore
impossible to call c code.
I'm not even sure what alignment is supposed to mean for strings -
characters are single bytes, so do they just need single byte alignment?
What are you hoping to learn by checking the alignment flag? Do you have
some code that requires a specific alignment?
On 19 Nov 2014 02:44, "pduch" [email protected] wrote:
import numpy as np
a = np.zeros(4, dtype=np.dtype('|S4', align=True))
print a.flags['ALIGNED'] # -- should be True
Works with 1.8 but not with 1.9
—
Reply to this email directly or view it on GitHub #5293.
—
Reply to this email directly or view it on GitHub #5293 (comment).
you are using a 32 bit platform and the array is 8 byte aligned?
the issue is that in the new indexing code the string is copied by the itemsize so it needs itemsize alignment. Though currently it uses 16 bytes for all string types which could probably be reduced to the itemsize which would allow S4 to work but not S16 or S9.
I think the new indexing is the only code that tries to copy based on the itemsize so possibly we could revert back to no alignment requirements for sizes not powers of 2 and itemsize alignment for sizes of power 2, that would still leave S16 as problematic as we need to check what is the maximum algnment the compiler needs to copy. GCC provides this information in a macro, other compilers likely not.
or we decouple the aligned flag of an array from the actual required alignment.
the aligned flag then only indicates the alignment of the type which is redundant information for C programs and useless inside of numpy. But it probably makes third party users happy.
Yeah, that's why I'm looking for information on how people are actually
using the aligned flag. :-)
@pduch: In general there shouldn't be any problem with calling C code with
an unaligned array. (After all, numpy is itself written in C, and it
handles unaligned arrays fine!) It should only be a problem if the specific
C code in question is written in such a way that it has specific alignment
requirements. Can you elaborate on what you're doing exactly?
On 19 Nov 2014 14:44, "Julian Taylor" [email protected] wrote:
or we decouple the aligned flag of an array from the actual required
alignment.
the aligned flag then only indicates the alignment of the type which is
redundant information for C programs and useless inside of numpy. But it
probably makes third party users happy.
—
Reply to this email directly or view it on GitHub #5293 (comment).
Here is the full version... :)
I am on Windows-x86_64 architecture using Python 2.7.2. I have not tested
it on our Linux box yet......
We are calling, from Python, an optimizer with a C api. It has been working
for years but since we upgraded to 1.9, we are getting the following error:
ArgumentError: .....: array must have flags ['C_CONTIGUOUS', 'ALIGNED']
We are using ctypes to call the C api. The parameter causing the problem is
a char array:
my_array = zeros(26, dtype='|S1')
When I check the flags on this, ALIGNED is always False. It was True with
previous versions.
or we decouple the aligned flag of an array from the actual required
alignment.
the aligned flag then only indicates the alignment of the type which is
redundant information for C programs and useless inside of numpy. But it
probably makes third party users happy.
—
Reply to this email directly or view it on GitHub #5293 (comment).
nevermind there is a bug in the alignment check it also needs uses the copy alignment for the strides which is of course false for stride 1, that needs to be fixed
nevermind there is a bug in the alignment check it also needs uses the
copy alignment for the strides which is of course false for stride 1, that
needs to be fixed
—
Reply to this email directly or view it on GitHub #5293 (comment).
Activity
njsmith commentedon Nov 19, 2014
I'm not even sure what alignment is supposed to mean for strings -
characters are single bytes, so do they just need single byte alignment?
What are you hoping to learn by checking the alignment flag? Do you have
some code that requires a specific alignment?
On 19 Nov 2014 02:44, "pduch" [email protected] wrote:
pduch commentedon Nov 19, 2014
I need to call c code with an array of character. It has been working for
years but with 1.9 it is not.
The issue is that it is impossible to get the align flag to true, therefore
impossible to call c code.
2014-11-19 8:30 GMT-05:00 Nathaniel J. Smith [email protected]:
juliantaylor commentedon Nov 19, 2014
you are using a 32 bit platform and the array is 8 byte aligned?
the issue is that in the new indexing code the string is copied by the itemsize so it needs itemsize alignment. Though currently it uses 16 bytes for all string types which could probably be reduced to the itemsize which would allow S4 to work but not S16 or S9.
I think the new indexing is the only code that tries to copy based on the itemsize so possibly we could revert back to no alignment requirements for sizes not powers of 2 and itemsize alignment for sizes of power 2, that would still leave S16 as problematic as we need to check what is the maximum algnment the compiler needs to copy. GCC provides this information in a macro, other compilers likely not.
juliantaylor commentedon Nov 19, 2014
or we decouple the aligned flag of an array from the actual required alignment.
the aligned flag then only indicates the alignment of the type which is redundant information for C programs and useless inside of numpy. But it probably makes third party users happy.
njsmith commentedon Nov 19, 2014
Yeah, that's why I'm looking for information on how people are actually
using the aligned flag. :-)
@pduch: In general there shouldn't be any problem with calling C code with
an unaligned array. (After all, numpy is itself written in C, and it
handles unaligned arrays fine!) It should only be a problem if the specific
C code in question is written in such a way that it has specific alignment
requirements. Can you elaborate on what you're doing exactly?
On 19 Nov 2014 14:44, "Julian Taylor" [email protected] wrote:
pduch commentedon Nov 19, 2014
Hi Jilian,
Here is the full version... :)
I am on Windows-x86_64 architecture using Python 2.7.2. I have not tested
it on our Linux box yet......
We are calling, from Python, an optimizer with a C api. It has been working
for years but since we upgraded to 1.9, we are getting the following error:
ArgumentError: .....: array must have flags ['C_CONTIGUOUS', 'ALIGNED']
We are using ctypes to call the C api. The parameter causing the problem is
a char array:
my_array = zeros(26, dtype='|S1')
When I check the flags on this, ALIGNED is always False. It was True with
previous versions.
Thanks,
Patrick
2014-11-19 9:44 GMT-05:00 Julian Taylor [email protected]:
juliantaylor commentedon Nov 19, 2014
hm why should it be unaligned on win64, what is the alignment of the array? check with:
juliantaylor commentedon Nov 19, 2014
nevermind there is a bug in the alignment check it also needs uses the copy alignment for the strides which is of course false for stride 1, that needs to be fixed
pduch commentedon Nov 19, 2014
OK thanks a lot :)
Cheers,
P
2014-11-19 12:21 GMT-05:00 Julian Taylor [email protected]:
BUG: fix string arrays not being aligned
pduch commentedon Apr 22, 2015
thanks guys
2015-04-21 22:25 GMT-04:00 Julian Taylor [email protected]: