Repository navigation
Random V 2.0 task items #31
Description
Activity
This is great news.
Do we have the results of the tests written up somewhere?
I had a communication from Guy in April:
"My one recommendation is that you not standardize on a single algorithm.
Rather, make a SplittableRandom type class and provide multiple implementations."Are we saying that PCG is somehow not as suitable as splitmix? I guess a write up of the tests would help answer this question.
Pcg doesn't have a good quality split operator at present.
I do have the basic test run data lying around on my work computer, I just
Got a bit buried this fall, I'll see about digging it upOn Wednesday, December 9, 2015, idontgetoutmuch [email protected]
wrote:This is great news.
Do we have the results of the tests written up somewhere?
I had a communication from Guy in April:
"My one recommendation is that you not standardize on a single algorithm.
Rather, make a SplittableRandom type class and provide multiple
implementations."Are we saying that PCG is somehow not as suitable as splitmix? I guess a
write up of the tests would help answer this question.—
Reply to this email directly or view it on GitHub
#31 (comment).You are free to use the code in any way you see fit, I'll try to help as much as I can.
@cartazio what parts of C code would you like to move to Haskell? TestU01 code is a mess written in a bad style (global variables, uninformative names etc.), I didn't refactor partly because it had no tests and I could break it without noticing, so I made only patchwork additions.
I moved parts of splitmix to C only for performance, although I cannot claim that my Haskell code is optimal.
@idontgetoutmuch correctness tests can be run using https://github.com/nkartashov/prng-test
btw, if you find any of the tools lacking, I can add you to collaborators, if you like.@nkartashov I haven't tried
prng-testyet but see the issues I have raised forhtestu. Perhaps I should not be tryinghtestu?@idontgetoutmuch You mean discard it as a tool?
See my comment on nkartashov/htestu#2. I am going to find a way we can run all the tests in
htestuand store the results somewhere. It probably ought to be part of a CI process for RNG.@cartazio do we have a CI set up? I know very little about how to provide such a thing but I can investigate.
I have made some small progress e.g. I can now show that
tf-randompassessmallcrush:[OK,OK,OK,OK,OK,OK,OK,OK,OK,OK,OK,OK,OK,OK,OK]But I am having problems using
htestuat the command line - see: nkartashov/htestu#3.I am copying my comments from a closed ticket so they don't get lost.
It's not ideas that are in short supply. We know we need to a) test the various PRNGs against the various randomness tests (and in particular L'Ecuyers Test01 suite) and b) test their performance.
I have a colleague who is an expert in the field of RNGs but who does not know Haskell. I have created a binding from R to allow him to test their effectiveness https://github.com/idontgetoutmuch/RandomHaskellFromR. I don't think we need a Haskell binding to Test01 (since we can call it from R) but we have one anyway: https://github.com/idontgetoutmuch/RandomHaskellFromR.
Oneuseful thing someone could do would be to create a runnable and repeatable performance test suite maybe using https://github.com/nkartashov/prng-bench but note the issue I raised with it.
My suspicion is that SplitMix will probably be a good replacement but we don't know yet. QuickCheck uses https://hackage.haskell.org/package/tf-random; my testing of it with Test01 shows it passes smallcrush.
If I sound frustrated, it's because I am.
We have that code , and we need find the time to clean it up. I've
scheduled the time this weekend. Let's synch up after that :)Or is there other bits I'm overlooking? I was a bit buried under job stuff
this winter and I'm only now climbing back up (eg I did my first new
package to havksge in a while this week ;))On Thursday, May 5, 2016, idontgetoutmuch [email protected] wrote:
I am copying my comments from a closed ticket so they don't get lost.
It's not ideas that are in short supply. We know we need to a) test the
various PRNGs against the various randomness tests (and in particular
L'Ecuyers Test01 suite) and b) test their performance.I have a colleague who is an expert in the field of RNGs but who does not
know Haskell. I have created a binding from R to allow him to test their
effectiveness https://github.com/idontgetoutmuch/RandomHaskellFromR. I
don't think we need a Haskell binding to Test01 (since we can call it from
R) but we have one anyway:
https://github.com/idontgetoutmuch/RandomHaskellFromR.One useful thing someone could do would be to create a runnable and
repeatable performance test suite maybe using
https://github.com/nkartashov/prng-bench but note the issue I raised with
it.My suspicion is that SplitMix will probably be a good replacement but we
don't know yet. QuickCheck uses
https://hackage.haskell.org/package/tf-random; my testing of it with
Test01 shows it passes smallcrush.If I sound frustrated, it's because I am.
—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#31 (comment)RandomT should not use split like QuickCheck. First off, split can be slow, and splitting for every operation is pretty bad for performance. Second, it means that it violates the monad laws:
m >>= return /= m
However, it should support using split to feed one half into a calculation and use the other half as the new random state.
Split in the splitmix is cheap, agree it's problematic in current random
and tf random. But let me catchup on stuff and finish getting that branch
upAlso benchmarks :)
On Tuesday, May 10, 2016, Zemyla [email protected] wrote:
RandomT should not use split like QuickCheck. First off, split can be
slow, and splitting for every operation is pretty bad for performance.
Second, it means that it violates the monad laws:m >>= return /= m
However, it should support using split to feed one half into a calculation
and use the other half as the new random state.—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#31 (comment)splitmix is the only RNG we know of implemented in haskell that supports a
splitoperation that passes big crush when used as the sequencing operationThat's strange, I ran the
c_bigCrushtests from https://github.com/nkartashov/htestu withnextStreamFromGen,splitNextStreamFromGenandleftSplitStreamFromGenfor PCG and it passed them all. Are you where you where using the right generator? (the fast generator would be horrible for splitting, you'd want the multiple sequence generator (the standard one))Also I recently wrote a pure Haskell version of the standard generator if you're interested.
Is this a pure Haskell version of splitmix?
On Fri, Jun 17, 2016 at 2:51 PM, Chris [email protected] wrote:
splitmix is the only RNG we know of implemented in haskell that supports a
split operation that passes big crush when used as the sequencing
operationThat's strange, I ran the c_bigCrush tests from
https://github.com/nkartashov/htestu with nextStreamFromGen,
splitNextStreamFromGen and leftSplitStreamFromGen for PCG and it passed
them all. Are you where you where using the right generator? (the fast
generator would be horrible for splitting, you'd want the multiple sequence
generator (the standard one))Also I recently wrote a pure Haskell version
http://hackage.haskell.org/package/pcg-random-0.1.3.3/docs/System-Random-PCG-Pure.html
of the standard generator if you're interested.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#31 (comment), or mute
the thread
https://github.com/notifications/unsubscribe/AAAhUQ4jx5oAI2L4b_tVma6iV49WxHnRks5qMuysgaJpZM4Gxd7y
.No, a pure version of multiple sequence variant of PCG (I only had a pure version of the fast variant of PCG before).
17 remaining items
- Oops. Different moritz than the one I thought. I’ve got 1.2 mostly done on it’s branch in my gh repo. Just need to figure out how to split some big breaking changes between several major versions On Wed, Feb 21, 2018 at 5:59 PM Carter Schonwald <[email protected]> wrote:…Let’s find a good time to chat on irc over the next 3 days. I’ll try to find you when you’re active. Or leave a set of pms On Wed, Feb 21, 2018 at 3:41 PM Moritz Kiefer ***@***.***> wrote: > Any progress on this? I’m happy to try and help as my skills (and time) > permits! > > I think it would be helpful to start by forming a concrete set of TODO > items that need to be addressed before we can get the next release and then > focus only on those. Otherwise I fear that the trend of more and more ideas > creeping into the scope of the next release continues and we won’t see a > release in the next two years either. > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#31 (comment)>, or mute > the thread > <https://github.com/notifications/unsubscribe-auth/AAAQwkoaJPZhWpfzuPzwponvFgw3t86rks5tXH9zgaJpZM4Gxd7y> > . >
- I was stumped on how to have a migrable improvement in the current random type class without totally breaking every current user. I’ve some design ideas I’ll share in the next few days that I think will help Merrry Wednesday all! On Wed, Feb 21, 2018 at 6:05 PM Carter Schonwald <[email protected]> wrote:…Oops. Different moritz than the one I thought. I’ve got 1.2 mostly done on it’s branch in my gh repo. Just need to figure out how to split some big breaking changes between several major versions On Wed, Feb 21, 2018 at 5:59 PM Carter Schonwald < ***@***.***> wrote: > Let’s find a good time to chat on irc over the next 3 days. I’ll try to > find you when you’re active. Or leave a set of pms > > On Wed, Feb 21, 2018 at 3:41 PM Moritz Kiefer ***@***.***> > wrote: > >> Any progress on this? I’m happy to try and help as my skills (and time) >> permits! >> >> I think it would be helpful to start by forming a concrete set of TODO >> items that need to be addressed before we can get the next release and then >> focus only on those. Otherwise I fear that the trend of more and more ideas >> creeping into the scope of the next release continues and we won’t see a >> release in the next two years either. >> >> — >> You are receiving this because you were mentioned. >> Reply to this email directly, view it on GitHub >> <#31 (comment)>, >> or mute the thread >> <https://github.com/notifications/unsubscribe-auth/AAAQwkoaJPZhWpfzuPzwponvFgw3t86rks5tXH9zgaJpZM4Gxd7y> >> . >> >
- What's the blocker? Why can't we just push something?…On Feb 25, 2018 13:38, "idontgetoutmuch" ***@***.***> wrote: I am not holding my breath on any substantive progress on this - I think we are coming up to the 3rd birthday of when this issue was raised on the libraries mailing list: https://mail.haskell.org/pipermail/libraries/2015- April/025439.html I still think this package should be deprecated so at least no-one uses it by accident. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAhUbwgErNMY_fPtwkX9wwgOV0k1FL3ks5tYWJPgaJpZM4Gxd7y> .
@zaxtax the last bit I was stuck on (which the friendly inquiries last week prompted me to finally ) was what design to put in an intermediate release that wouldn't be a 100% breaking change for ( existing historical codes).
https://github.com/cartazio/random/tree/v1.2.0 is the current wip, doesn't include some of the cleanup, and i've also been figuring out how to formall characterize the set of types i can give clean semantics too, vs which ones are system / platform dependent. I want samplers to have clear meaning !
the system.random api there is mid refactor, because theres a subtle problem with the current api, its not portable between 32 bit and 64bit, and honestly those system dependent types should be under their own interface, at least wrt mathematical sampling code (vs system testing code)
(also this thread is strictly for technical discourse, anything else is not constructive, demotivating, and I will delete :) )
- How important is it to still support 32bit? Is it possible to cut an experimental release with just 64bit?…On Sun, Feb 25, 2018 at 8:38 PM, Carter Tazio Schonwald < ***@***.***> wrote: (also this thread is strictly for technical discourse, anything else is not constructive, demotivating, and I will delete :) ) — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAhUZYLSKyBVFz0V0I4CL2FwGYuRwimks5tYcS0gaJpZM4Gxd7y> .
@zaxtax as long as GHC tier 1 platforms include 32bit flavors, pretty important
thats not the actual blocker, even among 32bit platforms, some sizes of types can vary between OSes etc, though in haskell code thats usually isolated to C related types
either way, i have a satisfactory story for that issue now, so its now on the top of my oss queue... been buried under type system design work for a bit
- Are there any improvements you made that can be pushed out without the big API refactor?…On 25 Feb 2018 21:25, "Carter Tazio Schonwald" ***@***.***> wrote: either way, i have a satisfactory story for that issue now, so its now on the top of my oss queue... been buried under type system design work for a bit — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAhUctxeMjmW40kglt-982WySi4omFDks5tYc_XgaJpZM4Gxd7y> .
- That’s what I’m doing for the 1.2 release. Stay tuned :)…On Sun, Feb 25, 2018 at 4:31 PM zaxtax ***@***.***> wrote: Are there any improvements you made that can be pushed out without the big API refactor? On 25 Feb 2018 21:25, "Carter Tazio Schonwald" ***@***.***> wrote: > either way, i have a satisfactory story for that issue now, so its now on > the top of my oss queue... been buried under type system design work for a > bit > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <#31 (comment)>, or mute > the thread > < https://github.com/notifications/unsubscribe-auth/AAAhUctxeMjmW40kglt-982WySi4omFDks5tYc_XgaJpZM4Gxd7y > > . > — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#31 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAAQwsFFPGyAWicuTkbk-XazrsgGM5Kfks5tYdFDgaJpZM4Gxd7y> .
i'll be pushing periodically to https://github.com/haskell/random/tree/v1.2.0
I believe this discussion is obsolete now. Closing.
Reacted by Alexey Kuleshevich
based up on the lovely GSOC work by @nkartashov at https://github.com/nkartashov/SplitMix (and associated repos https://github.com/nkartashov/htestu for big crush/small crush statistical validation along and then https://github.com/nkartashov/prng-bench for performance comparison), @zaxtax and I are starting some planning work for Random Version 2.0
(and fyi to @idontgetoutmuch @gbaz et al)
let it be known
nextoperation that passes big crushsplitoperation that passes big crush when used as the sequencing operation(i need to do further auditing of how those crush tests were setup, but thats another sub item)
the following (approximate) task items, in part leveraging the excellent work by @nkartashov should form the foundation for prepping a v2 of random