Do all the grid failures occur only for the FPF tests? I'm wondering whether we would get max scores for everything if the FPF were removed. If other tests fail then we could use those to cull candidates, then use the distributions to select the best.
I know there's no shortage of BCFN fails because they get logged for verification of the glitch handling logic. Sometimes 100% of the reason is BCFN.
EDIT: I does look like less than optimal scores are usually because of FPF though.
EDIT2: All the Practrand report data is still intact - the 58000 files (s16 only), 242 MB uncompressed. I could automate a tally up of the various causes I guess.
Here are some combined scores for frequency distributions. The table belows show the top 10 for prank only and zrank only when the two rankings are added and sorted from lowest to highest. The lowest prank is the tiebreaker for equal prank+zrank. Candidates that are not in either top 10 have been omitted.
PPS: Took about double the time to run the odds vs the evens.
The best few (1st, 4th and 6th) scores there look very much good enough to be chosen. They wouldn't be obvious choices on the first summary chart, see below, but for the high distribution scoring.
[13 5 10 9] passes the single iterate auto-culling but not the double iterate. I'm not sure why this one is not in the above single iterate summary chart. I must have hand removed it just to get the numbers down.
[13 5 8 10] passes the single iterate auto-culling but not the double iterate. This candidate is actually there. The only one I think.
[7 2 14 10] passes both single iterate and double iterate auto-culling. Its lowest score doesn't occur in the culling scores. I probably hand removed it for looking too weak compared to others.
[6 2 11 3] also passes both single iterate and double iterate auto-culling. Again, it would've been smothered.
[5 2 6 5] badly fails both single iterate and double iterate auto-culling. Its lowest score is present in the culling scores.
[11 2 6 2] passes the single iterate auto-culling but not the double iterate. Another smothered candidate I guess.
Just completed double-iterated even-apertured of the first four candidates. Here's those two grids: (Don't compare the exponent averages with the all-apertured grids version)
Thanks a lot, Evan. I've just sent you an email with full prank and zrank scores, which I'll post here later. Are the double-iterated scores 16 x 16 or 32 x 32?
Oh, I see the list order has changed. First six held luckily.
The grids are 32x32 but skipping the odd sized apertures to speed things up.
Evan, if the FPF tests were disabled, would the single and double iterated scores be maximums across the board? In other words, do they just show when FPF fails?
Would it be possible to test without FPF some time? I think you looked at this before but I don't know how systematically. As the tests could run a long time I suggest only testing bit 0 as the lsb, to begin with anyway.
To what end? FPF does appear to be the reason for good candidates to score below optimal. Without it they'll all max out and we'll have no comparison.
I could try a select few candidates I guess to see if any specific cases have flaws. It would need to be full grids to see anything of value I'd think.
Anecdotally, there is occasional non-maximums on a good candidate without FPF tests. Candidate [13 5 10 9] at apertures 16>>5 and 16>>13 have both had a case of creeping BCFN fails that reduced those two scores to 2 GB - both would have given up at 8 GB. Every other case has scored the full 16 GB so far. Still haven't reached 8-bit sampling aperture size yet though.
Got a couple of 8 GB scores from aperture size 12 now, both due to marginal BCFN fails. And aperture size 11 has a bunch of em ... I guess it'll continue. Hmm, this'll slow progress quite a lot with the way I've got the BCFN glitch detection functioning.
Yeah, those BCFNs progressively get worse as the test length is forced up. The glitch detection logic deletes the longer tries that don't manage to straddle the glitch, ie: assumes the BCFN is real ... I guess those attempts could be left in with a filename suffix.
Tony,
#2 candidate, [13 13 14 13], in the distribution high score list has very peculiar Practrand scores in the grid. 16-bit sampling looks just fine, while 8-bit sampling takes a severe dive! I guess having three 13's as the constants isn't helping the situation.
#5 candidate, [7 1 8 6], isn't a whole better with rubbish 16-bit sampling. I'm wondering if that candidate fits one of Mellisa's don't-ever-use-those-combinations-of-constants criteria. And I'm also wondering if she has a single page with all the good rules to follow.
Tony,
#2 candidate, [13 13 14 13], in the distribution high score list has very peculiar Practrand scores in the grid. 16-bit sampling looks just fine, while 8-bit sampling takes a severe dive! I guess having three 13's as the constants isn't helping the situation.
#5 candidate, [7 1 8 6], isn't a whole better with rubbish 16-bit sampling. I'm wondering if that candidate fits one of Mellisa's don't-ever-use-those-combinations-of-constants criteria. And I'm also wondering if she has a single page with all the good rules to follow.
Thanks for all the test results, Evan. I kept quiet yesterday so you wouldn't stay up all night again.
I don't recall where Melissa mentions which xoroshiro constants not to use. Her last blog post is not as interesting as some others, but I mention it because I can't see the various Benchmark Results, only the titles. Can you?
It's interesting how double-iterating and using all 32 bits improves some candidates that will clearly do badly when single-iterated and [13,13,14,13] is a classic example. As I haven't shown Chris's 32-bit state, 32-bit output PRNG lately, here it is again:
unsigned int rng_state = 1;
unsigned int get_rng_output() {
unsigned int tmp = state;
state ^= state << 13;
state ^= state >> 17;
state ^= state << 5;
tmp += (tmp << 18) | (tmp >> 14);//barrel shift and add
tmp += ((tmp << 7) | (tmp >> 25)) ^ ((tmp << 11) | (tmp >> 21));//two barrel shifts, an xor, and an add
return tmp ^ state;
}
There is not enough time to do two 32-bit additions, so we can't use it even if we wanted to. XORO32 could produce a 32-bit unequidistributed output fairly close in quality to Chris's with a change in constants and always produces two 16-bit equidistributed outputs. The best of both worlds, I think.
Thanks for all the test results, Evan. I kept quiet yesterday so you wouldn't stay up all night again.
No worries, I'm not working at the moment anyway. Got laid off a couple months back when management decided to shift the factory overseas. I didn't particularly want to move with it.
I don't recall where Melissa mentions which xoroshiro constants not to use.
It sounded like a general rule of LCGs or maybe scramblers. It was a criticism in the first review of Xoshiro** I think ...
EDIT: Found it, it was about ease of invertibility (of the scrambler I think). Maybe, that's irrelevant to Xoroshiro because the scrambler is summing two components of the state - a feature that was dropped with Xoshiro.
Her last blog post is not as interesting as some others, but I mention it because I can't see the various Benchmark Results, only the titles. Can you?
Ha, those are all Structured Vector Graphics. Wow, I think the only place I've seen an SVG before is Wikipedia, but Wikipedia has automatic conversion to bitmap for browsers that can't handle an SVG.
I've emailed you the SVGs converted to PNGs.
It's interesting how double-iterating and using all 32 bits improves some candidates that will clearly do badly when single-iterated ...
That goes both ways. Hopefully there's something good in the three way mix with the distribution scores as well.
EDIT: Found it, it was about ease of invertibility (of the scrambler I think). Maybe, that's irrelevant to Xoroshiro because the scrambler is summing two components of the state - a feature that was dropped with Xoshiro.
More specifically, it's the ** scrambler that dropped the combining of two components of the state, not Xoshiro per se.
Xoshiro looks like it could also make use of the ++ scrambler but that one wasn't included in the published paper.
EDIT: Which probably means the Xoroshiro128** used for the Prop2 free-running generator has a notable weakness.
Comments
EDIT: I does look like less than optimal scores are usually because of FPF though.
EDIT2: All the Practrand report data is still intact - the 58000 files (s16 only), 242 MB uncompressed. I could automate a tally up of the various causes I guess.
PractRand scores needed for top four or five.
Started gridding your whole list a few hours back. Found it was still configured for scoring even sized apertures. Now doing the odds.
PS: You'll also find the post-culling even-apertured double-iterated chart in that ODS file too.
The best few (1st, 4th and 6th) scores there look very much good enough to be chosen. They wouldn't be obvious choices on the first summary chart, see below, but for the high distribution scoring.
https://forums.parallax.com/discussion/comment/1441850/#Comment_1441850
[13 5 8 10] passes the single iterate auto-culling but not the double iterate. This candidate is actually there. The only one I think.
[7 2 14 10] passes both single iterate and double iterate auto-culling. Its lowest score doesn't occur in the culling scores. I probably hand removed it for looking too weak compared to others.
[6 2 11 3] also passes both single iterate and double iterate auto-culling. Again, it would've been smothered.
[5 2 6 5] badly fails both single iterate and double iterate auto-culling. Its lowest score is present in the culling scores.
[11 2 6 2] passes the single iterate auto-culling but not the double iterate. Another smothered candidate I guess.
Thanks a lot, Evan. I've just sent you an email with full prank and zrank scores, which I'll post here later.
When one includes the distributions, I think the choice is between these two:
The fact that a and b constants are the same is a good sign.
The grids are 32x32 but skipping the odd sized apertures to speed things up.
Would it be possible to test without FPF some time? I think you looked at this before but I don't know how systematically. As the tests could run a long time I suggest only testing bit 0 as the lsb, to begin with anyway.
I could try a select few candidates I guess to see if any specific cases have flaws. It would need to be full grids to see anything of value I'd think.
I wouldn't test more than these four:
Attached is every grid PRreport for [7 10 10 9], both with and without FPF tests:
Lots of "early" BCFN_FF failures.
It's 6 AM, I'm off to bed.
As well as the Practrand reports, here's the grid score tables of those few candidates for both with and without FPF tests:
#2 candidate, [13 13 14 13], in the distribution high score list has very peculiar Practrand scores in the grid. 16-bit sampling looks just fine, while 8-bit sampling takes a severe dive! I guess having three 13's as the constants isn't helping the situation.
#5 candidate, [7 1 8 6], isn't a whole better with rubbish 16-bit sampling. I'm wondering if that candidate fits one of Mellisa's don't-ever-use-those-combinations-of-constants criteria. And I'm also wondering if she has a single page with all the good rules to follow.
Thanks for all the test results, Evan. I kept quiet yesterday so you wouldn't stay up all night again.
I don't recall where Melissa mentions which xoroshiro constants not to use. Her last blog post is not as interesting as some others, but I mention it because I can't see the various Benchmark Results, only the titles. Can you?
It's interesting how double-iterating and using all 32 bits improves some candidates that will clearly do badly when single-iterated and [13,13,14,13] is a classic example. As I haven't shown Chris's 32-bit state, 32-bit output PRNG lately, here it is again:
There is not enough time to do two 32-bit additions, so we can't use it even if we wanted to. XORO32 could produce a 32-bit unequidistributed output fairly close in quality to Chris's with a change in constants and always produces two 16-bit equidistributed outputs. The best of both worlds, I think.
It sounded like a general rule of LCGs or maybe scramblers. It was a criticism in the first review of Xoshiro** I think ...
EDIT: Found it, it was about ease of invertibility (of the scrambler I think). Maybe, that's irrelevant to Xoroshiro because the scrambler is summing two components of the state - a feature that was dropped with Xoshiro.
Ha, those are all Structured Vector Graphics. Wow, I think the only place I've seen an SVG before is Wikipedia, but Wikipedia has automatic conversion to bitmap for browsers that can't handle an SVG.
I've emailed you the SVGs converted to PNGs.
That goes both ways. Hopefully there's something good in the three way mix with the distribution scores as well.
More specifically, it's the ** scrambler that dropped the combining of two components of the state, not Xoshiro per se.
Xoshiro looks like it could also make use of the ++ scrambler but that one wasn't included in the published paper.
EDIT: Which probably means the Xoroshiro128** used for the Prop2 free-running generator has a notable weakness.
I thought all current browsers supported SVG.
SVG is a wonderful thing and used a lot now that we have libraries like D3 https://d3js.org/
What is ideal? I'm guessing approaching zero, maybe, given the 1.0 reference is the "classic biased-mod method".
Has an aversion to XP maybe. It certainly pushed me to find alternatives.