Coremark

brucee

I was at Radio Shack picking up a switch and noticed they had a Prop QuickStart on closeout sale for less than $10. So I grabbed it.

For my day job I've been doing some benchmarking of memory cache variations vs performance, so I have the latest Coremark code. While benchmarks are not the final word, they can and are a tool people use to compare CPUs. So I downloaded the Simple IDE and ported the code over, took a little longer than I thought. I will say Simple IDE is a good start, though I'd shoot myself if I had to use it for my main editor. This is not a knock on SIDE, I'd shoot myself if I had to switch editors anyway, and for most tools I use if it is more than a handful of lines, I'll switch to my tried and true and everyone has one. One suggestion there is that SIDE should check file time stamps to see if an outside editor has changed the file.

Anyway as to the results. I didn't try to setup an internal timer, and just used a stopwatch (there's an app for everything). And had to cut the iteration count down to 80 from the normal 8000.

The Prop 1 comes in at a score of 8.5 (0.43 per MHz)

I will publish this code here, after I cleanse it a bit. And let those in the know start tweaking.

let the flaming begin...

SRLM

Interesting. From this link, ARM has the following scores:

CM   CM/MHz
M4  510  3.4
M3  82   3.3
M0+ 111  2.3
M0  105  2.2

So, the Propeller (with this test) comes in at 1/50th the performance of an M0+.

Ken Gracey

With the main purpose of SimpleIDE and the C code we publish being for Education, speed isn't much of an issue. Plus, you should see what can be done on an ActivityBot with the compiler. It's more than enough for the intended purpose.

And you really need to look ahead, too. Our C compiler was developed to establish itself as existing prior to P2's arrival. For P2, we expect high speed and more efficiency.

Keep this in mind while you spread the fire. Are you the target audience?

Ken Gracey

jazzed

LMM mode would be faster.

The fastest would be COGC, but that is specialized and small sized.

Propeller was not designed for C.

Bill Henning

Huh?

105/8.5 = 12.35294

NOT 50x

And there are eight cores in Prop 1, one in the M0+

Also, which model did you compile to? CMM or LMM? Did you enable fcache? Those other results would be quite interesting.

SRLM wrote: »

Interesting. From this link, ARM has the following scores:

CM   CM/MHz
M4  510  3.4
M3  82   3.3
M0+ 111  2.3
M0  105  2.2

So, the Propeller (with this test) comes in at 1/50th the performance of an M0+.

brucee

Are you the target audience?

No, I am am really not, to me the P* was never really cost or performance competitive as I develop products that range into the consumer space, though some get used in only small volume.

Let me as background say that I value what Parallax has done in the education space, as it is important that we get the next generation of engineers going in this country. We should not import all of them. I've seen the wonder of a new student getting a motor to move or an LED to flash, and say "wow I did that". A much more rewarding experience for them than just hitting buttons in a video game. And yes in that market performance is not an issue. Probably why there are still some BASIC stamps out there, though they are wildly obsolete. My concern is that the assumption that the P2 will cure all ills and "if you build it they will come" to the tune of millions of units a year. The RPi has achieved almost a million units a year, pretty remarkable, but then I doubt anyone made any money from it, and that was not the designers intention. As both the P1 and P2 push the video capabilities it seems like that is the space they want to occupy. Problem is there is next to no margin in the RPi or BB and they get crossover into the low-moderate volume space because their price/performance is outstanding. RPi coremarks at 1300, BB probably faster, at best the P2 will an order of magnitude slower, even counting the multi-cores, even worse for large programs that have to go off chip to an SDRAM.

As for the 50x, that in per MHz is wrong and my fault as the first numbers I posted as I was running out the door, and updated later. But 50x was the aggregate performance against a PixyCam board I was using, which coremarked around 600 using an LPC4330. ( not counting the dual cores).

I will post the coremark code and project here later, though you can download it from the eembc website.

SRLM

Bill Henning wrote: »

Huh?

105/8.5 = 12.35294

NOT 50x

And there are eight cores in Prop 1, one in the M0+

Also, which model did you compile to? CMM or LMM? Did you enable fcache? Those other results would be quite interesting.

I'm comparing CM/MHz, since that's a more fair assessment:

2.3/0.043 = 53.48.

Of course, the numbers were edited last night to 0.43, so that's

2.3/0.43 = 5.3

Edit: but really, I'd like to see the code. It will be fun to try and optimize it for the Propeller.

brucee

Attached is a zip file of the coremark project for SimpleIDE (it is renamed .c so it can be attached, so rename it .zip and hopefully it will unzip)

After pulling out some debug info it runs about 10 coremarks/sec. Not knowing much about all the different options, it is just running using defaults for an Activity board

Still missing is setting up a timer function so it can report the result, rather than having to time it by hand.

Dave Hein

I get jibberish when I open the coremark.c file. Is this file encrypted? I think the codemark code requires a license to use, but I may be wrong on that. Their website indicates that you must be a member of EEMBC or have a license to use it. I don't know if this is a free membership/license or whether you have to pay to use it.

SRLM

@Dave Hein: You have to rename it to .zip.

brucee

I had to delete the file as it is copyrighted, sorry I was under the impression that they were freely readable, as someone told me that files were published so that you can verify the results. But I suppose you have to be a licensee of coremark to get a copy. EEMBC does offer university licenses which are probably free. Any professors out there want to jump through those hoops, so I can send you a copy.

jazzed

brucee wrote: »

Attached is a zip file of the coremark project for SimpleIDE (it is renamed .c so it can be attached, so rename it .zip and hopefully it will unzip)

After pulling out some debug info it runs about 10 coremarks/sec. Not knowing much about all the different options, it is just running using defaults for an Activity board

Still missing is setting up a timer function so it can report the result, rather than having to time it by hand.

The SimpleIDE zip project feature should actually work.

I'm not reposting a functional package because of Copyright/IP concerns.

Let me suggest however that you use Compiler Simple Printf instead of Linker Tiny Lib.

CMM doesn't crash, it just takes forever ;-)

Dave Hein

SRLM wrote: »

@Dave Hein: You have to rename it to .zip.

Oh, why didn't you just keep the name coremark.zip instead of renaming it to coremark.c. The forum software does allow attaching zip files. In fact, it usually does weird things to the names of C file, like truncating the name to 2 characters. I suppose I should delete the file from my download directory.

Heater.

No worries, the stupid coremark thing is available here: ....

Rarely have I heard of such a stupid benchmark or idiotic licensing for a benchmark.

Edit: Link deleted as I don't want Parallax to get embroiled in the EEMBC scam.

Dave Hein

Yes, it is a rather "interesting" licensing method. The source file says that you must discontinue use of the file if you did not receive the accompanying CoreMark license. I wonder what the actual license looks like.

jazzed

Dave Hein wrote: »

I wonder what the actual license looks like.

Just like any other exclusive license, it will enslave you, your children, and your grandchildren until governments, corporations, and lawyers disappear entirely or decide that intellectual property laws are not in our best interests.

rod1963

Yeah it looks like EEMBC primarily caters to industry and academia so they have the typical corporate mentality. Hobbyists and free software types go elsewhere.

As far as the CoreMark benchmark is concerned even the big dogs quote it for their processors. Hate it if you want but it seems to be a sales and bragging point for the big boys and their products. I suspect when the PII arrives people are going to be asking what's it's CoreMark given it's widespread use in industry.

Sure the P1 didn't do so good that was expected.

The PII should be a different story and any benchmark will show it.

Heater.

The license is in the source code. That is to say, there is no license unless you pay up.
I have no idea who these idiots are or who actually pays for this nonsense. For sure they should not be operating under a .org domain name. Read the gibberish here: http://www.eembc.org/techlit/coremark-whitepaper.pdf

User Name

brucee wrote: »

My concern is that the assumption that the P2 will cure all ills and "if you build it they will come" to the tune of millions of units a year.

1) Who is assuming this? Or is this your latest strawman to knock down?
2) Are you being paid to be concerned? If not, why not give it a rest?
3) Are you Chip or Ken's wife in disguise?

Seriously, your posts are such a downer. Are you convinced you're doing Parallax a favor? Why in the world would Parallax want to build an ARM look-alike and try to compete head-to-head with NXP, ST, TI, etc? Except for that, I am unable to ferret out any alternative strategy in what you post.