Concurrency is not parallelism

potatoheadpotatohead Posts: 10,070
edited 2017-04-29 - 18:49:49 in General Discussion


We have had this discussion before. This talk is illuminating.

Both the P1 and P2 are concurrent multiprocessor. I've stuck with this term, due to the subtle connotation associated with concurrency, as in multiple, different things happening at the same time, which Propellers excel at. And multiprocessor, which can express parallel execution. Combination of good ideas.

Anyway, enjoy. Great talk.

Comments

  • Link to paper referenced in the talk.

    https://www.cs.cmu.edu/~crary/819-f09/Hoare78.pdf
  • And related, classic papers:

    https://www.cs.cmu.edu/~crary/819-f09/



  • ercoerco Posts: 19,736
    Interesting.

    Is it Rob's contention that Parallax should change its name to Concurrenax?
  • I have to disagree... the Propeller is parallel. The difference I see is that software is only ever concurrent. Hardware is either parallel or non-parallel, never concurrent. Concurrent is a way to describe the architecture of a software application. Parallel describes whether or not concurrent software runs sequentially or not.
  • Yes, it is parallel, and concurrent.

    What I found interesting, in this talk, is the distinction between concurrent and parallel.

    Parallel things are similar, scaled in terms of processing. Concurrent is more than thing going on at the same time.

    A prop excels at both.

    It can do a bunch of things concurrently, and do them in fairly discrete ways. Soft peripherals are one example. Being a multiprocessor, with the COG isolation makes doing this easy and robust.

    Parallelism is there too, processing sprites or multiple calculations is an example here.

    Many devices that are multi core do these things too, but they lack the compartmentalization found on the Prop. And a parallel system, like say GPU shaders, lack concurrency, as they are generally lacking in ways that make it practical.

    I think it's notable how concurrency on a Propeller works well, and it's not emphasized to the degree it's parallel nature us, that's all.

    Setting all that aside, I also found the talk interesting in how the ideas in the GO language take one to a solution that can scale by design. Neat!

    Make me wonder on P2 someday. Get the basics for channels and workers in there and a mini "go" kind of language could prove educational and useful.

  • potatoheadpotatohead Posts: 10,070
    edited 2017-04-30 - 20:39:58
    Parallel describes whether or not concurrent software runs sequentially or not.

    Isn't there a bit more to that? In the examples given, a concurrent thing can run non sequentially, as in each task having it's own capacity to perform independent of the others.

    The differentiator appears to be whether the tasks are the same or not.

    If they are the same, it's parallel. If they are different, it's concurrent.

    A lot of the things people get done with a Prop demonstrate strong concurrency, but little parallelism.

  • potatohead wrote: »
    Parallel describes whether or not concurrent software runs sequentially or not.

    Isn't there a bit more to that? In the examples given, a concurrent thing can run non sequentially, as in each task having it's own capacity to perform independent of the others.

    The differentiator appears to be whether the tasks are the same or not.

    If they are the same, it's parallel. If they are different, it's concurrent.

    A lot of the things people get done with a Prop demonstrate strong concurrency, but little parallelism.

    The examples in the video above all fit your description, but I don't recall him saying that parallel things have to be the same. And from my reading elsewhere, I do not recall that either.
    potatohead wrote: »
    What I found interesting, in this talk, is the distinction between concurrent and parallel.

    Parallel things are similar, scaled in terms of processing. Concurrent is more than thing going on at the same time.

    I do reasonably clearly remember him saying the opposite about concurrency: that concurrency does not require multiple things happening simultaneously. His example of a single-core processor handling mouse, keyboard, and other tasks concurrently. The operating system is written in a concurrent fashion, such that these tasks are independent. But while running on a single-core machine, they will not execute at the same time. However, because they have been designed for concurrency, they may run at the same time given parallel hardware.
  • potatoheadpotatohead Posts: 10,070
    edited 2017-05-01 - 01:48:53
    It is defined by inference.

    All examples, demonstrate concurrency. They run on a single, or multi processor. True! Many use the word parallel here, and I've long felt that was too general. SGI actually made this same distinction in the context of it's multiprocessor systems.

    Once running and developed, parallel is used to denote more instances of that system running on other compute nodes. It's a cloning at this point, again like GPU shaders. The speaker referenced scaling frequently here.

    The core implication being, develop for concurrency, and that which you develop can be parallelized, though individual parts of it may not be.



  • I think "parallel" could also apply to pipelined computations, wherein each processor receives data from one before it and passes along its results to the next one in the stream. In this case, each processor is doing something different but contributing to a joint result in parallel fashion.

    -Phil
  • I think "parallel" could also apply to pipelined computations, wherein each processor receives data from one before it and passes along its results to the next one in the stream. In this case, each processor is doing something different but contributing to a joint result in parallel fashion.

    -Phil

    Yes, pipelining is an interesting point, as is Intel's Hyperthreading. The idea of parallel changes as you get a larger magnifying glass.
  • I agree on pipelining. Hyperthreading isn't quite so clear. It's about filling unused cycles more than parallel processing. More like a scheduler than it is parallel like pipelining is.

  • potatohead wrote: »
    I agree on pipelining. Hyperthreading isn't quite so clear. It's about filling unused cycles more than parallel processing. More like a scheduler than it is parallel like pipelining is.

    Good point. I'll concede that one :)
Sign In or Register to comment.