Suggested standard for LOCK's
Bill Henning
Posts: 6,445
Hi,
I suggest the following standard for using LOCKs to manage access to critical resources.
The reason for specifying lock numbers is to avoid having to negotiate which lock is used for which function in every driver and application.
Lock 0: XMM memory access lock (to allow multiple clients to one XMM solution)
Lock 1: SPI multiplexer access lock (to allow multiple clients to share a multiplexed SPI bus)
Lock 2: Filesystem lock (to allow multiple clients to access a file system)
Lock 3: Network lock (to allow multiple clients to share a network layer)
Lock 4-7: user application locks
The reasoning behind this allocation is:
Lock 0: XMM is the most basic extended resource, and parallel solutions are the highest bandwidth producer/consumer in expanded Prop configurations. I've been using this lock on Morpheus for years, sharing access between refreshing the display and updating the display quite nicely.
Lock 1: SPI multiplexers are here to stay. PropCade and C3 both use them, as do many future boards. The precise multiplexer scheme may vary between boards, but all of them need to manage access to multiple SPI peripherals by potentially multiple client cogs. This will allow safe sharing.
Lock 2: Filesystem lock - Kye's nice new file system driver allows multiple simultaneous open files, so will other future file systems.
Lock 3: Networking lock - for future use
It is important to statically allocate locks so that all the code running on Props can easily use them. Failing static allocation, a "locks.spin" could be used to export "lock#xmm lock#spi lock#fs lock#net" - but I personally (in this case) strongly prefer the static allocation of locks for these purposes.
I suggest the following standard for using LOCKs to manage access to critical resources.
The reason for specifying lock numbers is to avoid having to negotiate which lock is used for which function in every driver and application.
Lock 0: XMM memory access lock (to allow multiple clients to one XMM solution)
Lock 1: SPI multiplexer access lock (to allow multiple clients to share a multiplexed SPI bus)
Lock 2: Filesystem lock (to allow multiple clients to access a file system)
Lock 3: Network lock (to allow multiple clients to share a network layer)
Lock 4-7: user application locks
The reasoning behind this allocation is:
Lock 0: XMM is the most basic extended resource, and parallel solutions are the highest bandwidth producer/consumer in expanded Prop configurations. I've been using this lock on Morpheus for years, sharing access between refreshing the display and updating the display quite nicely.
Lock 1: SPI multiplexers are here to stay. PropCade and C3 both use them, as do many future boards. The precise multiplexer scheme may vary between boards, but all of them need to manage access to multiple SPI peripherals by potentially multiple client cogs. This will allow safe sharing.
Lock 2: Filesystem lock - Kye's nice new file system driver allows multiple simultaneous open files, so will other future file systems.
Lock 3: Networking lock - for future use
It is important to statically allocate locks so that all the code running on Props can easily use them. Failing static allocation, a "locks.spin" could be used to export "lock#xmm lock#spi lock#fs lock#net" - but I personally (in this case) strongly prefer the static allocation of locks for these purposes.
Comments
It is also MUCH easier to document for new users... ie
"Use lock 0 to get exclusive access to external memory, and lock 1 to get exclusive access to SPI devices"
than
"if you want to get exclusive access to memory, your main program must do memlock := newlock, then you must pass the memlock variable to every client cog that needs to access xmm. To pass memlock, you have to write it into a parameter block, whose handle you pass to the client". (same for SPI, wrong newlock syntax, I know)
-Phil
I have a tremendous amount of respect for your knowledge and skills, and am very appreciative of all the help you give on the forums - but
I respectfully disagree with your double quoting "advantages" - those advantages are quite real due to the limited resources of a 32KB prop, "big system" and "computer science" desired features are often at loggerheads with saving memory.
I am well aware that modern CS education is focused on readability and flexibility, but I am afraid that has lost sight of embedded systems based on small microcontrolers; also for those new to the propeller it is significantly easier to teach and show examples using fixed lock numbering for the most basic locking.
I totally agree that in an ideal world, with less limited resources, dynamic allocation is the way to go. I just don't think the Prop is in that world.
The good news is that people who want to use dynamic allocation can of course do so :-) ... and those that want fixed locks can do so as well.
-Phil
Main code:
OBJ client1 : "some_client_cog" client2 : "some_client_cog" client3 : "some_client_cog" client4 : "some_client_cog" VAR long xmm_lock, spi_lock, fs_lock, net_lock PUB main ' error checking code omitted for clarity xmm_lock := locknew spi_lock := locknew fs_lock := locknew net_lock := locknew client1.start(@xmm_lock {, other parameters}) client2.start(@xmm_lock {, other parameters}) client3.start(@xmm_lock {, other parameters}) client4.start(@xmm_lock {, other parameters}) '-------------------------------------------- Now the clients need code for picking up all four locks, storing them etcAt a guess, 10-20 longs overhead per client, so 40-80 longs / 160-320 bytes for the app: 0.5%-1% of the props memory just to be dynamic
Notes:
- a little could be saved by packing the lock id's four per long, but that's also against readability, and still needs unpacking in client
- a more readable "client1.start(@xmm_lock,@spi_lock {, other parameters} would use even more memory
- I have several applications that share an xmm lock with six cogs that need arbitrated xmm access
How is it going to pass it to multiple clients without storing it first?
Here is one of my applications that shares the xmm lock:
2x vga_reader - must be started twice, on two cogs, with some parameters different
1x gpu_cog - started once, different object/cog
2x sprite_engine - started twice, needs some different parameters launches worker cogs based on parameters
That's five cogs that need access to the same lock -> lock number must be stored in the main code, or be static
- vga_reader cogs refresh scan line buffers from the page being displayed
- gpu_cog draws onto either displayed or next to be displayed page
- sprite engine draws onto the next to be displayed page
Another application I am working on has:
2x vga reader cogs
1x gpu cog
2+ XLMM cogs
Again, all need access to the xmm lock
PUB start ifnot (mylock) mylock := locknew + 1 ... other start stuff... return mylock - 1 DAT mylock byte 0The host program can then store the driver's lock in a byte variable for later use.
Better still, higher-level methods in the driver itself can do the locking and unlocking, without the host having to know the lock number.
-Phil
-Phil
I needed to do it this way to get the flexibility and speed for the external frame buffer access and GPU / sprite access for XMM, and for arbitrary SPI device support for SPI.
I realize this is not the approach for everyone; and to be compatible with locknew, I will locknew lock 0 & 1 for my purposes, and let other client apps use locknew as usual.
The danger you (or someone who uses your objects) might encounter with a static assignment is having locks 0 and 1 preempted by another object that starts ahead of your own, using locknew.
-Phil
While conceptually nice, a lock assigner wastes a cog.
I agree about the danger; which can be averted by using locknew in the main code to reserve locks 0&1 (which I will start doing)
I respect people not wanting to statically allocate, so I will locknew 0&1 at the start of my main object - this way I can "statically" use them, yet be compatible with everyone else.
I know that I am not using it in the nicest CS way; however I am willing to sacrifice some niceness for speed and memory savings in this low level case.
The purpose of this thread was to avoid that danger, which as a result of this discussion is now taken care of, by my using my main code to pre-allocate lock 0 & 1 for my xmm and spi bus hardware locks.
The problem with the static lock number is then the user applications have to handle the case where locknew returns one of the defined lock values.
This is the approach I will be taking in the next rev of my internal apps.
-Phil
This leads me to a question that is OT for this thread, but maybe with some guidance on an appropriate title I could start a new thread...
The question is, is there an "accepted" format for creating objects and drivers that are NOT dependent on running from SPIN? It seems that with the push for C and other non-Spin languages on the prop that objects and drivers will be needed that do not have a dependency on SPIN.
I know I've seen some threads where Bill has discussed comms via mailboxes, just wondering if there is anything formalized.
C.W.
I would like to see a stand method for doing mailboxes. This would make it easier to develop code that could interwork with other drivers. Another issue that concerns me is that 8 locks may not be enough for some applications. There should be a way to extend the number of locks by implementing software locks. Perhaps one of the hardward locks could be used to add additional software locks.
-Phil
Agreed, and I intend to pre-allocate 0&1 in my startup code for my evil hardware purposes
CW:
I "standardized" on a four-long mailbox, with the first long being the command (sometimes encoding some data), second being a destination pointer, third source pointer, fourth a count - however for some drivers I use the longs differently... and sometimes I use more than 4 longs. Cavet Emptor.
Dave:
Phil's followup is a great way of getting additional locks for resources that are not extremely time critical - one "multiplex lock" could easily handle many individual slower locks.
Phil:
The master lock / soft lock approach is a very good one, and works well.
There would be a problem running a driver that requires lock number 0 under SpinSim. SpinSim implements an implicit locknew when it performs a boot, and the lock number is saved in a memory-mapped mailbox location. This lock is used for the conio and fileio system services. Anything running under SpinSim would have a problem if it required lock number 0. Since SpinSim doesn't support pin I/O (except for serial on P31 and P30) it's doubtful that it would be used to run hardware drivers. However, I have tested some memory caching code on SpinSim.
-Phil
Yes, this is what Catalina does in the multithreaded version of the kernel. Since you can have hundreds of C threads executing on each of the 8 cogs, I (potentially) needed many more than the 8 physical locks. So I added the ability to create arbitrary sized "pools" of virtual locks. Each virtual lock in the pool works much like a physical lock. Access to each pool is managed by one physical lock. Seems to work quite well (although perhaps the efficiency could be improved a little).
Ross.