OBEX examples with locks

frank freedman

Can anyone recommend a few good examples of OBEX items using Locks?

Thanks

FF

Cluso99

I have never found the need to use locks.
Are you sure you need them Frank?

Phil Pilgrim (PhiPi)

Frank, you might look at some of the I2C drivers. For multiple devices on one bus and multiple instances of the driver object (say, one for each device), locks would be mandatory.

-Phil

frank freedman

Cluso99,

Actually I wanted to be able to set off a timing generator and enable only one pass per request, have n ADCs controlled by the timing generator collecting data and storing in cog at PASM speed, have each write their data to either a shared buffer or separate buffer for transfer out to another cog or external process. Currently I have gotten this working using I/O lines but after going through the book sanandak has up on spin/pasm/c, I thought I would give this a try. Maybe save an I/O pin or two. He shows a good example of this, but I have to look harder at what he is doing on the PASM side.

Not need, just want to understand how to use them at the bit level in PASM. An inquiring mind wants to know (at least while the gray matter can still gain more than it looses......)

Thanks,
FF

DavidZemon

I have an example in C++ of thread-safe printer class: https://github.com/parallaxinc/PropWare/blob/develop/PropWare/hmi/output/synchronousprinter.h

frank freedman

Thanks y'all!

ChrisGadd

Not in the obex, but here's a snippet from an object that reads ASCII strings from an external device and copies it to hub memory when complete, where code in another cog processes it.

' Spin methods=================
PUB setLock                                             '' Set lock to prohibit updates to text string
  repeat until not lockset(lock)

PUB clrLock                                             '' Clear lock to allow updates to text string
  lockclr(lock)

'PASM snippet==========================
:lock_loop              lockset   lock_id                     wc                ' Loop until shared rx_array is locked by this routine
          if_c          jmp       #:lock_loop                                   '  prevents rx_array from being accessed by recorder until all bytes updated
                        mov       loop_counter,#13                              ' Prepare to copy 13 bytes from internal array to shared rx_array
                        movd      :copy_loop,#int_array                       
                        mov       ptr,rx_address
:copy_loop              wrbyte    0-0,ptr                                       ' Copy a register from the internal array to the shared array
                        add       :copy_loop,d1                                 ' Address next internal register
                        add       ptr,#1                                        ' Address next shared register
                        djnz      loop_counter,#:copy_loop                      ' Repeat for all registers
                        lockclr   lock_id                                       ' Unlock shared registers

Seairth

Just because some of this was not immediately obvious to me, I figured I'd share a few notes about the locks:

• there are actually two 8-bit fields internally. One is used with LOCKNEW/LOCKRET and the other is used with LOCKSET/LOCKCLR.
• The LOCKNEW/LOCKRET bit field is designed for you to "reserve" a lock ID/index. In other words, when you call LOCKNEW, you get back the index of the first bit in the field that is zero (which is also implicitly set to one to indicate that the ID has been reserved). LOCKRET simply clears that bit to return the ID to the reservation pool.
• The LOCKSET/LOCKCLR bit field is what we primarily think of as the LOCK bits. You call LOCKSET to set a bit (and get back the prior state of the bit) and LOCKCLR to clear the bit (and get back the prior state of the bit).
• The two bit fields are completely separate from one another. In other words, you don't have to use LOCKNEW/LOCKRET in order to use LOCKSET/LOCKCLR (or vice versa, for that matter).
• If you have complete control over your code and want to statically assign lock IDs, then you only need LOCKSET/LOCKCLR. This can be convenient, but it's also more brittle.
• If you are writing a library to be used by others or are using other people's libraries, you should use LOCKNEW/LOCKRET to acquire the IDs you use with LOCKSET/LOCKCLR. This requires extra effort (you have to pass the IDs between cogs), but also more robust.
• Note that it is possible to mix the two usage patterns, but don't do it unless you have a good reason.

DavidZemon

Thanks Seairth! That's some useful info

Tracy Allen

You might find this thread interesting from a few years ago,
http://forums.parallax.com/discussion/109727/does-anyone-use-locks/p1

Following up on Phil's comment, one OBEX object that implements locks for the i2c buss is Kye's, I2C_ROMEngine. http://obex.parallax.com/object/23
It is specifically for eeprom. Multiple cogs can access the eeprom, mediated by locks to prevent collisions. A couple of things about it. All of the reads and writes are funneled through the readpage and writepage methods, and those commence by waiting for the assigned lock to be free, and finish by clearing it. The start method either uses lock that is passed in by the caller, or, if the entry parameter is -1, it checks out a newlock. Notice that both the pin numbers and also the lock number are stored in the DAT area, so that one instance of the object shares those, a system-wide resource. The Start method should only be called once, usually at the top level, and the object should be declared in every other object that needs to use the resource, whether or not they will be launched into a separate cog. These other objects should not call the Start method, but they have access to all the methods to read and write data due to the shared nature of DAT variables.

I use my own i2c object to sort out access to hardware devices with the same considerations, setlock before the start, and clearlock after the stop for each transaction, where several cogs have to hit the same i2c bus, a system resource with shared DAT pins and lock. Greatly simplifies the code and its development.

frank freedman

Thanks Tracy, that link was interesting, seems about evenly split in opinion pro/con. I think sanandak's analogy of the single shared track in his book provides a good argument to locks in certain situations. The main thing was the actual how to use them as example code for PASM seems pretty scarce. That was the reason for looking into examples.

(Is there any sort of catalog/index of the objects in OBEX?) Asking if anyone knew of good examples in OBEX was a way to circumvent having to look at every object to see if it used a lock and how. If there was a downloadable image of the OBEX not zipped or anything it could at least be searched. Is there any sort of catalog for this?

So, now to check the examples cited.

Thanks to all

DavidZemon

frank freedman wrote: »

(Is there any sort of catalog/index of the objects in OBEX?) Asking if anyone knew of good examples in OBEX was a way to circumvent having to look at every object to see if it used a lock and how. If there was a downloadable image of the OBEX not zipped or anything it could at least be searched. Is there any sort of catalog for this?

Been asked for many times. No such luck as of yet. You'd think, for all the times people have asked for such a thing, someone would have created an HTML scraper capable of downloading every object in the OBEX by now.

Dave Hein

Here's an HTML file that lists all the OBEX objects. Maybe someone could create a spreadsheet that contains keywords for all the objects.

DavidZemon

Nice! I'm making progress. Attached is a Python file and the resulting csv file that extracts the HTML table into a csv table. Next up is to parse each of the links and find the object download links.

DavidZemon

Done! I'm going to clean up the Python script a bit, add some argument parsing to it so that the input and output isn't hardcoded before uploading it to GitHub. In the meantime, here's the COMPLETE OBEX!

https://david.zemon.name/downloads/complete_obex.zip
The compressed zip is a little over 50 MB and the expand zip is a little over 60 MB. Not too shabby overall. Only took about 30 seconds to download the whole thing over my ~30 Mbps connection.

UPDATE:
Git repo is online https://github.com/DavidZemon/ObexDownloader

DavidZemon

Dave,

I haven't been able to reproduce your index file. I'd like to get this script running on a schedule from CI server so we have always-up-to-date OBEX downloads. How did you get the file?

Dave Hein

Starting at http://obex.parallax.com/projects I selected "- All -" for "projects per". I then tried saving the page to a file from my browser, but this didn't give the full links. The links referenced a local file, which could have been edited to reference the project page on the web. However, rather than editing the links I just did a copy and paste from the browser screen to a document editor. I removed a couple of unused columns and saved it to an HTML file.

MikeDYur

DavidZemon, I like the whole idea of this, I don't know how many times I have downloaded duplicate files on OBEX. I search by date, hoping I have already got the old stuff. Allways find a new direction to go, and find something I missed. Though this won't help me in particular, because I don't have a Linux machine online, but it sure will be a time saver for people that do.

Dave Hein

To answer the OP's question about lock, I found two entries in the OBEX that use locks. I did a "grep -r -i lockset" in the complete_obex direction and got matches in "Lock-Bit Demo" and "Wheel_Controller". However, my search didn't look in zip files where there are probably many more examples of using a lock. I know that the CLib object uses locks, and there are probably many more examples in zip files in the OBEX.

DavidZemon

Dave Hein wrote: »

Starting at http://obex.parallax.com/projects I selected "- All -" for "projects per". I then tried saving the page to a file from my browser, but this didn't give the full links. The links referenced a local file, which could have been edited to reference the project page on the web. However, rather than editing the links I just did a copy and paste from the browser screen to a document editor. I removed a couple of unused columns and saved it to an HTML file.

Okay, thanks. That helps. I'll see if I can make it work from that.

DavidZemon

Dave Hein wrote: »

To answer the OP's question about lock, I found two entries in the OBEX that use locks. I did a "grep -r -i lockset" in the complete_obex direction and got matches in "Lock-Bit Demo" and "Wheel_Controller". However, my search didn't look in zip files where there are probably many more examples of using a lock. I know that the CLib object uses locks, and there are probably many more examples in zip files in the OBEX.

That's a reasonable problem... I think I'll update the script to unzip all the archives and, before the script completes, compress the entire folder into a single archive. That should make the whole thing far more usable.

Ariba

I think there are not many LOCK usage entries in the OBEX because locks are normally used on the higher level application code, not on low level drivers.
E.g. a serial-object that is used to show text on a terminal from different cogs sending at the same time:
- With no locks: the serial data gets corrupted on bit level, the terminal receives no vaild data bytes.
- with locks on byte level: the communication works, but you get a wild mix of characters from all sending cogs.
- with locks on string level: you see a whole strings from one of the cogs on the terminal, but still not usable if you need several strings, or strings with decimal numbers for a message.
- with locks on CarriagReturn characters: shows full lines from one cog on the terminal, but not usable if you need several lines to show together.

What I want to say: On what level you need to lock the serial object to one of the cogs depends a lot on the application, and this can only be decided by the one that programs the application, and not by the driver object.
The Serial object should be locked to one cog as long as necessary, but as short as possible. The other cogs are normally blocked while they wait to get the lock.

Andy

DavidZemon

I've uploaded a new "complete_obex.zip" to the same link (https://david.zemon.name/downloads/complete_obex.zip). This time, every zip is extracted. Originally I was deleting the zips after extraction but I ended up leaving the zips in place so that files which get overwritten during extraction can still be retreived (I'm looking at you adns2620).

Dave Hein

I searched the unzipped files for lockset, and found 29 different OBEX projects that contained it. They are listed below. Note, I only show one file per project since some projects contain many occurrences of lockset. Unfortunately, grep doesn't handle 16-bit UNICODE, so the only files I found were in 8-bit ASCII. I know there are at least 2 other projects that use locks, and probably many more.

1Mbaud FullDuplexSerial (Fixed baud-rate)/asm_write_ex.spin: repeat while (lockset(lockId))
3-Axis CNC Control Package/SD-MMC_FATEngine.spin: repeat while(lockset(cardLockID - 1))
640 x 480 VGA Tile Map Driver w_ Mouse Cursor/VGA64_TMPEngine.spin: repeat while(lockset(lockNumber - 1))
74C922 Keypad Driver/74c922buffer.spin:repeat until not lockset(SemID) ' Lets lock the memory
74C92X Keypad Buffer and Driver/74c92Xbuffer.spin:repeat until not lockset(SemID) ' Lock it
Combo PS2 Keyboard and Mouse Driver/PS2_HIDEngine.spin: repeat while(lockset(keyboardLockNumber - 1))
DS1307 RTC Driver/DS1307_RTCEngine.spin: repeat while(lockset(lockNumber - 1))
FAT16_32 Full File System Driver/Full File System Driver with DS1302 RTC/DS1302_SD-MMC_FATEngine.spin: repeat while(lockset(cardLockID - 1))
Full Duplex Serial Port Driver/Full-Duplex_COMEngine.spin: while(lockset(lockNumber - 1))
Generic I2C EEPROM Driver/I2C_ROMEngine.spin: repeat while(lockset(lockNumber - 1))
ILI9325 320x240 TFT driver/touchSPI.spin:select lockset lock wc ' try to claim spi lock
IR Kit/ir_reader_nec.spin: repeat while lockset(lock)
KISS WAV Player Driver/SD-MMC_FATEngine.spin: repeat while(lockset(cardLockID - 1))
KISS WAV Recorder Driver/SD-MMC_FATEngine.spin: repeat while(lockset(cardLockID - 1))
Lock-Bit Demo/simple_multicore_demo3d.spin: repeat until lockset(LockID) == False 'Wait in this loop for lock to open
Nordic nRF2401 Rf Tranceiver Handler/TRF24G.spin: repeat until not lockset( rf_sem )
Octal Button Debouncer/D8C_BUTEngine.spin: repeat while(lockset(lockNumber - 1))
PROPSHELL/PROPSHELL-master/Full-Duplex_COMEngine.spin: while(lockset(lockNumber - 1))
Propeller Backpack TV Overlay/prop_backpack_tv_overlay2.spin: repeat while lockset(timelock)
Pulsadis detector/pulsadis dual processor for Obex/Full-Duplex_COMEngine.spin: while(lockset(lockNumber - 1))
SYSLOG - Multicog debug_log to SD_Serial/DS1307_RTCEngine.spin: repeat while(lockset(lockNumber - 1))
Servos and Encoders Calibration/SD-MMC_FATEngine.spin: repeat while(lockset(cardLockID - 1))
Settings/driver_socket.spin: repeat while NOT lockset(SocketLockid)
Switch Debounce/DebounceCog.c: lockset(SemID);
Trending Barometer/I2C_ROMEngine.spin: repeat while(lockset(lockNumber - 1))
Wheel_Controller/Wheel_Controller.spin: repeat until not lockset(mutex_id)
Wiimote IR blob tracking camera/wiicamera.spin: add clockset,clkpin_ 'Set up clock
mdb_RealTimeClock/MDB_RealTimeClock.spin: repeat until not LockSet(gLockId)
uSDPropLoader/DS1307_RTCEngine.spin: repeat while(lockset(lockNumber - 1))

MikeDYur

Cool, thanks for posting.

localroger

Locks are kind of a specialty thing. For most purposes the fact that Hub RAM writes of bytes/words/longs are atomic is good enough, if you do a write/read back/then act type of scenario. But in some cases there are a few pathological conditions that can arise because of Hub timing that might fool a cog into thinking it has secured access at the same time another has. Locks have the advantage that they take effect immediately, not waiting for the Hub to rotate around to that cog.

That said, I've written some pretty intricate PASM code and never felt the need for a lock.

Mark_T

I don't think I have anything in the Obex with locks but I've certainly used them for guarding ring-buffer insert operations. Can't recall if it was actually needed or just done as its the right thing on a multi-processor architecture.

Dave Hein

There are times when an atomic read-modify-write is necessary. The hub slots aren't sufficient to perform this function. The locks are the only thing that provide such a feature. I have used locks to ensure that only one cog has access to a serial port, and I have also used a lock so that multiple cogs can access an SD card file driver. Multiple cog access to these drivers would not work without some kind of locking mechanism.

Tracy Allen

David, thanks for the OBEX unzipped repository!

Interesting that a lot of those OBEX objects that use locks are Kye's. (Kwabena W. Agyeman). Those that contain the word "Engine". Sadly Kye hasn't been around here for over a year. I think he's still involved in computer vision and imaging. All of the OBEX examples show lockset in a SPIN context, none that I can see in PASM. It's not hard to see why that is so, in the context of objects that mix SPIN and PASM.

Tracy Allen

Blast from the past:
A highly off-book use of locks in PASM can be found it a puzzle from Kuroneko,
http://forums.parallax.com/discussion/129585/resolved-puzzle-connection-terminated
Locks are used to communicate a long from one PASM cog to another without using other HUB ram or a pin.
It's a good one to sort through for education.

sanandak

In our project we use locks in a couple of places. One is in a clock application - one cog is setting the time once a second (and calculating the new y/m/d h:m:s) and a lock is used to prevent other cogs from reading the time while those numbers are changing. Because 6 numbers are being set, we can't depend on the atomic nature of a single hub operation...

The attached is (shameless plug) from my book on leanpub

Propeller Programming

Sridhar