Viewport DDE or DLL data or .NET object

henry99 · 2009-07-01 18:01

Is it possible to get the data to stream out of Viewport via DDE or a DLL call or .Net object?

The best would be a .NET object which can be imported into Matlab or C# or VB easily.

I would imagine the following format would be ideal:
Object.GetVariableNames -> List of Strings
Object.GetVariableValue("VariableName") -> Double Array
Object.SetVariableValue("VariableName") -> Double Array
Object.Connect(speed)
Object.Disconnect(speed)

Then data sharing can be much easier to implement.

I could write this myself but why bother if someone better than me like Hanna can do it quickly? [noparse]:)[/noparse]

Also how does Viewport handle arrays?

henry99 · 2009-07-01 18:07

Even sweeter would to add the following events:

Object.DataChangedEvent

with the following support properties:

Object.ChangedEvent("VariableName",Threshold)
Object.ChangedEventEnable("VariableName")
Object.ChangedEventDisable("VariableName")

That would be the bomb because then I could use drop the object into my C# program and write a few callbacks when the data changed. Otherwise I guess I could just poll it in a timer .net component but the response would not be as good.

CounterRotatingProps · 2009-07-01 20:17

henry,

do you think callbacks would always be faster? I mean flag/semaphore then burst might be fast too.

Just curious, even though you'd like this in viewport, have you messed around with propellent.dll ?

Hanno,

if you happen to read this, I'd like to second henry's very good suggestions - this would open up some very interesting instrumentation and applications independent of viewport addons.·

cheers,
Howard

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

Post Edited (CounterRotatingProps) : 7/1/2009 8:23:12 PM GMT

henry99 · 2009-07-01 20:38

In C# .NET, callbacks are easier to program.

If you check a flag, you'd need to setup a timer object with an expiry and that would bog things down.

I'm not that familiar with semaphores.

I think callbacks are much more intuitive also. For example, when the data changes past the threshold, the callback is triggered and I do some processing. Like if the voltage changes or the encoder value changes,

The fastest solution would probably be DMA like on the NI PCI cards but I hardly use it because of the added complexity. The key is to make it as easy as possible. A limited selection of methods and variables to get data in and out is better than 100 functions people will never use.

The code will be easy to write, you have one cog that sends out serial at the max speed and you can use the serial .net object from the .NET library to translate it to callbacks/update variables etc.

Again I'm sort of lazy and would rather pay someone $100 for an off the shelf product than spend time designing it myself.

Currently, I often have my matlab or c# program read in and parse serial data. However, this is definitely not as nice as a packaged .Net component that takes care of all those details that I take short cuts around.

henry99 · 2009-07-01 20:41

Also if someone is willing to write the propeller code, I can write the .NET component if we decide on a protocol.

It'd be very nice to add some basic data compression to get the fastest data stream over the USB as possible.

I'd be willing to donate my time to the open source project if someone was willing to write the prop ASM (which I haven't learned yet as I just use spin).

Hanno · 2009-07-01 20:43

Hi,
Great inspiration Howard and excellent architecture Henry. This makes lots of sense and will make it into ViewPort soon- exactly as Henry has specified. I just have to finish up something that is so exciting I stayed up till 3am last night- will be out soon and rock your world!
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

henry99 · 2009-07-01 20:49

Excellent - thanks Hanno [noparse]:)[/noparse]

Once I have a fast way to get data into Matlab or C#, there will be a lot of cool applications for sure! I would totally throw away most of my NI data cards.

The matlab image processing functions are much more advanced. Also you can write CUDA applications for faster processing.

You can buy an Asus EEE 900 netbook with Windows and Atom 1.6ghz processor for $150 on sale as the PC!

Hanno: I hope it comes as an easy to use .Net component like the .Net Serial component!

henry99 · 2009-07-01 20:59

Just a suggestion, I suggest the following changes to my original idea:

Make the event:

Object.DataChangedEvent

with the following support properties:

Object.ChangedEvent("VariableName",Threshold)
where Threshold>0 means the event is enabled, otherwise it is disabled (no need for enable/disable methods)

Object.ChangedVariables(List of Strings with names of variables that have changed so that you don't need to read all the variables in)

Also modify or overload:

Object.GetVariableValue("VariableName") -> Double Array + Long Integer Timestamp (always good to know what time the sample was at)

CounterRotatingProps · 2009-07-01 21:06

Oh cool !

Yes, do it as a .NET serial component by all means --- they're easy across all the languages.

> CUDA

Very interesting, henry, CUDA in many ways is like the opposite end of the spectrum of dev. platforms on micros. (Hard to imagine a Propeller running more than 8 "threads" much less 12,000!) I like CUDA a lot --- if I wasn't stuck on a boring set of projects right now, I'd be codin' cuda ;0

cheers
-Howard

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

henry99 · 2009-07-01 21:14

CUDA is sweet especially on image processing when you have lots of parallel process data. My Quadro FX has 240 cores and 4gb of very fast memory!

I tried Jacket from Accelereyes for matlab and you only need very minimal changes.

Most of my data acquistion needs are pretty simple and it's much easier to write advanced algorithms in matlab than prop asm so can't wait for the .NET component [noparse]:)[/noparse]

henry99 · 2009-07-01 21:19

Also not sure if Viewport does on the fly data compression, thresholding and buffering currently but all 3 will increase the throughput dramatically.

Hanno · 2009-07-01 21:22

Thanks Henry for refining the api- I'll get to it in a couple days- so feel free to continue improving it, better now than when I'm finished [noparse]:)[/noparse]
Yes, CUDA is awesome- and already built into a prototype of a ViewPort feature. See my google tech talk for details- basically I integrated the Nvidia PhysX physics simulation package with ViewPort. The idea is that people can model their robots and environments within the simulated world, controlled by spin running on the propeller, all managed by ViewPort. It's like a "dreamworld"- the spin program gets sensor readings from the simulated world and controls actuators that act on the simulated world. Then, when everything works, you just replace the simulated sensors/actuators with real ones and you're robot is set! It's working- just have to get time to productize it..
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

CounterRotatingProps · 2009-07-01 21:26

Viewport can do 2Mbps with the right setup.

I think someone also wrote a OCX wrapper (yuk!) for the propellent dll, but I've not tried it. (Because it would probably hang.)

[noparse][[/noparse]EDIT - Hanno, you're a genius ! ]

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔

henry99 · 2009-07-01 21:42

The FTDI chip does 3mbps I think. USB 2 is supposed to support 12mbps I think. So 2mbps isn't that fast.

Honestly I don't know what I'd be streaming that much data besides video or audio (which is much easier using a usb webcam or audio card).

Pairing with those low cost netbooks is great. For 150 you get a tiny notebook with a battery to power the prop from the usb port and can do all the processing.

I've also used the the Intel D945GCLF2D mini-ITX Atom 2x1.6Ghz to great success which is < $100.

Hanno · 2009-07-01 22:02

Yes, the FTDI chip does 3mbps. It's pretty straightforward to bit bang from the Propeller at 2 or 3mbps. What's hard is making sure that the data arrives through all the buffers and protocols in a windows environment. ViewPort's 2mbps transfer rate let's it quickly update the display for the instruments, measure many variables at high time resolution, stream vision processed video from the Propeller, and transfer the entire 32kb of memory at every whenever you step with the debugger.
Yes, low cost netbooks are awesome with the Propeller- particularly with ViewPort.
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

henry99 · 2009-07-01 22:20

Those usb cameras can run at 640*480*30 fps at 8bit greyscale=9mbps. Sustained data transfer at 12mbps doesn't seem like a problem for my computer when downloading from the internet and file copies from HD to HD probably are on the order of 1000mbps without any data loss.

Maybe it would require special purpose drivers/code but it certainly seems possible without any special PC mods.

Data packets should be time stamped with the counter clk count from the propeller then transferred asynch to the PC.

Hanno · 2009-07-01 22:48

Completely agree with you that sending a couple mbps to a pc should be non-trivial. However, poor support for fast serial communication in windows, buggy ftdi drivers, and lack of hardware handshaking makes it a big pain. It's much easier to get two Propeller chips to talk to each other- there it's easy to understand what's going on.
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

Hanno · 2009-07-12 21:46

I'm making progress on building a DDE server into ViewPort. As Henry and Howard discussed above, this will make it much easier for other applications to interact with the Propeller.

As a teaser, to continually get the current value of a spin variable called "temperature" into your excel spreadsheet you just need to type this into a cell:
"=viewport|getvalue!temperature"

Isn't that beautiful?
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

Mike Huselton · 2009-07-12 23:22

Hanno,

You da man!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
JMH

henry99 · 2009-07-18 02:32

Hi Hanno,
Thanks for your updates.

Sorry for the delay as I've been traveling.

DDE might work ok but I think the .NET component is still the way to go.

Since the FTDI chip just emulates a serial port and the serial output commands are already easily communicated in C#, VB, Matlab, etc, making the interface easy to use for programmers is def the way to go.

Hanno: let me know if you need any clarifications for my suggestions

Hanno · 2009-07-18 03:27

I've done some testing with my DDE server and it easily passes even the 24kByte video buffer to DDE clients without any problems. So, even though DDE is ancient, I believe it's the best communication tool for this application. It's just so easy to integrate into things like Excel! I'll provide a number of interfaces to retrieve the various arrays and variables that can be shared as well as metadata.
If you really want to program in .net, check out ViewPort's developer's kit: http://mydancebot.com/viewport/dkhelp.php
Once the DDE communication is set up, it should be easy to use that to integrate into pretty much anything.
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

Hanno · 2009-07-18 03:34

One more thing... Yes, the FTDI chip provides a serial interface. And yes, doing very basic communication at slow datarates isn't too bad with all the baggage that programming on modern OS entails. Supporting high datarates with all of ViewPort's functionality was very difficult and took me a long time. Here some buzzwords: packets, markers, escape sequence, variable packetsize, interleaved data, markers, magic bytes, full-duplex....

A dde client by contrast is trivial. To get the live value of a spin variable into excel just type this into a cell:
"=viewport|getvalue!temperature"
Do this for whatever variables you want to calculate/graph!
Coming soon....
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

henry99 · 2009-07-18 07:40

DDE may be trivial in Excel but it's a pain in something modern like C#, VB or matlab when .net is available. Maybe you can make some C#, VB and Matlab examples available when you have the chance. Then again, if a .Net component were supplied, no such example would be necessary as it'd be self explanatory.

I'm sure quite a bit of work went into Viewport, I'm not doubting that and that's why I didn't want to tackle writing a .Net component myself.

However, again a drop in .Net component makes it much better to use and if one isn't available, I can just use the drop in microsoft .Net serial port and use standard serial comm to send data back and forth.

Hanno, maybe you would consider writing the .net component and selling it as a different product when you have the chance. Certainly you already worked out the communication protocol and data transfer so that's the hard part.

Hanno · 2009-07-18 20:12

Henry,
Working on it!
Next release will have a built-in DDE server, a sample Excel spreadsheet which shows how to do engine performance tuning, a VB.net sample client, and a C# sample project.
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

Mike Huselton · 2009-07-18 23:31

Hey, Hanno! It's been too long...

DDE is just fine for data transfers - Dynamic Data Exchange is, after all, what we are talking about.
C# or VB wrappers add little, if any, utility. Your idea to post some example code is all that the average person needs.

Great Job! Say hey to the folks...

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
JMH

henry99 · 2009-07-19 04:11

IMHO I disagree with James. A .net component makes programming much easier over DDE.

I used to know how to use DDE in C++ but I have no idea how to do it in now and I don't think it's event driven so you need to parse individually DDE messages.

Often times you want to update the display when the variable changes.

How are you going to know when the variable changes? In C# with a .Net control, you just write the callback function.

In DDE I'm pretty sure you need poll for a DDE event or write a special DDE callback and parse it then stick all your code into the one massive callback.

Polling works fine in single thread programs but not for windows.

Without callback support, I'd just stick with the included free microsoft .Net serial component.

Maybe I should write it myself [noparse]:)[/noparse]

Hanno · 2009-07-19 19:28

Thanks jmh and henry for the good discussion. I'll do my best to make everyone happy. The C#/VB samples will show how to program using events- making it easy to know when things change.
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

Hanno · 2009-07-23 22:56

I've integrated a DDE server and client into ViewPort that supports the following commands:

dde.init("vp","system"): connects to ViewPort DDE server

dde.execute("connect"): connects to Propeller

dde.execute("disconnect"): disconnects Propeller- releases com port

dde.request("list"): returns a list of variables whose values can be read/written

dde.request("detail(var)"): returns information about a variable, for example,
   detail("io") returns "io|array[noparse][[/noparse]400]|base 16"

dde.poke("var",value): sets a Propeller variable to a value, for example, poke(servo,1500) 
   sets the servo variable to a value of 1500.

dde.execute("link(var,ddecommand)"): links a Propeller variable to a DDE variable, for 
   example, link(servo,excel|sheet1.xls!r1c1) links the servo variable on the Propeller to the 
   contents of the top, left cell of sheet1.xls in Excel.  When you change that cell in Excel, the 
   servo variable will change on your Propeller- potentially moving a servo. 

dde.request(varlist): returns a list of comma separated values for the list of variables you
    specified.  For example, vp|get|temp_wind_time can return "20,15,8:05"

dde.StartAdvise(varlist): your dde client will be advised 10 times/second with a list of comma 
   separated values for the list of variables you specified.  For example, a varlist of 
   vp|get|temp_wind_time will result in events with a parameter of "20,15,8:05" 

dde.StartAdvise(trigger): sets a trigger.  When the condition is valid, the trigger result is set
   to the values of the variables in the varlist.  For example, StartAdvise("rise!temp_20_temp_time") 
   will set up a new trigger that fires when the Propeller variable "temp" rises above 20.  The 
   application will get notified with a comma-delimmeted string of values- for example, "21,90,8:05".

Variables that you can read/write/advise/trigger include the video and io arrays, as well as the configuration strings. I've included support for both synchronous read/write, as well as the "advise" event based callback feature for read/write. Here's some Visual Basic code that works for Excel macros- I'll post a C#.net example later.

Dim a, b, channel As Integer
Sub StartVP()
    'Start DDE connection to ViewPort
        channel = DDEInitiate("vp", "system")
    'Connect ViewPort to Propeller
        DDEExecute channel, "connect"
    'Set cell to list of names of Propeller variables
        ActiveSheet.Cells(5, 5).Value = DDERequest(channel, "list")
    'Set cell to detail of variable "a"
        ActiveSheet.Cells(3, 2).Value = DDERequest(channel, "detail(a)")
    'Set servo variable to value of cell
        DDEPoke channel, "servo", Sheets("Sheet1").Range("A1")
    'Link servo variable to cell- when the cell changes, the variable on the Propeller will be updated
        DDEExecute channel, "link(servo,excel|sheet1.xls!r1c1)]"
    'Read temp variable into cell 
        ActiveSheet.Cells(4, 2).Value = DDERequest(channel, "temp")
    'Continually update cell with value of "temp" variable from Propeller
        ActiveSheet.Cells(6, 2).Value = "=vp|get!temp"
        ActiveWorkbook.SetLinkOnData "vp|get!temp", "gotData"
    'Start a trigger that fires when "temp" rises above 20.  Then, return temp and time
        ActiveSheet.Cells(7, 2).Value = "=vp|rise!temp_20_temp_time"
        ActiveWorkbook.SetLinkOnData "vp|rise!temp_20_temp_time", "gotTrigger"
End Sub
Sub gotData()
    'Data received, update cell
        ActiveSheet.Cells(4, 5).Value = a
        a = a + 1
End Sub
Sub gotTrigger()
    'Trigger received, update cell
        ActiveSheet.Cells(5, 5).Value = b
        b = b + 1
End Sub
Sub StopVP()
    'Stop getting trigger/value updates
        ActiveSheet.Cells(6, 2).Value = ""
        ActiveSheet.Cells(7, 2).Value = ""
    'Disconnect Propeller and terminate DDE
        DDEExecute channel, "disconnect"
        DDETerminate channel
End Sub

Am I missing anything?
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

henry99 · 2009-07-23 23:03

Hi Hanno,
Looks good!

I'd be very interested to see the C#/Matlab example. Can you increase that refresh rate to more than 10Hz reliably? Matlab in particular will be cool to do a lot of hardcore math easily and send back to the prop.

Also, I saw a demo of your Viewport controlling a DC motor + optical encoder using a PID loop. What motor driver did you use for it? I'm trying to find something that is as much plug and play as possible.

Hanno · 2009-07-23 23:14

I'll make the refresh rate configurable- I tested a 24,000 byte array at 10hz with no visible cpu usage- so shouldn't be a problem. I'll post a full C# sample project soon, Matlab should be easy, but would take me a while to learn- maybe you can give it a shot?
I use the LMD18200, beware that the minimum voltage is 12V, otherwise it's perfect.
Hanno

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Download a free trial of ViewPort- the premier visual debugger for the Propeller
Includes full debugger, simulated instruments, fuzzy logic, and OpenCV for computer vision. Now a Parallax Product!

TreeLab · 2009-07-24 21:15

Hi Hanno;
This looks terrific, and will serve our needs perfectly. Is the data in the longer arrays/vectors buffered? If a request for data is received while VP is reading data from the Conduit, will it receive a 'pure' snapshot of the data as recieved in one transfer from the prop, or will it be an overlay of some old/some new results?

Do you sleep?

Cheers!
Paul Rowntree

PS : WHen would you expect to post this version?

Cheers!

Viewport DDE or DLL data or .NET object

Comments