PropellerIDE 0.30.0 Released - projectview, propman, packthing, propterm, memorymap

Brett Weir · 2015-04-23 01:18

Hello all,

After months of work, I'm happy to announce that PropellerIDE 0.30.0 is now available for you to try.

[size=+1]Get PropellerIDE[/size]
[size=+1]What's in this release?[/size]

This new version includes a host of new features, including a brand new project view, memory map, even a brand new loader! So many new things are coming down the pipe that I'm sure there will be lots of things that don't work quite right, so download PropellerIDE and tell me the good and the bad so that the next release can be even better!

Please view the release notes to get the full scoop on what this release contains, as well as known issues you may encounter using it.

I can't wait to hear all of your feedback!

[size=+2]Important Notes[/size]

p1load.exe has been replaced with propman.exe, so if you have a previous installation, you will have to change the loader path to propman.exe.

[size=+1]Windows[/size]

The FTDI drivers must be installed separately (they kept getting flagged as viruses... coincidence?)
http://www.ftdichip.com/Drivers/VCP.htm

[size=+1]Debian[/size]

Qt 5.3 is required or above is required, which is first available in Ubuntu 14.10. For 14.04 or below, you will need to add the Utopic Unicorn repositories.

sudo echo "deb http://cz.archive.ubuntu.com/ubuntu utopic main" >> /etc/apt/sources.list
sudo apt-get update

Installing the binary is straight-forward, but you will have to force installation of its dependencies.

sudo dpkg -i propelleride-0.30.0-amd64.deb
sudo apt-get install -f

Raspberry Pi is a similar story. Get instructions on how to install Qt5 on Raspberry Pi here: https://github.com/parallaxinc/PropellerIDE#raspberry-pi---raspbian-wheezy

TrapperBob · 2015-04-23 04:14

Brett,

File=>Quit does not seem to work from the Menu (Windows 7 64-bit)

Brett Weir · 2015-04-23 05:02

Ah, bingo, just forgot to connect the signal. Checked in a fix. Thanks.

George Sutton · 2015-04-23 05:19

Thank you for your time and effort with this.

VonSzarvas · 2015-04-23 06:20

Well done Brett - a great milestone.

Especially I like this comment from the release notes: "This is a major step in increasing coverage of the Language definition format, the structural change that will allow PropBASIC to be supported as just another IDE language."

Happy coding :-)

dgately · 2015-04-23 09:23

Nice work, Brett!

One thing I noticed (running on Mac OS X 10.10.3)...

The Project View re-inits the display whenever I select an item such as a CON or PUB. For a large project that has many items, that require the list to scroll, that re-init causes the Project View to display from the top and I lose my place in the scrolled list of items (see images below)...

Here I've selected CON "MAX_ENCODERS" and want the source to scroll to that CON. But, want to keep the Project View list as it is:

attachment.php?attachmentid=113983&d=1429805513

attachment.php?attachmentid=113983&d=1429805513

After I've selected the CON "MAX_ENCODERS", the Propject View has scrolled (re-displayed) from the top:

attachment.php?attachmentid=113984&d=1429805520

attachment.php?attachmentid=113984&d=1429805520

I would like the project view list to stay scrolled where it was when I selected that CON...

dgately

dgately · 2015-04-23 10:13

And... a couple of other issues:

1. Project View states "(NOT FOUND)" for some objects that use "_" in their file names when it is clear that they DO exist (but not all)...

attachment.php?attachmentid=113985&d=1429807952

attachment.php?attachmentid=113985&d=1429807952

2. Nested objects sometimes do not display a list of their internals... In the image above note that FullDuplexSerialPlus.spin within the Project View area does NOT list any CON, PUB, PRI, etc. items. And, the image below shows that XBee_Object.spin does NOT list its CON, PUB, PRI, etc... items

attachment.php?attachmentid=113986&d=1429808266

attachment.php?attachmentid=113986&d=1429808266

3. Some files (UTF8, ASCII, not sure which ones) do NOT display properly, losing their carriage-returns or linefeeds. These files display all content as a single line. PropellerIDE should reformat these files, as SimpleIDE does. In the image below, note that this source file: "jm_ezmod_8x.spin" provided by none-other-than "JonnyMac" :-) displays as a single and very long line of code. The code does compile, but is difficult to view. In order to display this file properly, I had to copy the text out to another text editor, re-save it and re-open the file in PropellerIDE. I believe that the SimpleIDE source code includes methods to remedy this issue...

attachment.php?attachmentid=113987&d=1429808637

attachment.php?attachmentid=113987&d=1429808637

dgately

DavidZemon · 2015-04-23 10:40

I'm continuouly impressed by this. It almost makes me want to go back to Spin :P

Brett Weir · 2015-04-23 14:51

dgately wrote: »

Nice work, Brett!

One thing I noticed (running on Mac OS X 10.10.3)...

The Project View re-inits the display whenever I select an item such as a CON or PUB. For a large project that has many items, that require the list to scroll, that re-init causes the Project View to display from the top and I lose my place in the scrolled list of items (see images below)...

Here I've selected CON "MAX_ENCODERS" and want the source to scroll to that CON. But, want to keep the Project View list as it is:

After I've selected the CON "MAX_ENCODERS", the Propject View has scrolled (re-displayed) from the top:

I would like the project view list to stay scrolled where it was when I selected that CON...

dgately

Hi Dennis,

Thanks for the feedback! Yes, currently interaction with the projectview results in rebuilding the view, and with no mechanism to restore the old position in place, you'll lose it. I will be working on making this aspect more graceful for the next version. I'll add that to the list of known issues in the release notes.

Brett Weir · 2015-04-23 15:28

Let me go down these issues one by one.

dgately wrote: »

1. Project View states "(NOT FOUND)" for some objects that use "_" in their file names when it is clear that they DO exist (but not all)...

Maybe you can help me to understand what's going on in this picture. It looks like Quadrature_Encoder is including itself as a dependency. If so, that will lead to undefined behavior, and projectview is actually supposed to reject that with a big warning. But I am surprised that it is showing up without the underscore. That's peculiar. Let me know either way, but I will take a closer look at that one.

dgately wrote: »

2. Nested objects sometimes do not display a list of their internals... In the image above note that FullDuplexSerialPlus.spin within the Project View area does NOT list any CON, PUB, PRI, etc. items. And, the image below shows that XBee_Object.spin does NOT list its CON, PUB, PRI, etc... items

Do you think you can email me this project so I can test with it?

dgately wrote: »

3. Some files (UTF8, ASCII, not sure which ones) do NOT display properly, losing their carriage-returns or linefeeds. These files display all content as a single line. PropellerIDE should reformat these files, as SimpleIDE does. In the image below, note that this source file: "jm_ezmod_8x.spin" provided by none-other-than "JonnyMac" :-) displays as a single and very long line of code. The code does compile, but is difficult to view. In order to display this file properly, I had to copy the text out to another text editor, re-save it and re-open the file in PropellerIDE. I believe that the SimpleIDE source code includes methods to remedy this issue...

I can tell you that PropellerIDE currently has difficulty with CR-only line endings, but for whatever reason, that's what most Spin files are even though they aren't the native line ending on any modern operating system. =|

Can you open a terminal and run the "file" command on your project to confirm my suspicion?

As you said, the only real way to fix it is to reformat the files on opening, but this is a destructive operation and can have unexpected results, especially if a file has been edited by multiple users on multiple platforms and now has a potpourri of different line endings. I suppose I could try to convert all line endings to a "one true line ending", but then when you try to open a spin file in a different text editor, who knows what'll happen. I usually use something like dos2unix when it creates problems.

It really is a problem and I haven't found a great solution for it. =\

Brett Weir · 2015-04-23 16:01

SwimDude0614 wrote: »

I'm continuouly impressed by this. It almost makes me want to go back to Spin :P

Thanks. I've always been fond of Spin, and my hope is that I keeping making PropellerIDE better and better, more people will give Spin a try. Comments like yours make all of this worthwhile.

Brett Weir · 2015-04-23 16:02

George Sutton wrote: »

Thank you for your time and effort with this.

You're welcome. I'm happy to be a part of this project.

Brett Weir · 2015-04-23 16:04

VonSzarvas wrote: »

Well done Brett - a great milestone.

Especially I like this comment from the release notes: "This is a major step in increasing coverage of the Language definition format, the structural change that will allow PropBASIC to be supported as just another IDE language."

Happy coding :-)

Thanks! I'm pretty excited about that one. I've been working on the Language format for awhile now, and it's going to allow PropellerIDE to do a lot of things it couldn't before in the future. I'll be able to make a home for all the different Propeller languages that may not already have one.

dgately · 2015-04-23 16:43

Brett Weir wrote: »

Let me go down these issues one by one.

Maybe you can help me to understand what's going on in this picture. It looks like Quadrature_Encoder is including itself as a dependency. If so, that will lead to undefined behavior, and projectview is actually supposed to reject that with a big warning. But I am surprised that it is showing up without the underscore. That's peculiar. Let me know either way, but I will take a closer look at that one.

Actually, Quadrature_Encoder.spin does NOT include itself. It merely mentions itself as an included object, "Quadrature Encoder" in a comment. I've stripped out most of the source, but here's where it is mentioned in Quadrature_Encoder.spin:

{{
**************************
* FUNCTIONAL DESCRIPTION *
**************************
...
To use this object: 
...
Example Code:
           
OBJ
  Encoder : "Quadrature Encoder"
...
}}

Do you think you can email me this project so I can test with it?

Here it is: StingrayRCProject.zip

I can tell you that PropellerIDE currently has difficulty with CR-only line endings, but for whatever reason, that's what most Spin files are even though they aren't the native line ending on any modern operating system. =|

Can you open a terminal and run the "file" command on your project to confirm my suspicion?

Running "file" on any plain text type of document gives me an error, so I'm not sure what I "should" see, but I've included hex dumps of a 'bad' file and a 'repaired' file:

$ file jm_hfcp.spin 
jm_hfcp.spin: ERROR: line 163: regex error 17, (illegal byte sequence)

$ file FullDuplexSerial.spin
FullDuplexSerial.spin: ERROR: line 163: regex error 17, (illegal byte sequence)

Let's see a hexdump of a file that shows no CRs or LFs in PropellerIDE:

$ hexdump -C -n 400 jm_hd485.spin 
00000000  ff fe 27 00 27 00 20 00  3d 00 3d 00 3d 00 3d 00  |..'.'. .=.=.=.=.|
00000010  3d 00 3d 00 3d 00 3d 00  3d 00 3d 00 3d 00 3d 00  |=.=.=.=.=.=.=.=.|
... <== 0xA0 lines of the same thing...
000000c0  3d 00 3d 00 3d 00 3d 00  3d 00 0d 00 0a 00 27 00  |=.=.=.=.=.....'.|
000000d0  27 00 0d 00 0a 00 27 00  27 00 20 00 20 00 20 00  |'.....'.'. . . .|
000000e0  46 00 69 00 6c 00 65 00  2e 00 2e 00 2e 00 2e 00  |F.i.l.e.........|
000000f0  2e 00 2e 00 2e 00 20 00  6a 00 6d 00 5f 00 68 00  |...... .j.m._.h.|
00000100  64 00 34 00 38 00 35 00  2e 00 73 00 70 00 69 00  |d.4.8.5...s.p.i.|
00000110  6e 00 0d 00 0a 00 27 00  27 00 20 00 20 00 20 00  |n.....'.'. . . .|
00000120  50 00 75 00 72 00 70 00  6f 00 73 00 65 00 2e 00  |P.u.r.p.o.s.e...|
00000130  2e 00 2e 00 2e 00 20 00  48 00 61 00 6c 00 66 00  |...... .H.a.l.f.|
00000140  2d 00 64 00 75 00 70 00  6c 00 65 00 78 00 2c 00  |-.d.u.p.l.e.x.,.|
00000150  20 00 74 00 72 00 75 00  65 00 2d 00 6d 00 6f 00  | .t.r.u.e.-.m.o.|
00000160  64 00 65 00 20 00 73 00  65 00 72 00 69 00 61 00  |d.e. .s.e.r.i.a.|
00000170  6c 00 20 00 49 00 4f 00  20 00 66 00 6f 00 72 00  |l. .I.O. .f.o.r.|
00000180  20 00 45 00 49 00 41 00  2d 00 34 00 38 00 35 00  | .E.I.A.-.4.8.5.|
00000190

And, a file that was "repaired" by re-saving in TextEdi:

$ hexdump -C -n 400 jm_hfcp.spin 
00000000  27 27 20 3d 3d 3d 3d 3d  3d 3d 3d 3d 3d 3d 3d 3d  |'' =============|
00000010  3d 3d 3d 3d 3d 3d 3d 3d  3d 3d 3d 3d 3d 3d 3d 3d  |================|
... <== 0x50 lines of the same thing...
00000060  3d 3d 3d 3d 0a 27 27 0a  27 27 20 20 20 46 69 6c  |====.''.''   Fil|
00000070  65 2e 2e 2e 2e 2e 2e 2e  20 6a 6d 5f 68 66 63 70  |e....... jm_hfcp|
00000080  2e 73 70 69 6e 0a 27 27  20 20 20 50 75 72 70 6f  |.spin.''   Purpo|
00000090  73 65 2e 2e 2e 2e 20 48  75 6d 61 6e 20 46 72 69  |se.... Human Fri|
000000a0  65 6e 64 6c 79 20 43 6f  6e 74 72 6f 6c 20 50 72  |endly Control Pr|
000000b0  6f 74 6f 63 6f 6c 0a 27  27 20 20 20 41 75 74 68  |otocol.''   Auth|
000000c0  6f 72 2e 2e 2e 2e 2e 20  4a 6f 6e 20 22 4a 6f 6e  |or..... Jon "Jon|
000000d0  6e 79 4d 61 63 22 20 4d  63 50 68 61 6c 65 6e 20  |nyMac" McPhalen |
000000e0  28 61 6b 61 20 4a 6f 6e  20 57 69 6c 6c 69 61 6d  |(aka Jon William|
000000f0  73 29 0a 27 27 20 20 20  20 20 20 20 20 20 20 20  |s).''

dgately

jmg · 2015-04-23 16:48

Brett Weir wrote: »

Thanks! I'm pretty excited about that one. I've been working on the Language format for awhile now, and it's going to allow PropellerIDE to do a lot of things it couldn't before in the future. I'll be able to make a home for all the different Propeller languages that may not already have one.

Given Lazarus/FPC now works nicely on a Pi2, there is interest in adding a PropPASCAL to the family tree.
Most decent editors have Language highlighters, so this is an 'expected feature' on any IDE

Brett Weir · 2015-04-23 18:09

jmg wrote: »

Given Lazarus/FPC now works nicely on a Pi2, there is interest in adding a PropPASCAL to the family tree.
Most decent editors have Language highlighters, so this is an 'expected feature' on any IDE

That'd be pretty interesting to see. PropBASIC is to be the proving ground for this kind of support, but keep in touch and we can talk more about it once I have the proof of concept in place.

Brett Weir · 2015-04-23 18:27

dgately wrote: »
Actually, Quadrature_Encoder.spin does NOT include itself. It merely mentions itself as an included object, "Quadrature Encoder" in a comment. I've stripped out most of the source, but here's where it is mentioned in Quadrature_Encoder.spin:
{{
**************************
* FUNCTIONAL DESCRIPTION *
**************************
...
To use this object: 
...
Example Code:
           
OBJ
  Encoder : "Quadrature Encoder"
...
}}

Ahhh, okay, that's the problem then. My rule parser isn't smart enough to identify this kind of feature yet. There will definitely be limitations on how accurate the project parsing can be without writing a full-on parser for each language, but I haven't hit that limit yet. I'll keep this in mind when I get back to working on the Language engine. I can filter comments before detecting language features.

dgately wrote: »

Do you think you can email me this project so I can test with it?

Here it is: StingrayRCProject.zip

Thanks!

dgately wrote: »
Running "file" on any plain text type of document gives me an error, so I'm not sure what I "should" see, but I've included hex dumps of a 'bad' file and a 'repaired' file:
$ file jm_hfcp.spin 
jm_hfcp.spin: ERROR: line 163: regex error 17, (illegal byte sequence)

$ file FullDuplexSerial.spin
FullDuplexSerial.spin: ERROR: line 163: regex error 17, (illegal byte sequence)

That's really bizarre. OS X probably includes a less powerful version of file. Well, anyway, I ran file on your project and got this:

FullDuplexSerialPlus.spin: Little-endian UTF-16 Unicode text
HB-25.spin:                ASCII text
jm_hfcp.spin:              ASCII text
jm_strings.spin:           ASCII text, with CR line terminators
Quadrature_Encoder.spin:   UTF-8 Unicode text
stingrayRC.binary:         data
stingrayRC.spin:           UTF-8 Unicode text
XBee_Object_2.spin:        UTF-8 Unicode text
XBee_Object.spin:          Little-endian UTF-16 Unicode text, with CRLF line terminators

The file you reported isn't here, but jm_strings.spin also has CR line terminators and will show up the same wrong way. Try it out.

I gave some more thought to this issue and I was probably overthinking it earlier. There is probably a good solution to this so I'll put it on my to-do list.

dgately · 2015-04-23 19:26

Brett Weir wrote: »

The file you reported isn't here, but jm_strings.spin also has CR line terminators and will show up the same wrong way. Try it out.

I gave some more thought to this issue and I was probably overthinking it earlier. There is probably a good solution to this so I'll put it on my to-do list.

Thanks, Brett!

dgately

Brett Weir · 2015-04-23 19:35

Checked in a fix for the line ending thing, Dennis, if you want to make a build and test it out.

FredBlais · 2015-04-24 07:45

Changes made to the preferences (like changing a color in a theme) are not kept when the program re-open.
Maybe a small improvement, the status bar could be splitted in two and when we build, we could write build successful and the program size in the status bar. The pop-up go away pretty fast after building.
Could be nice to have close all or close all except this for the open tabs/files.
It would be also useful to be able to display included objects in the left pane and be able to open them from there to be able to modify them without having to seek the file in the filesystem.

Thanks

evanh · 2015-04-24 21:04

FredBlais wrote: »

Changes made to the preferences ...

What preferences! Speaking of which, I've not got any menu bar. Using Kubuntu 14.04, no apparent install nor runtime launch errors so I presume I've got the base QT5 libs already installed.

evanh · 2015-04-25 19:33

Learning a little strace+grep ... Wowwwwww, over 2500 open()'s on startup! ... and this is just a small app! I've never really done any GUI coding before but I'm starting to get where all the bloat comes from.

Okay, I've got round the first prefs problem, deleting the old prefs at ~/.config/Parallax/PropellerIDE.conf so it would recreate suitable new defaults has got the build button to work. I still don't have any menu's though.

evanh · 2015-04-25 20:17

Three issues with comms:

- When attempting to run/download a program to the Prop, it resets the Prop but no program is downloaded and no apparent error. When I do this at the command prompt, again the Prop is reset and, I get the following:

$ propman -d /dev/ttyUSB0 ESC_Demo.binary 
bool QSerialPort::setBaudRate(qint32, QSerialPort::Directions): device not open
Connecting to 'ttyUSB0'...    [ NOT FOUND ]

- When a debug terminal window is open I get the following error upon run/download:

bool QSerialPort::setBaudRate(qint32, QSerialPort::Directions): device not open
"Device or resource busy"

- Oddly, only the USB based comport shows up in PropellerIDE but both my ports are listed with propman:

$ propman -l
/dev/ttyUSB0
/dev/ttyS0

evanh · 2015-04-25 20:54

Font not settable in PropellerIDE.conf. There is a section I can hand edit for setting the font but it seems to be ignored at run time.

Mike Green · 2015-04-27 11:07

I just updated my MacOS (10.10.3) PropellerIDE to 0.30 from 0.25. I downloaded the .dmg file and replaced the existing app with the new one. I started it and found that I couldn't quit. It doesn't seem to respond to a Quit menu item click or the keyboard shortcut. It does quit if you close the main window.

Brett Weir · 2015-04-27 13:09

evanh wrote: »

What preferences! Speaking of which, I've not got any menu bar. Using Kubuntu 14.04, no apparent install nor runtime launch errors so I presume I've got the base QT5 libs already installed.

What the hello?? That's the strangest thing I've ever heard. I'll set up a Kubuntu installation later today and see if I can reproduce the issue you're having. In the meantime, press F5 and Preferences will pop up. >.<

Brett Weir · 2015-04-27 13:14

evanh wrote: »

Learning a little strace+grep ... Wowwwwww, over 2500 open()'s on startup! ... and this is just a small app! I've never really done any GUI coding before but I'm starting to get where all the bloat comes from.

Yeah, GUI stuff makes everything a lot bigger. There's lots of room for optimization in this particular program though. Some things are opened multiple times unnecessarily when doing things like building the project view because my focus has been on getting it to work at all first. As they say, "premature optimization is the root of all evil." =P

Brett Weir · 2015-04-27 14:23

evanh wrote: »
- When attempting to run/download a program to the Prop, it resets the Prop but no program is downloaded and no apparent error. When I do this at the command prompt, again the Prop is reset and, I get the following:
$ propman -d /dev/ttyUSB0 ESC_Demo.binary 
bool QSerialPort::setBaudRate(qint32, QSerialPort::Directions): device not open
Connecting to 'ttyUSB0'...    [ NOT FOUND ]

The issue here is because on older versions of Qt5, the baud rate can't be set before opening the port, even though the documentation says otherwise. I plan to change how this works in the future, but for now, upgrading to Qt5.3 or later should fix this and allow you to download software.

evanh wrote: »
- When a debug terminal window is open I get the following error upon run/download:
bool QSerialPort::setBaudRate(qint32, QSerialPort::Directions): device not open
"Device or resource busy"

That's because both the terminal and loader have exclusive access to the port while in use, so you have to have only one enabled at a time.

evanh wrote: »
- Oddly, only the USB based comport shows up in PropellerIDE but both my ports are listed with propman:
$ propman -l
/dev/ttyUSB0
/dev/ttyS0

Both PropellerIDE and PropellerManager have their own port enumerators as they're separate applications currently (more on this below).

* * * * *

Traditionally in Propeller dev tools, the serial terminal and loader are two applications, and usually separate from the main IDE, but this gives the IDE more responsibility in managing hardware, and on UNIX platforms, makes using the serial terminal less than practical. So to address this, I began writing PropellerManager, a new library for Propeller devices that will have many advantages.

It will no longer be necessary to disable the serial terminal to download new code, change the baud rate, or change the port of the current terminal window.
PropellerManager will be embedded into PropellerIDE as a library, which will allow PropellerIDE to work on mobile platforms.
PropellerIDE will be able to identify devices, download code, and use the terminal of both serial and Wi-Fi targets without needing to know the difference between them.

PropellerManager is still in development and there is much work left before I will be able to reach these goals. So for now, the PropellerManager library is being used by propman, a command-line replacement for p1load. It won't be until the next release that things start getting more interesting.

Brett Weir · 2015-04-27 14:25

evanh wrote: »

Font not settable in PropellerIDE.conf. There is a section I can hand edit for setting the font but it seems to be ignored at run time.

Ah, yes, this is broken for now. Custom fonts are overwritten by theme settings currently.

Brett Weir · 2015-04-27 14:25

Mike Green wrote: »

I just updated my MacOS (10.10.3) PropellerIDE to 0.30 from 0.25. I downloaded the .dmg file and replaced the existing app with the new one. I started it and found that I couldn't quit. It doesn't seem to respond to a Quit menu item click or the keyboard shortcut. It does quit if you close the main window.

Ah, yes, that was another little flub on my part. The Quit signal is just not connected but I already checked in the fix. Thanks.

evanh · 2015-04-27 15:06

Brett Weir wrote: »

What the hello?? That's the strangest thing I've ever heard. I'll set up a Kubuntu installation later today and see if I can reproduce the issue you're having. In the meantime, press F5 and Preferences will pop up. >.<

Nice replies. Very informative, thanks. I'm looking forward to release 0.40.

PS: I'm wary of integrating with Ubuntu 14.10 packages so I'm going to leave it for now.