Shop Learn
Propeller GUI touchscreen and full color display - Page 2 — Parallax Forums

Propeller GUI touchscreen and full color display

2456724

Comments

  • average joeaverage joe Posts: 795
    edited 2012-01-30 20:14
    I'm still in the design phase of my project and extremely overwhelmed by the sheer number of decisions. I'm finishing up the audio hardware for my project right now and trying to imagine some sort of user interface. I THINK I would like to use:

    2 - 3.2" touchscreens
    4 - 2x20 or 2x40, maybe 4x40? parallel lcds
    16 - Rotary encoders with push buttons.
    maybe a few other buttons and rotary encoders? This might be overkill, but I like flat menu hierarchy and expect to have a large number of parameters to adjust. My thought is to use one prop for gui and another for control of audio board.

    Dr_Acula wrote: »
    At the simplest level you can just connect propeller pins to the display without latches and work in 16 bit mode.
    Have you experimented with connecting prop pins directly to the display? and if so, what are you're thoughts?
  • cavelambcavelamb Posts: 689
    edited 2012-01-30 23:08
    That's pretty slick looking, Dr. A.
    I was trying to explain my blinky LED to my girlfriend last night.
    She looked at it blink, then looked at her smart phone.
    And shrugged....
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-01-30 23:55
    Have you experimented with connecting prop pins directly to the display? and if so, what are you're thoughts?

    Not on a breadboard but I know it will work. It will be much faster too.
    I was trying to explain my blinky LED to my girlfriend last night.
    She looked at it blink, then looked at her smart phone.
    And shrugged....

    Yeah, my kids do that too. Mind you, they have to fake some sort of interest, because I pay their phone bills!

    Just for kicks, I have been looking at C++ code to translate .jpg into .raw files. Add that to some C++ code out there for internet browsers to translate html code which comes in well under the 1meg of ram we have, and I believe it is not crazy to start thinking about a propeller based internet browser.

    It will need a C engine that can (behind the scenes) do caching and handle moving blocks of code in and out of ram and in between that, also handle blocks of graphic memory in and out of the same ram. Not hard and that should fit into one cog.
  • cavelambcavelamb Posts: 689
    edited 2012-01-31 00:35
    Dr_Acula wrote: »
    Not on a breadboard but I know it will work. It will be much faster too.



    Yeah, my kids do that too. Mind you, they have to fake some sort of interest, because I pay their phone bills!

    Just for kicks, I have been looking at C++ code to translate .jpg into .raw files. Add that to some C++ code out there for internet browsers to translate html code which comes in well under the 1meg of ram we have, and I believe it is not crazy to start thinking about a propeller based internet browser.

    It will need a C engine that can (behind the scenes) do caching and handle moving blocks of code in and out of ram and in between that, also handle blocks of graphic memory in and out of the same ram. Not hard and that should fit into one cog.

    That just blows my mind... Somebody starts talking about doing something way over the top, something that I normally associate with a full blown computer,
    then he adds something like, "Not hard and that should fit into one cog"...

    Tilt....
  • Heater.Heater. Posts: 21,233
    edited 2012-01-31 00:49
    Cavelamb,
    If we are talking large C programs that can run a Prop we are talking external memory and an XMM kernel to fetch and execute instructions from it. That kernel can be very small and simple and easily fit into one COG.
    The actual code and data is in external memory which currently goes up to 32MB on the Prop with SDRAM. I belive basic HTML renderer can easily be fit in 32MB. There are many small machines out there with small memory and web browsers.
    So, not so mind blowing. Catch is that it will be pretty slow on a Prop.

    Edit: What I meant to say was "A web browser on the Prop is clearly impossible". Just to be sure that it gets done:)
  • average joeaverage joe Posts: 795
    edited 2012-01-31 01:21
    cavelamb wrote: »
    That's pretty slick looking, Dr. A.
    I was trying to explain my blinky LED to my girlfriend last night.
    She looked at it blink, then looked at her smart phone.
    And shrugged....

    My wife does that to me all the time. lol
    Dr_Acula wrote: »
    Not on a breadboard but I know it will work. It will be much faster too.
    That's what I was hoping to hear. I'll have to start working on that right away. Now to wait the 20 something days it's going to take the display to get here.


    Dr_Acula wrote: »
    Not on a breadboard but I know it will work. It will be much faster too.
    That's what I was hoping to hear. I was thinking of working on an lcd touch GUI running on a dedicated propeller. I'll have to start working on that right away. Now to wait the 20 something days it's going to take the display to get here. After reviewing datasheets, I'm thinking I could multiplex the serial lines for the sd card and the touchpad ADC. Data in, Data out, Clock =3 pins, 2 pins for chip selects, that's 5 pins for sd and touch. The lcd uses 16 data pins, RS, WR, RD for a total of 19 display pins. Using a total of 24 pins, plus 2 for prop eeprom and 2 for propPlug, leaves 4 pins for communicating to system bus? Or would it be better to multiplex the 16 lcd data pins with the system bus and use the 4 lines for addressing? I don't think I'd need the busy line. I still don't understand penirq. I would like to use two pins for sigma-delta ADC, maybe one for DAC. 1 line is not sufficient for a high speed bi-directional bus IMO. I don't expect to be sending more than a few parameters at a time from prop to prop. Just bulk dumps when loading or editing a patch? Maybe a 138 to decode CS signals? And that's just the hardware end of it.
    I am trying to get my head around the software end of it as well. I want a high performance display, and would like to use no more than 5 cogs. 1 cog for pasm lcd driver, 1 cog for touch, 1 cog to interface with second prop, and the rest for video generation. I would like to display audio waveform captured from sigma ADC *in 1 cog* and display it as a choice of waveform, spectrum, or possibly other format. I think this could be done in a couple cogs? These 3 cogs need not be dedicated. Or something like that.
    Any hints on where to get started?
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-01 03:48
    Got a keyboard working!

    Works best with a stylus - it is a little small for fingers though a keyboard in landscape mode will be larger.

    All things being equal, I think it works more reliably than my Pandapad Adroid tablet's keyboard.

    Most keys working - just need to work on Backspace and !@#$%<>,.?, and then return a string to the calling program.

    Now we need scrolling textboxes...

    @averagejoe
    1) You can save one pin on the display as you never need "read". So that is 18 pins for the display.
    2) Yes 5 pins to share SD card and touchscreen. These never are used together, as you touch the screen, it then goes away very busy and reads files etc, and only when files/pictures/icons/programs/fonts are loaded is it ready for the touchscreen again.
    3) penirq is not needed. It detects when you touch the screen, but (Rayman I think) worked out that if the touchscreen is not being touched it returns invalid numbers equivalent to touching a point way off the screen, so you just keep polling the touchscreen looking for valid numbers that must be on the screen somewhere.
    4) You might be able to save some cogs. There is no real time video generation as the display has its own internal ram. Send out the data then forget about it. The touchscreen does not need a cog as it is running at "human input" speeds so Spin is plenty fast enough.

    At the moment I'm buffering the fonts into an external ram, but if you only have one font loaded at any one time and it is a small one (12 point like on the photo below) then that could easily be stored in hub ram.

    Things like the background photo can go straight from the SD card out to the display.

    Also my code attached has a TV attached because I needed this for debugging, but enough is working on this display that this is not needed any more. So that object can go and save some memory. Plus lots of the SD card object and the string object are not used and BST could be used to omit the unused code.
    640 x 480 - 51K
  • idbruceidbruce Posts: 6,173
    edited 2012-02-01 03:57
    WOW! You are on a very creative frenzy! Don't stop! Go Doctor! Go! :)
  • idbruceidbruce Posts: 6,173
    edited 2012-02-01 04:08
    Hey Doc

    Have you ever hacked one of the Franklin products, such as the ones found on their web site (http://www.franklin.com/)? I was wondering how difficult it would be to hack one of these.

    Bruce
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-01 05:05
    Thanks idbruce. I've been wanting to get the keyboard working for some time.

    I need to think about a font library - need lots of different sizes, plus landscape and portrait, and maybe different colors as well.

    re the Franklin products, could we hack them? Could we think about emulating them??

    I'm bursting with ideas now. Partly I'm building this because my Android has a beautiful display but it doesn't have enough smarts to flash a led on and off. But the propeller now has the display, and it can also interact with the real world. How about a series of simple circuits you draw on the screen and wire up as software and then you touch the screen and a real led turns on and off? Or an analog voltage, a real analog meter, and a picture of an analog meter on the screen.

    This is stuff that can be done easily on a laptop of course, but on the prop it can be done smaller, faster booting, cheaper and using much less power.

    Too many ideas!

    Just for fun, I've got some real LED starburst displays on my desk that I never got round to wiring up. And some 7x5 LED matrix ones. They need lots of multiplexing and other support circuitry. They make me feel guilty that I haven't got them working

    But a display can be a font http://www.dafont.com/theme.php?cat=302 and a font can be any color or size you want. Now we can emulate displays in software! Ha - emulate a $14 20x4 LCD display on a $17 touchscreen *grin*
    640 x 480 - 54K
  • BitsBits Posts: 414
    edited 2012-02-01 05:42
    What type of display is it? I did not see it mentioned in the thread. Nice work btw.
  • WBA ConsultingWBA Consulting Posts: 2,930
    edited 2012-02-01 09:12
    Wow, that keyboard is truly amazing! I second IDBruce, go, go, go! So, if I understand correctly, I can create a bitmap of the screen I want, display it, then simply use the touchscreen data to modify parts of the screen, to have a full fledged GUI?!?!?!? I so wish I had time to get the hardware and follow this thread in real time.
  • idbruceidbruce Posts: 6,173
    edited 2012-02-01 12:57
    @Andrew

    It is not very often that people agree with me :) So I will take a moment and bask in the glory :)

    And just as you said,
    I so wish I had time to get the hardware and follow this thread in real time.
    I second that emotion

    Bruce
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-01 13:50
    Thanks everyone for all the encouragement!

    @bits
    What type of display is it? I did not see it mentioned in the thread.

    There are a couple of links on post #28 of this thread. $16.80 with free shipping.

    @WBA
    So, if I understand correctly, I can create a bitmap of the screen I want, display it, then simply use the touchscreen data to modify parts of the screen, to have a full fledged GUI?!?!?!?

    Yes that is correct. For instance, the keyboard started off life as a photo I got off the internet, but then I modified it in paintshop, moved some keys around, made the text display longer etc. Then worked out the size of the keys in pixels and used this to create hotspots in the touchscreen.

    The touchscreen output is 0-100% in x and y direction. So for the QWERTY row, it ended up being between 60 and 69%.

    Decode this to an address value. The first row was numbers 0-9 so for the second row, the address value is 10-19 as there are 10 keys.

    The cool thing about spin is that all this takes longer to describe than the actual code.
            if (yval => 60) and (yval =< 69)
              address := 10 + (xval / 10)                   ' address = 10 to 19    QWERTYUIOP
    
    and then use a lookup table to get the ascii value
    ascii := lookupz(address: "1","2","3","4","5","6","7","8","9","0","Q","W","E","R","T","Y","U","I","O","P","A","S","D","F","G","H","J","K","L","Z","X","C","V","B","N","M"," ")
    
    and then check to see if CapsLock is off and if so, change all the letters to lower case
              if (capslock <> 0) and (ascii => 65) and (ascii =< 90) 
                ascii += 32                                 ' lower case
    


    Some things are very easy to do. Draw a picture of a switch on the screen. Each time you touch that area of the screen, redraw the switch as either open or closed. Then toggle a propeller pin.
  • idbruceidbruce Posts: 6,173
    edited 2012-02-01 15:07
    Some things are very easy to do. Draw a picture of a switch on the screen. Each time you touch that area of the screen, redraw the switch as either open or closed. Then toggle a propeller pin.

    LabVIEW from National Instruments springs to mind
  • sevssevs Posts: 50
    edited 2012-02-03 06:18
    Sorry if this is a stupid question Dr_Acula, but would you expect a significant improvement if you wrote your screen driver in PASM? Or is this not really what is limiting things? For example, in your video where you are showing the images you can see the screen scroll down.

    This really is a fantastic piece of gear you have developed. I havnt been able to do any micro work for probably close to 12 months, and back then when i was playing with the prop a bit i never thought something like this was possible. Just before i stopped getting micro play time i was just starting to learn PASM, so im no expert, but if it could help i could try and get the brain back into gear and have a think over it. The prop finally having parallax backed C dev is even more reason for me to play. I have been wanting to learn C. This chip continues to blow my mind with what it can do. I really am excited for the next iteration.

    Kudos again.
  • Heater.Heater. Posts: 21,233
    edited 2012-02-03 08:09
    Sevs,
    ...thought something like this was impossible.
    Ha..this is Propeller land. Just when you have that thought someone is busy making it happen!
  • idbruceidbruce Posts: 6,173
    edited 2012-02-03 11:43
    Ha..this is Propeller land. Just when you have that thought someone is busy making it happen!

    Yea, but it took a doctor to conquer it! :)
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-05 14:13
    This little demonstration shows text in portrait mode, then rotated 90 degrees for landscape mode. The icons and background can also be rotated.
  • average joeaverage joe Posts: 795
    edited 2012-02-08 17:08
    Dr.
    I just got my 3.2" touchscreen today and still haven't etched a board. Looking at your board and source has got me wondering. I didn't connect a reset line and I'm wondering how that will affect initialization.

    I believe I have worked out a workaround for font addressing. I plan on using a 2 or 4 gig sd card for storage. I have also read about replacing the eeprom with up to a 64M? I am not sure how yet but the main idea is to set aside some space on the sd card for a RAW partition. This raw file contains *?*aligned pixel data for the font. Each pixel is 1 bit, on or off. Only one color text can be drawn at a time, but that text could be any color or drawn anywhere on the screen. A lookup table is stored on eeprom to convert asci characters to a start and end address for the pixel data. This data is then read out to corresponding hub ram. Another cog or two composite text on top of static drawn background. Or something like this.

    Off to etch a circuit board!! Keep up the awesome work!
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-08 17:18
    Sounds good. You might need that reset line.

    Re font addressing, that could save some storage space. The very first font driver I wrote worked like that. You will need to read out a byte, or a long. A long will convert to 16 pixels, and you could write a pasm routine that takes one long and then sends out the 32 bytes (2 per pixel) to the display. That will save a lot of storage space.

    There are 6 cogs free so lots of spare cogs.

    Start simple and work up. The hardest part is getting just one little pixel to display on the screen.

    I'm still tweaking the font format file. I *think* it is stable, but just now I found a bug where a value of -1 got converted to 255 so I have gone back and put all values in the table in the same format as a propeller long, ie 4 bytes, LSB first, 0xFFFFFFFF means -1.

    Let me know when you get a pixel on the screen and we can work on fonts.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-08 17:29
    Step by step guide to building a GUI. This is a little calculator program.

    1) Find a picture of a calculator on the internet
    2) Modify it using paintshop. Remove buttons that are not needed, and resize it to fit in the 320x240 display
    3) Select a suitable font for the display. I chose a 7 segment style font. Convert using BMfont and then a vb.net program
    4) Make the background of the font the same as the display.
    5) Put the .raw picture and the .ifn font file onto the SD card.
    6) Write some code. The code below uses existing routines to run the touchscreen and the display, so this is just the additional code needed.
    PUB Calculator | xval,yval,address,ascii,operand
        FullScreenRaw(string("calcpic.raw"))                                      ' draw the calculator
        ILIloadfont(string("DSdig.ifn"))
        str.builtstring(true)       ' clear the working string 
        repeat
          yval := TouchValue >> 8                               ' decode yval 0-100%               
          xval := TouchValue & 255                              ' decode xval 0-100%
          if (xval <> 255)  and (yval <> 255)                    ' valid keypress
            address :=0
            case (yval*32)/10                                    ' convert 0-100 to 0-320 as easier to work with pixels off the picture
              60..101: address :=0                               ' row of a 4x6 matrix
              104..140: address :=4
              146..183: address :=8
              187..222: address :=12
              228..265: address :=16
              271..307: address :=20
            case (xval*24)/10                                   ' column convert percent to 0-239
              0..66: address += 0
              67..119: address +=1
              120..172: address +=2
              173..239: address +=3
            ascii := lookupz(address: "X","Q","Q","Q","C",8,"/","*","7","8","9","-","4","5","6","+","1","2","3","=","0","0",".","=")
            case ascii
              "X"             : return                                              ' close the calculator
              "C"             : str.builtstring(true)                               ' clear the current string
                                CalcClear
              "0".."9",8      : str.buildstring(ascii)                              ' build the string. Can also handle backspace
                                CalcClear                            
                                ILIprintstring(str.builtstring(false))              ' reprint the string each new char in case a backspace 
              "+","-","*","/" : str.copy(str.builtstring(true),@lineoftext1)        ' move the string to lineoftext1 and reset
                                operand := ascii
              "=" :             xval := str.decimaltointeger(@lineoftext1)          ' first operand. recycle xval and yval
                                yval := str.decimaltointeger(str.builtstring(true)) ' second operand
                                case operand
                                      "+" : address := xval + yval                  ' add
                                      "-" : address := xval - yval                  ' subtract
                                      "*" : address := xval * yval                  ' multiply
                                      "/" : address := xval / yval                  ' integer division
                                str.str(@lineoftext2,address)                       ' convert answer to string
                                str.trimstring(@lineoftext2) ' if negative, leave the -, if positive remove the leading space
                                CalcClear                                           ' clear the display ready to print the answer
                                ILIprintstring(@lineoftext2)                        ' print the answer
            delay.pause1ms(250)' only display once so delay half a second to "debounce"
    
    PRI CalcClear
         ClearFontBackground(25,21,220,48)       ' clear an area the same color as the font background        
         ILISetcursor(25,15) 
    

    I can post the full source of all programs if required.
    240 x 320 - 15K
    760 x 504 - 97K
    468 x 268 - 48K
    842 x 622 - 107K
  • average joeaverage joe Posts: 795
    edited 2012-02-08 17:34
    I have a long way to go, but should be able to manage. I'm not used to working with graphics so I expect some learning curve there. I have a spare pin, so that will be used for reset. I am trying to keep addressing functions simple to keep data transfers as quick as possible.
    Dr_Acula wrote: »
    The very first font driver I wrote worked like that.
    Is that the first font driver in the post? I skimmed the source in Backup 15 1 12. Lots off gems in there.

    I'm using the TFT320QVT, looks similar to yours in the datasheet and I'm not sure of supply voltages. LED should go to 5v, can't figure out acceptable current. Vcc should go to 3.3v rail? It has a 3v precision regulator.
    http://cgi.ebay.com/ws/eBayISAPI.dll?ViewItem&item=200475566068
  • TylerSkylerTylerSkyler Posts: 72
    edited 2012-02-10 09:36
    This thing is getting more impressive by the day! I picked up one of those screens from ebay, for 18 bucks why not? I would like to know if you could post details to the way you wired your screen to the prop and what pins are needed and not needed and such. This would be great(The EgoChina specs don't give great detail)!

    Thanks,
    Tyler
  • WBA ConsultingWBA Consulting Posts: 2,930
    edited 2012-02-10 12:57
    Dr_Acula wrote: »
    Step by step guide to building a GUI.

    Great, now I have no excuse not to get one and start messing with. As I said before, EXCELLENT WORK! Keep it going.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-10 19:51
    How small can the characters go?

    Lots of nice fonts around the 6x10 mark, but if you want a decent VT100 terminal, it really needs 80 columns. That means only 4 pixels wide. And you have to give up one of those for the space between the characters! Is it possible to build a legible font only 3 pixels wide?

    Well, I'm glad you asked because I would have said no, but then I found the clever boffins here http://robey.lag.net/2010/01/23/tiny-monospace-font.html have gone and done it. Only small catch is that I could not read the .bdf file (tried multiple programs) so in the end I wrote my own translator in vb.net.

    So we have an 80x40 column text display now. White on blue seems the most legible. I'm working on connecting this to the VT100 display code from the old Zicog projects.

    Other larger font sizes are easy to add. I have a 53x24 one that is easier on the eyes. Also the real keyboard is connected and it is quicker for touchtyping.

    Once a proper terminal is working maybe we can look at getting wordstar going using the zicog?

    Attachments below - latest spin program, and also the vb.net program used to create fonts and icons.


    Looks like I have some questions to answer above, I'll get to those in a bit...
  • Oldbitcollector (Jeff)Oldbitcollector (Jeff) Posts: 8,091
    edited 2012-02-10 20:26
    Is that really on that 2.5" screen!!?? I can't imagine using Wordstar at that size! :)

    Awesome work you are doing here.

    OBC
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2012-02-10 21:04
    @OBC - yes this is the little display. Mine is 2.4".

    The Wordstar screenshot is fake - it is text from a file on the sd card rather than a real Z80 emulation. But it is legible and usable I believe. When Cluso gets the file I/O working from CP/M this is going to be very useful. As an example, the location of the icons on the screen could be stored in a text file and you could edit that text file to move them around.

    This is a single color font but every character could have any foreground and background if desired.

    @TyperSkyler, there are 16 data lines and 5 control lines. They can go directly to the propeller pins.
    You will run out of prop pins to do the touchscreen but they can be added later.
    I never got the SD card on my display to work - maybe the pins are different. I used my own SD socket but it is wired the same way. 4 pins to propeller pins with pullups.
    I'm using external ram and I am still inventing new faster circuits every day. But you can still get things working without external ram. Just move things from the SD card straight to the display.

    @average joe - the very first bootup message file is JDLCD.ifn and I've attached this below. Also a courier font. Without external ram you will need to read these in for every letter but at least that will prove the concept.

    Re the power supply, VCC on the board is 5V (it has its own internal 3.3V reg). And I connected the LED via a 220 ohm resistor to 5V.
    640 x 480 - 78K
  • Cluso99Cluso99 Posts: 17,970
    edited 2012-02-10 21:28
    I cannot believe that this is a 3bit wide font and I can read it. I would also have said it was impossible. Great work Drac.
  • potatoheadpotatohead Posts: 10,217
    edited 2012-02-10 23:59
    That font is seriously impressive. Very nice work happening here.
Sign In or Register to comment.