GAP Buffer Needed functions?...and Display Portability Thoughts?
CassLan
Posts: 586
Hi all,
This is a two parter..somewhat related:
Part 1)
GAP BUFFER FUNCTIONS:
Been working on the GAP buffer as an approach to tackle the text editor project, specifically implementing the GAP buffer with MagIO's modification to fsrw.
I've re-written my implementation (and interpretation) in a much more organized manner, and added several functions that are neccesary for my project.
These are the functions I have completed and tested extensively:
VMInitialize - Assign Blank or null value to a large area of VM, and write it to the SD. This will make VM Dumps more accurate for debugging.
GAPFileLoad - Establish a GAP Buffer based on 4 Varialbles and load it from a file
GAPFileSave - Create a file and fill it with contents of GAP Buffer
GAPTransfer - Move Data from LHB<->RHB in either direction, resizing LHB or RHB in accordance
GAPInsert - Insert Data into GAP Buffer in either side, resizing in accordance
GAPDelete - Delete Data from GAP buffer from either side, resizing in accordance
GAPResize - Resize the GAP between LHB and RHB, shifting Data as Needed
So, I think with these functions I can cross memory management off my list and move ahead to the part I'm interested in·
Does anyone see any functions missing for GAP Buffer stuff above? This is my interpretation as a memory layout:
Part 2)
PROJECT PORTABILITY
I'm trying to write the editor in a way that it is oblivious to the VM functions of the modified fsrw and only uses these above functions..I'm hoping that in doing it this way if another approach to memory management comes into play..than these above functions ONLY can be re-written to adapt to it. (RAM, EEPROM, HD?? ..etc)
One of the issues I'm concerened with is that people have to format the sdcard..create the swap.sys...copy it first..etc, and currently that is good enough for me, but I see it immediately destroys the chance for·CP/M folks to use this, because they use the first contigious sectors for there own memory tricks. I think that in the not too distant future we will see Large Data blocks implemented with standard files on the SD with no need to format..copy first. The FSRW guys are just adding features like every time I refresh the page
The same portability issues come to mind for me with the display driver, at first I was integrating the display driver into the editor code..this would allow me direct access to the display buffer to increase speed, now I'm thinking this too should be an "oblivious to the editor" type of situation, I know some people prefer VGA to TV, and If I can work within a specific set of common functions across many display drivers...this editor can become a very usefull tool.
Does anyone have any suggestions on what Display Drivers I should look at as a starting point for cross-object compatibility?
Off the top of my head for TV:
TV Text @ half height - Baggers
8x8 NTSC (80 Collumns woot) - potatohead
and I don't usually use VGA so what I see in my Parallax folder is:
VGA_Text·- Chip(I don't think this has enough collumns to be usefull for the text editor)
VGA_HighResText - Chip (Wow this one is cool)
I would appreciate display driver suggestions, so I can compare what they have etc, I·expect having to make a few "middle-man" functions in the editor to execute the correct command for whatever display object is chosen, I think this is a good way to go.
So whats your Favorite Display Driver??
40 Columns Minimum
25 Rows Minimum
Thanks,
Rick
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
This is a two parter..somewhat related:
Part 1)
GAP BUFFER FUNCTIONS:
Been working on the GAP buffer as an approach to tackle the text editor project, specifically implementing the GAP buffer with MagIO's modification to fsrw.
I've re-written my implementation (and interpretation) in a much more organized manner, and added several functions that are neccesary for my project.
These are the functions I have completed and tested extensively:
VMInitialize - Assign Blank or null value to a large area of VM, and write it to the SD. This will make VM Dumps more accurate for debugging.
GAPFileLoad - Establish a GAP Buffer based on 4 Varialbles and load it from a file
GAPFileSave - Create a file and fill it with contents of GAP Buffer
GAPTransfer - Move Data from LHB<->RHB in either direction, resizing LHB or RHB in accordance
GAPInsert - Insert Data into GAP Buffer in either side, resizing in accordance
GAPDelete - Delete Data from GAP buffer from either side, resizing in accordance
GAPResize - Resize the GAP between LHB and RHB, shifting Data as Needed
So, I think with these functions I can cross memory management off my list and move ahead to the part I'm interested in·
Does anyone see any functions missing for GAP Buffer stuff above? This is my interpretation as a memory layout:
' GAP Buffer Layout ' ' .....###########..........###################........ ' | | | | ' LHBOffset LHB RHBOffset RHB ' ' LHBOffset - Location of first byte of LHB Data ' LHB - Location of first byte of GAP ' RHBOffset - Location of first byte of RHB Data ' RHB - Location of first byte after RHB ' ' Given the Logic of the above mapping: ' ' LHBSize = LHB - LHBOffset ' RHBSize = RHB - RHBOffset ' FileSize = LHBSize + RHBSize ' GAPSize = RHBOffset - LHB
Part 2)
PROJECT PORTABILITY
I'm trying to write the editor in a way that it is oblivious to the VM functions of the modified fsrw and only uses these above functions..I'm hoping that in doing it this way if another approach to memory management comes into play..than these above functions ONLY can be re-written to adapt to it. (RAM, EEPROM, HD??
..etc)One of the issues I'm concerened with is that people have to format the sdcard..create the swap.sys...copy it first..etc, and currently that is good enough for me, but I see it immediately destroys the chance for·CP/M folks to use this, because they use the first contigious sectors for there own memory tricks. I think that in the not too distant future we will see Large Data blocks implemented with standard files on the SD with no need to format..copy first. The FSRW guys are just adding features like every time I refresh the page
The same portability issues come to mind for me with the display driver, at first I was integrating the display driver into the editor code..this would allow me direct access to the display buffer to increase speed, now I'm thinking this too should be an "oblivious to the editor" type of situation, I know some people prefer VGA to TV, and If I can work within a specific set of common functions across many display drivers...this editor can become a very usefull tool.
Does anyone have any suggestions on what Display Drivers I should look at as a starting point for cross-object compatibility?
Off the top of my head for TV:
TV Text @ half height - Baggers
8x8 NTSC (80 Collumns woot) - potatohead
and I don't usually use VGA so what I see in my Parallax folder is:
VGA_Text·- Chip(I don't think this has enough collumns to be usefull for the text editor)
VGA_HighResText - Chip (Wow this one is cool)
I would appreciate display driver suggestions, so I can compare what they have etc, I·expect having to make a few "middle-man" functions in the editor to execute the correct command for whatever display object is chosen, I think this is a good way to go.
So whats your Favorite Display Driver??
40 Columns Minimum
25 Rows Minimum
Thanks,
Rick
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
Comments
A full screen paint on every keystroke is possible on some of them, but not all of them. I thought those things would be slower, but SPIN appears plenty fast! The only place I've seen flicker get bad is on the TV_Text driver, I never looked at why though. Wanted 8x8, got that done, and that's generally what I use.
I've not looked much at Kye's drivers for VGA as I don't use VGA for portability reasons. (capture card in window on PC) I would check them out though. I just tried to find the 80 column VGA text driver with the IBM DOS font and couldn't. Somebody posted up a nice DOS text looking GUI quite a while back that I liked. Maybe somebody knows more about that.
So that's a minimum function screen redraw using:
Character out -- just outputs to current screen cursor location, with scrolling when screen is full.
Character at location -- also sets screen position and allows one character to be set
Color at -- set color of that character cell (this won't be portable, requiring some middle function and adjustment capability for sure)
String out
String at
Character out, and at can be used to do detail things, where string out and at can draw whole lines quick.
Truth is, the whole screen can be dumped to the display using a position function to set cursor at upper left, then just dump the buffer in one shot. I think all the display drivers do that, and if they don't, it's no big deal to add what it takes to do it. Only problem with that is flicker on some drivers, depending on how quickly they write to the screen, and how the display driver works. I was going to add a draw during VBLANK flag to mine, but it's quick enough that flicker doesn't seem to be a big deal. I think TV_Text is slow enough that flicker does happen, particularly with color changes.
Another case would be writing out the screen one line at a time with the string out functions. If number of rows and columns varies, that might be sweet.
If you need something specific, I'm happy to get it added to Potatotext, and it can be ported to other displays as needed. Another option would be to get Potatotext rendering on VGA...
For total portability, don't use color at all, but to set the initial display color set. Maybe have an option? It would be nice to have menus, status line, and text select highlights use color on those display drivers that can use it.
It would be seriously great to have it work at 40, 64, 80. I personally am not interested in less than 40, maybe others are.
Another non portable thing is fonts... Basically, there is the Parallax ROM font, and various 8x8 fonts... ???
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Propeller Wiki: Share the coolness!
8x8 color 80 Column NTSC Text Object
Safety Tip: Life is as good as YOU think it is!
Post Edited (potatohead) : 12/19/2009 6:13:31 PM GMT
I looked in the obex and can't find any of kye's VGA drivers, also realized that the link in my sig for·searching the forums with google doesn't work
80 col with IBM font sounds sweet!
If you know the forum location of kye's vga text drivers that would be great, I'm going to do some more searching Since I plan on delivering the contents of the screen to whatever display driver is chosen, I don't think the String at will come from the display driver.
The color function will require some medeling across drivers, but will be worth it, I want to have user selectable settings for color preferences. This is true....I plan on relying on the ROM font..or at least an equivalent for the boxes, lines etc.
I think changeable fonts·should be a future consideration as the project progresses. I think the right idea is to stick to the basic functions like cursor location and character printing...but thanks
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
Where is OBC? He's got some font tools... Maybe that would be a groovy Propeller project? Load in a font, edit it, then have SPIN write it back out to the SD card as a ready to load SPIN file. Dang, we are getting some where on this chip now. That's totally doable!
As for redraw, I saw something cool. There is a guy authoring a pretty nice word processor for the Atari 8 bitters. He's got a cool display speed management trick I liked. On that machine, an 80 Column display is a bitmap deal. SLOW... So, what he does is defer full screen changes, limiting the display update to a coupla rows. When the user pauses for more than a few hundred MS, he draws the whole thing, buffering keystrokes during the draw.
Good work on the gap buffer code, BTW.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Propeller Wiki: Share the coolness!
8x8 color 80 Column NTSC Text Object
Safety Tip: Life is as good as YOU think it is!
I like 640x320 on the two color driver with xScaling = 2, and yScaling = 2. This gives you 80·columns by 15 rows of all the characters found in the propeller chip's·ROM. You'll have to add acess to the draw character functions in the bmp#Engine·to the bmp#txtEngine to draw the box characters since the printing function weeds them out regularly.
Also, you'll need to mod the terminal so that it doesn't scroll the whole screen if you want it not to do that, etc.
You should be able to handle everything there. Its all well commented.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
Thank you!
I'm looking for a minimum of 25 Rows.
So,
· X_Character_Scaling = 2
· Y_Character_Scaling = 1
Yeilds a 40 Columns x 30 Rows, the rendering at this scale cuts off the top line of pixels which is important for characters like "7". But I get the feeling this is more of a terminal type driver for debugging? Drawing a screen on characters takes·a little time, I understand its drawing each pixel..it's very slick.
Any chance you have another driver up your sleeve more appropriate for a text editor?
So far the only one I can find is VGA_Text High Res...which is great but the only one I can find.
Thanks,
Rick
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
I think you have a little problem of understanding the GAP buffer - or rather it's implementation using VM.
This is what you wrote and you say you have implemented a resize function. Resize somehow sounds like a lot of work for poor prop ;o), as you might talk about moving the whole content of buffers. That's not necessary!
Better GAP buffer Layout: < start of file end of file > ########.............................############ LHB | GAP | RHB | | LHB pointer RHB pointerI'd simply define the whole GAP buffer to be say 2MB, I think that's by far more than we need·for editing SPIN files. The above picture already shows a picture of the file loaded and edited. Directly after opening·it looks like that:
Why? Because when you open a file, the cursor is usually at position 0 in the file. If you enter text in insert mode, you'd simply append it to the LHB and move the LHB pointer. If you delete one character, you'd simply move the RHB one to the right. (No need to really delete it in the buffer).
If you move your cursor to the right, you simply copy the character from RHB pointer to LHB pointer, and move both pointers one to the right. (No need to delete from RHB - maybe you go back without further changes, then the right character is already there, which avoids writebacks)
If you move backwards, you copy the character from LHB pointer to RHB pointer and move both pointers one to the left.
About the swap.sys ... it only has to be continuous and it does not have to be the first file in the file system. I tweaked the fsrw-popen to return the number of the first sector of a file. So, what VM is doing is to open swap.sys first and use the sectors from there on.
All my other VGA drivers are not freeware to be given out. However, I did release a beta version of one of my more powerful drivers on this thread: http://forums.parallax.com/forums/default.aspx?f=25&m=335776
The beta version has a few odd things about it and isn't the best. But the ASM driver portion of it works perfectly. The spin parts of it also work, but they aren't the best and highly optimized.
However, for your purposes it should be fast enough. Use the display character, and display characters functions in it to build a terminal and all the other stuff you need.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
No I'm pretty clear on it. Sure the LHB can start at zero...but I decided that it shouldn't be forced to start at zero, this is to be open to new types of memory management later, although I can see that might seem unclear.
About the re-size function:
The gap itself..the space between the LHB and the RHB does get smaller everytime you type or paste, and although the gap might start at a size of 25_000, it is possible that it will get too small, so I decided to put a resize function just incase.
Actually I cheated a little bit
Ahh ok, well that is good news Mag!!
Does that mean that I will be able to detect the presence of swap.sys, and if it does not exist..have fsrw create a new file (very large of course) to use as the swap.sys file?? How can we guarantee that the file is continuous in sectors if it is not the first file in the filesystem??
Kye:
OK no sweat man, thanks I will check that out.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
Post Edited (CassLan) : 12/20/2009 2:50:42 AM GMT
This is a good VGA driver for a Texteditor. 64 x 24 characters with ROM font and only 2 cogs.
Written by Chip Gracey, I don't know why this is not in the OBEX.
I have written a TV_Text / VGA_Text compatible layer to this driver, attached as VGA_Text64.spin
Hope it is useful
Andy
Thats 4 Choices for people now [noparse]:)[/noparse]
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
ok ... so I missinterpreted your resize function. But yes .. it perfectly makes sense when VM somtimes will also work with RAM. And the offsets are good for that as well, as VM will later on map EEPROM, RAM and SD to different memory locations.
So, one comment about the buffer size when using SD card (you gave a GAP size of 25_000 as an example):
From my point of view it's best to have a GAP buffer which is 2 times the max. filesize you expect. Because then the whole file fits into RHB and a copy fits into LHB. This is good as while scrolling through the buffer there is no need to writeback. When the GAP is to small, you end up in writing back just for scrolling. This will wear out the SD card much faster.
Once we can switch to RAM, of course the goal is to use as little memory as possible or it's limited to the available memory.
So, maybe it makes sense to give the size of GAP buffer as a constant or as start parameter to be flexible.
To make a file continuous for sure, the SD-card needs to be defragmented before creating the file. Defragmenting would be to much efford for the propeller. Maybe it's possible to tweak FSRW again to find enough empty clusters in a row for the given filesize.
Yes! Luckily with your VM Functions...the GAP size can be 10_000_000...or whatever [noparse]:)[/noparse] So I don't really think the resize will ever be used, it was just incase, but trust me the gap size in the default settings will be HUGE
Possibly...Thats a very interesting idea. I think that with the speed requirement of this particular project..its possible to re-write those Gap functions to work with FSRW as it is now, just using a couple of regular temp files. Its just above me at the moment to even think about that but I'm quite sure its possible.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
I understand what your saying, but its not mine to rename, so I'm refering to it as the author calls it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
If the gap gets too small, the fix *should* be simple: move one sector from the D region to the E region or from the B region to the A region
and then shift the D region or B region (respectively); the gap just increased by 512 bytes!
This is the beauty of the gap buffer idea; all the operations are so simple and elegant.
You never have to worry about wearing out the SD card; I'm not sure you'd be able to accomplish that with the Propeller. Worry about that
when people start experiencing it.
I already noticed your complains in the Virtual Memory thread. I think it's childish to complain over and over again. My decision is to name it virtual memory - period.
Just have a look at the wikipedia article about virtual memory:
http://en.wikipedia.org/wiki/Virtual_memory
I still believe that the concept I implemented is not far away from what you call virtual memory. The only thing missing is the trancparency. But as wikipedia states "All implementations (excluding emulators) require hardware support." - which we don't have in the native propeller programming language. Of course a tweaked SPIN interpreter could use virtual memory ... maybe I'll do that somewhen ... but first I have other priorities.
Yes ... the current version is more like a memory mapping of a file, but that's only the current status. As I already explained I plan also to map RAM/EEPROM ... into the address-space. Maybe you don't see the big picture and that's why you don't understand the naming.
And ... please ... accept my decision or ignore ... I won't continue to waste my time with argueing
Post Edited (MagIO2) : 12/20/2009 7:51:37 PM GMT
What that phrase means by "requires", in Wikipedia, is that VM simply can't be done without the hardware that does the mapping and exception generation.
Without the required hardware, the emulating software has to perform an additional look-up and comparison on every individual processor memory fetch. Which could be multiple times per machine code instruction. That is a huge hit on performance!
Virtual Awsome Page File Random Acess Flash Memory - VAPFRAFM
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
I agree with evanh mostly on the meaning of Virtual Memory, but a cease and desist order is not appropriate [noparse]:)[/noparse]
Just needs the virtual part removed since there is no virtualisation occuring.
here is another TV_Text compliant layer for the VGA_HiRes_Text driver. This should allow up
to 128x64 characters on a VGA screen.
Andy
Thats really a great contribution, and I think that will really sum up all the drivers I will need to start
This is really great, these 4 display options are really going to allow people to have their needs and preferences met:
For folks who like to use a TV:
1) TV_Text @ Half Height - 40x30; NTSC & PAL
2) 80 Column Color 8x8 NTSC Text Driver - 64x25; Font Changing
For Folks who like to use a VGA Monitor:
3) TV_Text64 - 64x24
4) TV_Text128 - 128x64
And since the program is going to take into account variable height and width displays...as long as a few minimum requirements are met..I don't see why future display drivers will have any issue running this.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·
If I was able to at least invert the foreground/Background of certain characters on the same line, I think it would be fine.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
NYC Area Prop Club
Prop Forum Search (Via Google)
·