'Worst case' provides 1 process with 6KB PRAM.
'Best case' provides 2 processes with 12KB PRAM each
'Even better case' provides 4 processes with 6KB PRAM each.
Let's start by looking at how we could make this work on the Morpheus (the original target system for uOS):
MEMORY:
---- CPU 1:
2KB mouse driver
2KB keyboard driver
4KB for the CPU2 interface code (in Spin)
6KB microSD card driver
8KB for the managers (in Spin)
8 KB PRAM
2 KB SPI SRAM driver
Evidently, a tight squeeze, and this configuration doesn't even support the flash or RTC. Some code refactoring would definitely have to be done; how plausible that would be, I would see after I had a complete(ish) implementation finished.
---- CPU 2:
2KB SRAM driver
2KB Graphics driver (FullDuplexSerial modified to conform to the uOS standard.)
8 KB PRAM
4KB CPU1 interface code (in Spin)
2KB XT card driver (uses the VGA pins)
8KB for more managers (in SPin)
As you can see, CPU 2 would have about 6KB of unused space. Moving first process to CPU2 would even things out, but it would also make interfacing the storage devices slower.
I already have a custom serial terminal that provides mouse and keyboard support, so those could be dropped from CPU 1 as well.
Now let's see how all this fits together with the PropC3 (my second choice of machine):
The PropC3 has only 1 CPU, but all of its SPI devices share a single bus (which is where an RTC would probably end up, too):
6KB SPI driver (handles everything on the bus except SD)
6KB SD driver
2KB keyboard driver
2KB FullDuplexSerialMod graphics driver
8 KB process manager AND XT card driver
8 KB PRAM
The SPI driver might very probably be quite tough to fit into 6KB, in which case maybe flash support could be dropped.
As you can see, it probably IS doable, although speedy it will not be.
At the moment, I'm performing experiments with a HYDRA and a PMC (no XT); I'll post my results when I'm finished with that.
PropellerGuy, you have an interesting concept for your OS. However, it seems to be anything but tiny. If I understand correctly there is hardly any memory left for utilities. I think you have over estimated the amount of memory needed for your drivers. The SPI driver shouldn't require 6KB, nor should the serial driver require 2KB. And I don't understand the need for a proprietary assembly language, with it's own VM using up a cog. Many drivers will run entirely in PASM within a COG, and no extra hub RAM would be needed. Or they might need some hub RAM for buffers, mailbox and a control structure.
It's not clear whether you allow utilities in anything other than your proprietary assembly. The OS would be more flexible if you allow for utilities written in Spin, C or some other language also. I'm interested to see what you actually implement in the next few months/years.
'Worst case' provides 1 process with 6KB PRAM.
'Best case' provides 2 processes with 12KB PRAM each
'Even better case' provides 4 processes with 6KB PRAM each.
.......
As you can see, it probably IS doable, although speedy it will not be.
At the moment, I'm performing experiments with a HYDRA and a PMC (no XT); I'll post my results when I'm finished with that.
This reads more like a shopping list of the obex rather than "maths". There is no sense in the memory figures that you quote as why would you have 4K combined for mouse and keyboard? I don't understand what you are getting at with "managers" and SRAM drivers for instance. These nice round 2K figures may be the maximum size of a cog image but it could just as well be 100 bytes, and reusable too, so your quoting of these figures shows you don't understand this.
Have you done anything like this before? I know you wrote a fairly simple Spin program based on a graphics engine for a LaserMaze game, what mainly looks like gamepad actions, but it's a quantum leap from there to an O/S. It feels like you are wrapped up in playing a game, except in real life you have to make it work and there are no cheat codes.
Where though do all the buffers go and what is left to actually run the target application?
I've been following this thread, and my takeaway is the same as it was earlier- that the concept of easily reusable objects DOES constitute an "OS", one that is custom tailored to your particular application. You need serial ports? Invoke the serial port object. Switching from a serial LCD to a VGA monitor? Switch your LCD object for the VGA object.
My opinion is that my efforts are better spent on writing better objects, and trying to think ahead when writing them. At the end I might end up with something you could call an OS.
So, those are worst-case scenarios there. That's what the memory map might look like if I used nothing but unmodified Parallax drivers (except FullDuplexSerial).
In the real world, I would assume much better memory usage.
This is not coming out of the blue, either. I developed a miniscule OS about a year back, but I didn't post it to OBEX because it was rather buggy and it had no practical applications whatsoever (it could play HiLo, but that was about it.)
I am attaching a skeleton for uOS. This is actually a heavily modified version of the original OS I developed.
On utilities: This is an idea meant to come to fruition with the Prop 2 - the Prop 1 is probably not powerful enough to run a good desktop interface. The utilities would be applications (or one application) serving specific functions that the OS doesn't provide.
With the Prop 1, they're probably not that useful, unless a programming Jedi comes along and decides to write a multitasking desktop that fits in two cogs.
This version doesn't provide very much AT ALL: no SD, no mouse.
To use it, run the EEPROM formatting program, press any key in the PST, and watch the EEPROM get formatted with an empty filesystem (a 64KB EEPROM is required)
Then modify the main program to support your configuration (its set up for the HYDRA right now) and run it.
And I would love to see what people can make with uASM. More functionality will be coming up in the next release.
Enjoy!
EDIT: The memory allocation is ABSOLUTE worst-case - that is, every ASM driver takes up a full 2K. I would normally assume MUCH more efficient space usage.
A Spin-based interpreter of uASM is going to be painfully slow. You're planning on writing the interpreter in PASM, correct?
You're file system is almost a FAT32 file system with 4K clusters. Why not just use FAT32, and then you'll have compatibility with other devices?
The file system probably is not appropriate for EEPROM since it will rewrite the FAT and directory areas many times. An EEPROM file system should incorporate some form of wear-leveling.
You uASM would be difficult to use without an assembler. Are you planning on writing an assembler?
Even with an assembler it would be tedious to write large programs in uASM. Are you planning on writing a compiler that compiles a HHL into uASM?
The answer to all these questions is a resounding YES, except the part about using FAT32.
I have been unable to find a good document on the FAT32 - do you know of one?
If I can find one, I'll write a tiny FAT32 wrapper so the drive can be kept on a file in the FAT32 system.
I think my filesystem is simpler (and the occupation map organization is meant to make it simple for multi-sector programs to find their other constituent parts.)
Hmmm ... this just doesn't seem to be the way to go.
I think we'll just use PFS(3) for now, because it requires a lot less code to interface; from what I've seen, FAT is pretty complicated. Moreover, FAT clusters are not necessarily 4KB, in fact most Parallax drivers seem to recommend that you format cards with 32KB clusters, and this is probably the format their SD cards are shipped in.
Furthermore, FAT32 is only compatible with drives larger than a certain size, whereas PFS(3) can be used on any drive from 16KB to 8TB.
Finally, it looks like PFS(3) would be slightly faster on large drives that are mostly full, because FAT requires you to traverse the linked list to find empty space, whereas with PFS(3) you can just read in the occupation map 1024 entries (4KB) at a time and scan with a quick loop.
Hmmm ... this just doesn't seem to be the way to go.
I think we'll just use PFS(3) for now, because it requires a lot less code to interface; from what I've seen, FAT is pretty complicated. Moreover, FAT clusters are not necessarily 4KB, in fact most Parallax drivers seem to recommend that you format cards with 32KB clusters, and this is probably the format their SD cards are shipped in.
Furthermore, FAT32 is only compatible with drives larger than a certain size, whereas PFS(3) can be used on any drive from 16KB to 8TB.
Finally, it looks like PFS(3) would be slightly faster on large drives that are mostly full, because FAT requires you to traverse the linked list to find empty space, whereas with PFS(3) you can just read in the occupation map 1024 entries (4KB) at a time and scan with a quick loop.
Pray tell even with just a measly 4GB card, how are you going to get huge files on there in the first place and off there in the second place? What use is all the storage unless you can read/write it on another computer? I use FAT32 for compatibility, not because I like it, but I do optimize how I use it. Then again, I have Ethernet which allows me to get files on and off the system even without FAT32, but still no Prop is an island, it needs to be able to interface to the world.
btw, all my cards seem to have 4k clusters, that's the way they come or when formatted and in the real world you can't expect everyone to change for you, you have to accommodate for them.
This still feels like a game of some description, however I'm sure it will be a great learning exercise for you, especially with all this feedback.
The file system probably is not appropriate for EEPROM since it will rewrite the FAT and directory areas many times.
An EEPROM file system should incorporate some form of wear-leveling.
+1 Wear-levelling definitely would be a requirement of an OS in this hardware format.
Interfacing to a computer is why I'm interested in putting a pseudo-drive on a FAT32 card; then I can use the same card for other stuff too, or switch out between multiple SD drives on one card, or write emulators on the PC. And I still won't have to write applications in uASM that deal with FAT.
And I could format the pseudo-drives on a PC, too!
And I still won't have to write applications in uASM that deal with FAT.
Yes, that uASM is another thing. You're going to have to write an assembler before you do any serious coding. And then you need a uASM interpreter in PASM -- A Spin uASM interpreter is going to be painfully slow. And then you're going to have to write a compiler that converts a HLL to uASM. Otherwise you're going to be writing thousands of lines of code in assembly. The only human I know that can do that is Chip.
Why don't you write your code in Spin instead of uASM? It probably won't be much slower. Or (I can't believe I'm going to suggest this) write it in Forth. There are a few Forth interpreters kicking around. I don't see why you need to invent a whole new language for the Prop when the Prop already has a plethora of languages.
We always need new languages for the Prop. If only to add to Humanoido's famously long list of languages that have been used to create programs for Propellers.
Even my compile to LMM toy language is on the list.
We always need new languages for the Prop. If only to add to Humanoido's famously long list of languages that have been used to create programs for Propellers.
Even my compile to LMM toy language is on the list.
Bit further down that page I have attached a slightly more advanced language (it has different number types) that generates x86 assembler.
They both compile to assembler and I used Cliff L. Biffle's Prop assembler to generate the executables.
Those were desperate times. There was only the Prop Tool for Windows so as a Linux user I started down the own language route. (Well, not actually smart enough to write a Spin or C compiler )
I should put this up on github with some better documentation. Then I might even be tempted to work on it some more. If I remember correctly the way I did jumps and calls was not as optimal as Bill Henning intended in LMM. But then the generated code is grossly sub-optimal anyway.
Edit: Oh and I also started building my own PASM assembler. It was a dismal failure. I had this crazy idea to learn something of modern Ada so started writing the assembler in Ada. All in all biting off more than I could chew. No, I'm not posing that code anywhere.
I most certainly DO NOT plan to write an HLL compiler in uASM -- ugh. What I'll probably end up doing is this:
1) Create a bottom-line assembler in uASM.
2) Create an HLL compiler on my PC with Python.
3) Compile the compiler.
4) Put the compiler on uOS with the assembler.
However, writing a compiler isn't very high on my to-do list, because I'm still writing the OS itself right now.
I most certainly DO NOT plan to write an HLL compiler in uASM -- ugh.
Who suggested that you write a compiler in uASM? That would be insane. I suggested that you write a compiler that GENERATED uASM. Is that what you were responding to?
How are you going to deal with nested functions without a stack even if those functions are non-recursive? You have to save the return addresses somewhere.
E.g. In pseudo code:
procedure f1
call f2
return
procedure f2
call f3
return
call f1
TINY? No. Not unless you want to grow it from a toy language to something useful.
How are you going to deal with nested functions without a stack even if those functions are non-recursive? You have to save the return addresses somewhere.
E.g. In pseudo code:
procedure f1
call f2
return
procedure f2
call f3
return
call f1
TINY? No. Not unless you want to grow it from a toy language to something useful.
He could always compile them down to jmpret instructions! :-)
... does anybody think it would be worthwhile to give each process a stack?
If so, how large should it be?
Definitely, you must provide a stack to each process. You can use some default stack size of a few hundred bytes, but allow some way to provide for a larger stack size.
Comments
'Worst case' provides 1 process with 6KB PRAM.
'Best case' provides 2 processes with 12KB PRAM each
'Even better case' provides 4 processes with 6KB PRAM each.
Let's start by looking at how we could make this work on the Morpheus (the original target system for uOS):
MEMORY:
---- CPU 1:
2KB mouse driver
2KB keyboard driver
4KB for the CPU2 interface code (in Spin)
6KB microSD card driver
8KB for the managers (in Spin)
8 KB PRAM
2 KB SPI SRAM driver
Evidently, a tight squeeze, and this configuration doesn't even support the flash or RTC. Some code refactoring would definitely have to be done; how plausible that would be, I would see after I had a complete(ish) implementation finished.
---- CPU 2:
2KB SRAM driver
2KB Graphics driver (FullDuplexSerial modified to conform to the uOS standard.)
8 KB PRAM
4KB CPU1 interface code (in Spin)
2KB XT card driver (uses the VGA pins)
8KB for more managers (in SPin)
As you can see, CPU 2 would have about 6KB of unused space. Moving first process to CPU2 would even things out, but it would also make interfacing the storage devices slower.
I already have a custom serial terminal that provides mouse and keyboard support, so those could be dropped from CPU 1 as well.
Now let's see how all this fits together with the PropC3 (my second choice of machine):
The PropC3 has only 1 CPU, but all of its SPI devices share a single bus (which is where an RTC would probably end up, too):
6KB SPI driver (handles everything on the bus except SD)
6KB SD driver
2KB keyboard driver
2KB FullDuplexSerialMod graphics driver
8 KB process manager AND XT card driver
8 KB PRAM
The SPI driver might very probably be quite tough to fit into 6KB, in which case maybe flash support could be dropped.
As you can see, it probably IS doable, although speedy it will not be.
At the moment, I'm performing experiments with a HYDRA and a PMC (no XT); I'll post my results when I'm finished with that.
It's not clear whether you allow utilities in anything other than your proprietary assembly. The OS would be more flexible if you allow for utilities written in Spin, C or some other language also. I'm interested to see what you actually implement in the next few months/years.
This reads more like a shopping list of the obex rather than "maths". There is no sense in the memory figures that you quote as why would you have 4K combined for mouse and keyboard? I don't understand what you are getting at with "managers" and SRAM drivers for instance. These nice round 2K figures may be the maximum size of a cog image but it could just as well be 100 bytes, and reusable too, so your quoting of these figures shows you don't understand this.
Have you done anything like this before? I know you wrote a fairly simple Spin program based on a graphics engine for a LaserMaze game, what mainly looks like gamepad actions, but it's a quantum leap from there to an O/S. It feels like you are wrapped up in playing a game, except in real life you have to make it work and there are no cheat codes.
Where though do all the buffers go and what is left to actually run the target application?
and Dave brought up some points too....
My opinion is that my efforts are better spent on writing better objects, and trying to think ahead when writing them. At the end I might end up with something you could call an OS.
Like I said, my opinion.
In the real world, I would assume much better memory usage.
This is not coming out of the blue, either. I developed a miniscule OS about a year back, but I didn't post it to OBEX because it was rather buggy and it had no practical applications whatsoever (it could play HiLo, but that was about it.)
I am attaching a skeleton for uOS. This is actually a heavily modified version of the original OS I developed.
On utilities: This is an idea meant to come to fruition with the Prop 2 - the Prop 1 is probably not powerful enough to run a good desktop interface. The utilities would be applications (or one application) serving specific functions that the OS doesn't provide.
With the Prop 1, they're probably not that useful, unless a programming Jedi comes along and decides to write a multitasking desktop that fits in two cogs.
And here's the download for uOS 0.3.1: uOS v0.3.1.zip
This version doesn't provide very much AT ALL: no SD, no mouse.
To use it, run the EEPROM formatting program, press any key in the PST, and watch the EEPROM get formatted with an empty filesystem (a 64KB EEPROM is required)
Then modify the main program to support your configuration (its set up for the HYDRA right now) and run it.
And I would love to see what people can make with uASM. More functionality will be coming up in the next release.
Enjoy!
EDIT: The memory allocation is ABSOLUTE worst-case - that is, every ASM driver takes up a full 2K. I would normally assume MUCH more efficient space usage.
You're file system is almost a FAT32 file system with 4K clusters. Why not just use FAT32, and then you'll have compatibility with other devices?
The file system probably is not appropriate for EEPROM since it will rewrite the FAT and directory areas many times. An EEPROM file system should incorporate some form of wear-leveling.
You uASM would be difficult to use without an assembler. Are you planning on writing an assembler?
Even with an assembler it would be tedious to write large programs in uASM. Are you planning on writing a compiler that compiles a HHL into uASM?
I have been unable to find a good document on the FAT32 - do you know of one?
If I can find one, I'll write a tiny FAT32 wrapper so the drive can be kept on a file in the FAT32 system.
I think my filesystem is simpler (and the occupation map organization is meant to make it simple for multi-sector programs to find their other constituent parts.)
But if FAT32 looks good ...
Thank you very much!!
I'm taking a look at that.
Thank you very much!!
I'm taking a look at that.
I think we'll just use PFS(3) for now, because it requires a lot less code to interface; from what I've seen, FAT is pretty complicated. Moreover, FAT clusters are not necessarily 4KB, in fact most Parallax drivers seem to recommend that you format cards with 32KB clusters, and this is probably the format their SD cards are shipped in.
Furthermore, FAT32 is only compatible with drives larger than a certain size, whereas PFS(3) can be used on any drive from 16KB to 8TB.
Finally, it looks like PFS(3) would be slightly faster on large drives that are mostly full, because FAT requires you to traverse the linked list to find empty space, whereas with PFS(3) you can just read in the occupation map 1024 entries (4KB) at a time and scan with a quick loop.
http://forums.parallax.com/showthread.php/160014-uOS-A-small-yet-portable-OS-for-the-Propeller
Pray tell even with just a measly 4GB card, how are you going to get huge files on there in the first place and off there in the second place? What use is all the storage unless you can read/write it on another computer? I use FAT32 for compatibility, not because I like it, but I do optimize how I use it. Then again, I have Ethernet which allows me to get files on and off the system even without FAT32, but still no Prop is an island, it needs to be able to interface to the world.
btw, all my cards seem to have 4k clusters, that's the way they come or when formatted and in the real world you can't expect everyone to change for you, you have to accommodate for them.
This still feels like a game of some description, however I'm sure it will be a great learning exercise for you, especially with all this feedback.
And I could format the pseudo-drives on a PC, too!
Why don't you write your code in Spin instead of uASM? It probably won't be much slower. Or (I can't believe I'm going to suggest this) write it in Forth. There are a few Forth interpreters kicking around. I don't see why you need to invent a whole new language for the Prop when the Prop already has a plethora of languages.
Even my compile to LMM toy language is on the list.
Everybody should have a language of their own
Search, search...
Here we, go PropTiny, from 2009: http://forums.parallax.com/showthread.php/113228-PropTiny-Jack-Crenshaw-s-TINY-language-for-the-Prop
Bit further down that page I have attached a slightly more advanced language (it has different number types) that generates x86 assembler.
They both compile to assembler and I used Cliff L. Biffle's Prop assembler to generate the executables.
Those were desperate times. There was only the Prop Tool for Windows so as a Linux user I started down the own language route. (Well, not actually smart enough to write a Spin or C compiler
I should put this up on github with some better documentation. Then I might even be tempted to work on it some more. If I remember correctly the way I did jumps and calls was not as optimal as Bill Henning intended in LMM. But then the generated code is grossly sub-optimal anyway.
Edit: Oh and I also started building my own PASM assembler. It was a dismal failure. I had this crazy idea to learn something of modern Ada so started writing the assembler in Ada. All in all biting off more than I could chew. No, I'm not posing that code anywhere.
1) Create a bottom-line assembler in uASM.
2) Create an HLL compiler on my PC with Python.
3) Compile the compiler.
4) Put the compiler on uOS with the assembler.
However, writing a compiler isn't very high on my to-do list, because I'm still writing the OS itself right now.
Because it's tailored to the needs of uOS, and the code size limit is a lot larger than PASM's (plus code paging is significantly easier.)
By the way ...
... does anybody think it would be worthwhile to give each process a stack?
If so, how large should it be?
I'm considering this because a stack would make it simpler to deal with recursive functions.
Also, what language should the compiler compile?
TINY?
E.g. In pseudo code:
TINY? No. Not unless you want to grow it from a toy language to something useful.
He could always compile them down to jmpret instructions! :-)
Good point. What was I thinking?
I was even reading a bunch of jmpret instructions as I posted that. Silly me.
We don't need a stack to remember return addresses, as long a we have no recursive calls.
Presumably we do need a stack for parameter passing and local variables.
Is there any high level language that does not require the concept of a stack space?
therefore it did not need a stack.