Without any cache/buffering in spin - Each read, write for floppy & hard disk = 1 line x 4 = 4 lines changed (*512 -> *128)
Yes - just go with 4x128 - I will post the files in the next hour (nothing wasted if we don't go with it)
True (my idea originally when I didn't know enough)
Yoda & banked memory: TriBlade simulates banked memory in software, N8VEM presumably must have it in all in hardware. Hopefully, CPM3 will remain identical. TriBlade can certainly emulate the N8VEM but depending on how N8VEM implements it, there could be time impacts or not.
Yoda & monitor program: cannot comment???
Cluso & 128/256512 sector size: Yes, let CPM use 128 byte sectors and we will only read/write 128 bytes (from the packed 512 byte SD sectors). Later, if there is a speed improvement by letting CPM3 deblock (I now think that's unlikely as we can do it faster), we can do it without changing the SD files. (see "oops" post below)
Cluso & Drive letters: True
Cluso & SIMH Banked Memory: True
Heater's Summary: YES
I will post the disk images and code changes required shortly
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ Links to other interesting threads:
Re file compatability with N8VEM, take the 1meg eprom image for instance. That gets created in a simh and then you just change the name from .dsk to .bin, and import it into an eprom programmer program (or not even change the name if you like), and it is a disk. No need for vb6 programs along the way. Maybe the 137 byte thing needs a vb6 program but not the hard drive images.
Re heater "4) Hopefully we can drop an N8VEM console port emulation into some free I/O addresses." = already done. Working for the last week.
Re dpbs, this it the dpb table for the 1meg eprom. It is commented out in my version as I replaced it with the 32k eprom. Some tweaks to a few variables (I'm using a smaller eprom as I have the 4dsystems uDrive for mass storage. Which will soon be superceded by zicog).
When cluso posts the new code I might go through and see how close it is to the 1meg dpb.
From the N8VEM
;DPBLK0: ;DISK PARAMETER BLOCK (ROMDISK 1MB) ** RESERVED FOR FUTURE EXPANSION *
; SPT .DW 256 ; 256 SECTORS OF 128 BYTES PER 32K TRACK
; BSH .DB 2 ; BLOCK SHIFT FACTOR (SIZE OF ALLOCATION BLOCK)
; BLM .DB 15 ; PART OF THE ALLOCATION BLOCK SIZE MATH
; EXM .DB 1 ; DEFINES SIZE OF EXTENT (DIRECTORY INFO)
; DSM .DW 495 ; BLOCKSIZE [noparse][[/noparse]2048] * NUMBER OF BLOCKS +1 =DRIVE SIZE
; DRM .DW 255 ; NUMBER OF DIRECTORY ENTRIES
; AL0 .DB 11110000 ; BIT MAP OF SPACE ALLOCATED TO DIRECTORY
; AL1 .DB 00000000 ; DIRECTORY CAN HAVE UP TO 16 BLOCKS ALLOCATED
; CKS .DW 0 ; SIZE OF DIRECTORY CHECK [noparse][[/noparse]0 IF NON REMOVEABLE]
; OFF .DW 1 ; 1 TRACK RESERVED [noparse][[/noparse]FIRST 32K OF ROM]
DPBLK0: ; DISK PARAMETER BLOCK (ROMDISK 32KB WITH 16 2K TRACKS, 22K usable)
SPT_0: .DW 16 ; 16 SECTORS OF 128 BYTES PER 2K TRACK
BSH_0: .DB 3 ; BLOCK SHIFT FACTOR (SIZE OF ALLOCATION BLOCK)
BLM_0: .DB 7 ; PART OF THE ALLOCATION BLOCK SIZE MATH
EXM_0: .DB 0 ; DEFINES SIZE OF EXTENT (DIRECTORY INFO)
DSM_0: .DW 31 ; BLOCKSIZE [noparse][[/noparse]1024] * NUMBER OF BLOCKS + 1 = DRIVE SIZE
DRM_0: .DW 31 ; NUMBER OF DIRECTORY ENTRIES
AL0_0: .DB 10000000 ; BIT MAP OF SPACE ALLOCATED TO DIRECTORY
AL1_0: .DB 00000000 ; DIRECTORY CAN HAVE UP TO 16 BLOCKS ALLOCATED
CKS_0: .DW 0 ; SIZE OF DIRECTORY CHECK [noparse][[/noparse]0 IF NON REMOVEABLE]
OFF_0: .DW 5 ; FIRST 5 TRACKS TRACKS RESERVED (10K FOR SYSTEM)
; SYSTEM IS ROM LOADER, CCP, BDOS, CBIOS, AND MONITOR
I'm looking forward to speed tests on the new code. Meanwhile, back to check that ramdisk code...
Cluso: Bear with me for a minute, perhaps I'm tired, I'm not quite following you.
There must be a worthwhile speed up by packing 4 times 128 bytes sectors into 512 byte SD blocks. provided the CP/M file system is not to badly fragmented.
Lets say we want to read consecutive sectors (at boot up say):
We get the command to read the first sector. Fetch a 512 bytes SD block. Deliver the first 128 bytes of it as the result.
Then comes the command to read the next sector. We have remembered that we already have the SD block required in our buffer, so deliver the second 128 bytes as the result with no SD read.
And so on for the other two sectors.
So 4 sectors read but only one SD block read. sounds good to me.
No speed gain on writes unless we use delayed writes to SD block. I would skip delayed writes for now.
So I'm all for backing 4 sectors to one SD block. Not to mention the space and download time saving.
This is great looks like we have agreed a plan.
Comments Yoda ?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
I just started modifying the ZiCog demo code to support 128 byte sectors within the 512 byte sectors on the SD card. I recalled that I use the low level block access where I pass the address and length to the SD driver and it returns the 128 bytes only. I thought the address was converted to a byte offset on the SD card BUT this is not correct. It in fact uses the sector offset on the SD card.
So,·the driver only reads the first 128 bytes from the SD card (or at least only passes 128 bytes to the upper driver), which of course is faster. When you write the first 128 bytes to the SD card, it pads the remainder of the sector with zeros. Likewise faster.
However, if we convert to 4 x 128 byte sectors per SD sector, the SD low level driver requires the whole 512 bytes to be delivered to the upper driver and it take care of the unpacking and packing. This is going to have a severe impact on performance.
Since we are not short of space on the SD card, I am going to reverse my recomendation and stick with the current method·and use·only the first 128 bytes of each SD sector. However, I recommend we trim the floppy files to 8MB (=1MB usable). Trimming the files in fact·has no change on any software, except Heater's Prop Demo Board since he uses a single contiguous file which is already incompatible. Heater - I presume you will just concatenate the new files and ditch the 137 byte sectors????
Comments please ??? Sorry I goofed !!!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ Links to other interesting threads:
What is wrong with my description of reads of consecutive 128 byte sectors when packed 4 at a time into 512 byte SD blocks? For each 4 sectors to be read only one SD block needs to fetched. Sounds like a 4 times speed gain over what we do now to me.
Is it writes that are the problem? With packed sectors for each write we have to:
1) Fetch the 512 byte sector
2) Update on of the 4 "quarters" of it.
3) Write the thing out again.
Looks here like we have halved the write speed. (Unless "lazy writes" are implemented).
So 4 times gain on read, half speed loss on writes.
In real world use, compiling say, what is the perceived net effect?
Boot up will and program loading will all be faster.
Copying from drive to drive may still approach double our current speed. (4 times read speed) * (0.5 times write speed).
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
The read is based on caching so there is a certain overhead in that, providing it is sequential access. So, yes possibly the majority may be significantly faster.
Writes however, will be slower. This is because with only 1 x 128 byte sector there is no need to read first. Only 128 bytes are transferred and they are written followed by 384 nulls. Though, writes are much much less than reads.
So, while more complex than I thought, 4 x 128 will be faster. And CPM will not even know because it is outside it's knowledge.
I guess, I just thought it was an extremely simple matter, when in fact it is not quite so.
We have other things to do this week, so lets fix them and at the same time maybe we can try some tests with both before we decide. I will have hardware tomorrow evening but do not expect time till Tuesday evening or late, depending on Tuesday meeting.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ Links to other interesting threads:
I am happy to defer packing of sectors till V2. What we should do though is come up with a common set of disk geometries. Since I will be starting the uSD integration into propIO next week I would like to keep as much in common as possible. I really would like to keep the CBIOS as much as common as possible. I think I can probably move the conin and conout to the same place as the Altair if it doesn't conflict with the I/O on the N8VEM, it is a simple change for me. The disk ports will definitely be different which will cause ifdefs in the CBIOS. I need to look at how SIMH does bank memory (does SIMH do banked memory as that would have been beyond an Altair ?). If possible we could make this common or this will be another set of ifdefs. Either way if we want the ultimate commanality, we will introduce a set of ifdefs into the CBIOS. I don't think that is that bad as how often is the CBIOS changing?
In terms of booting I still like the monitor - kind of still in the spirit of a real z80 system, but we can make it work both ways.
Dave
Hi! I am with Dave on this. In my *opinion* the PropIO could easily handle one or both of the packed formats for a disk image. The Propeller and CBIOS hides all the implementation details from CP/M. If the CBIOS supports a 128 byte sector being passed from the PropIO, how the Propeller extracts the 128 bytes out of a 512 byte physical sector is an implementation. Single CP/M sector to single physical sector (loose) or 4 CP/M sector to single physical sector is entirely arbitrary and I think the Propeller could easily do both. The key would be to put some metadata in the disk image file setting up the rules so the disk images are interpreted correctly.
Alternatively, the Propeller could just pass 512 byte physical sectors to the CBIOS and let it handle the conversion effort. The N8VEM PropIO CBIOS would need two scenarios to decode the CP/M to physical mapping but assuming it can identify which disk image is being used at the moment, that is certainly doable.
Dave's point is 100% correct though; we should agree on a standard and take the time to do this so the agreement leverages the synergy, not adds to the confusion. I think we can agree on a common format or at least support each others works. I also agree that the N8VEM should boot to the RAM monitor but that is personal preference. Some people like to boot directly to CP/M and there are ROM images which do this very thing.
How about this; can we agree on a disk image format with metadata in the header to capture the DPH/DPB parameters *AND* the CP/M to Physical mapping construct? Then when the Propeller or ZiCog or PropIO or N8VEM CBIOS mounts one of these disk images, it goes through a metadata parsing algorithm and implements the correct decoding logic based on the situation it is facing?
One point of confusion that keeps popping up is that there is *NO* distinction between floppy drives and hard drives in CP/M itself. It only recognizes character IO (CONIN, CONOUT) and block devices (READ, WRITE). CP/M doesn't know or care about physical sector size nor DMA. All those details are abstracted into the CBIOS implementation. The N8VEM design leverages the abstraction by treating all block devices the same (ROM, RAM, IDE, ATAPI, and now floppy disks). Adding PropIO and uSD support is just another block device handled in the CBIOS and/or PropIO.
BTW, the N8VEM CBIOS is absolutely stuffed with ifdefs already as James can attest. This situation is about to get even more complicated because in parallel with massive developments on the PropIO (Dave), another builder (Dan) has taken up the challenge of writing a FDC CBIOS for true floppy drive support. He is doing great and true floppy drive support in the CBIOS is so close you can practically taste it from here;
This is a fabulous development but I also see two major parallel developments coming together in one massive merge in the not too distant future. Fortunately, the CBIOS changes so far to support PropIO are fairly minimal (character mode only) but that is about to change and add another block device to support uSD from the PropIO.
From my perspective, the ZiCog/Triblade/N8VEM PropIO have far more in common than differences. What differences there are can be addressed in disk image file metadata, Propeller code and/or CBIOS code, and generous use of ifdefs in the source code.
I hope this helps. I appreciate all the hard work you guys are doing. Just having this discussion makes me happy.
heater said...
Wow, that's lot of post to absorb here already. Might take a little while to comprehend it all.
After a nights sleep I'm changing my mind and warming up to packing of 4 times 128 bytes into 512 bytes SD sectors. Why:
1) All round speed gain for floppies and HDs (factor of 4?)
2) Quicker copying of images to/from SD card.
3) It cannot be so hard. Only a few lines of Spin. Leave out any delayed writes initially.
4) Looks like we want the image layout defined NOW.
5) Even if we have gigs of space it just seems aesthetically crappy to waste 32MB of SD for 8MB of data.
Yoda's points:
Yes banked memory was beyond the Altair. I guess SIMH added that enhancement for CPM/3.
I don't understand how this impacts PropIO - ZiCogTriBlade compatibility. Isn't that confined to the N8VEM's Z80 - memory interface? Or am I missing a point here?
Re: a monitor program. ZiCog does not have one but if a Z80 monitor is needed ZiCog could load it from an SD file into the CP/Ms RAM and then start the emulation. Easy. Perhaps we should think about doing that. Then it would be nice to have similar monitor programs in N8VEM and ZiCog. I like the idea, as you say "in the spirit of of a real Z80 system".
Cluso's points:
There is some confusion re-sector size in CP/M.
CP/M (at the BDOS level as far as I understand) only understands 128 byte "logical" sectors. However the BIOS can be written to work with different physical sector sizes. The BIOS then takes care of packing/unpacking the logical sectors into physical sectors. For SIMH and ZiCog/TriBlade and CP/M 2 the BIOS only supports 128 byte sectors all round. With CP/M 3 it has a different BIOS that will block/deblock 512 byte physical sectors.
Anyway I now feel we should shoot for packing 4 * 128 byte sectors into 512 byte SD blocks, as I said above. If I understand correctly you are into that as well. We will do the blocking/unblocking in Spin. Then sometime we can configure CP/M 3 to use 512 byte sectors and it will just use the same disk image if the blocking/deblocking is disabled in Spin.
Again, I don't thing the assignment of drive letters or quantities of floppy hard disks is an issue for PropIO - ZiCog compatibility. That should be configurable at build time (of the Spin emulation) or at run time.
As long as our actual images are interchangeable between the two systems.
That banked memory layout spec we are using I derived from the SIMH implementation of course. Turns out is very simple for us to implement in the PASM Z80 emulation. This is important because memory banking impacts the execution speed of every memory access by the emulator. If the banking model were any more complicated the emulation would get slowed.
In summary I suggest:
1) Let's do the sector packing/unpacking. In Spin. We can then present 512 byte sectors to N8VEM.
2) Let's define the current SIMH 1MB floppy and 8MB hard drives as a "standard". With the difference that for floppies the 137 byte Altair sectors are trimmed to 128 before packing into 512 bytes SD blocks. A simple script or BASIC prog is all that is required to do that trimming.
3) Let's figure out the biggest disk CP/M 3 supports for Cluso's "huge" disk standard.
4) Hopefully we can drop an N8VEM console port emulation into some free I/O addresses.
BINGO we have interchanges able disk images, more speed and smaller downloads.
What do you reckon?
Hi! Let me try on the points;
1. Speed is not really a big concern to me. Reliability and portability is more important. I prefer the 4 to 1 mapping but I don't see 1 to 1 mapping being an issue. This is translation and can even be handled dynamically. (ie, load the 4 to 1 mapped disk image and the Propeller "inflates" it to 1 to 1 for performance sake)
2. There is no standard for drives in CP/M other than the initial IBM 3740 8" SSSD FM formatted 128 byte physical sector disks Gary Kildall used back in primordial days. I recommend using metadata to capture DPH/DPB and physical mapping parameters to abstract the data because the variations are almost unlimited.
3. Haven't made the jump to CP/M 3 yet but it carries over a lot of the CP/M 2.2 structure.
4. console support is good. It is virtual on ZiCog but has real implementations on N8VEM. There are the UART, James keyboard & LCD interface, PropIO VGA & keyboard, my VDU will add another, and so on and so forth. The N8VEM CBIOS is going to have some serious ifdefs to support everything.
Regarding transfer time of disk images, if you'll accept my idea of DPH/DPB and mapping parameter metadata in the header of the file, how about adding a compression layer as well? Use a standardized compression format (gzip .gz or .zip) to squash the mostly empty disk image datafile. Then when the Propeller "mounts" the file, the disk image parameters are passed to the Propeller code and the compressed file expands as needed depending on context. The disk image file would be created dynamically as needed and address any sector mapping issues on the fly. I believe that would be in the "mount" command for disk images.
Sorry, had to take a quick break there for an hour of TV to watch Stephen Fry travel through Andrew Lynch's part of the world. It looked cold!
Ok, I just tried a little hack to see if I understood something. At bootup there are lots of .....
Found where the code is
UART.str(string("Loading SRAM... "))
crlf
repeat
ReadSDblock 'read from SD
WriteSRAMblock 'write to sram
blockno++
UART.tx(".")
while blockno < 128 'eof? (all 1MB)
A 'block' is 8k. And there are 128 of them which makes 1meg. So, what this is doing is loading the entire floppy drive image into ram. Only thing is, I only have 512k of ram. And no ram disk. So - I tried dropping the 128 back to just 8 blocks to load up just the 64k. And it still seems to work fine. Much faster bootup too.
Then I did some timing on those 8k blocks. If we accept, philosophically, that a computer does a lot more reads than writes, and maybe accept slower writes in return for faster reads it might lead to some different possibilities. The time between those '.'s is very short. Loading MBASIC is only 4 ., ie 4x8k. Yet, loading MBASIC takes a lot longer. Why is this so?
Next point is that the creation of a CP/M disk is probably going to involve a blank disk and then a series of PIP commands or R commands. Most of the files are not going to be fragmented at all.
I think reading sector by sector may be taking longer due to this code in the in_dskread
r := sd.initSDcard(spiDO,spiClk,spiDI,spiCS) 'init the SD
CheckError(r)
' this method relies on the SD file using contiguous sectors !!! Only the first 128 bytes of 512 bytes used
sd_block := (drive_base[noparse][[/noparse]altair_drive]) + (altair_track[noparse][[/noparse]altair_drive] * 32) + altair_sector
err := sd.readSDCard(sd_block, @disk_buff, 128) 'read a sector
' CheckError(err)
r := sd.stopSDcard 'stop the SD (free up pins, but keep cog running)
CheckError(r)
So that is starting and stopping the sd card for every sector.
Going back to the boot code, there is this
PRI ReadSDblock | i, n, r 'read blockno of file for count into buffer
r := sd.initSDCard(spiDO,spiClk,spiDI,spiCS) 'init SD card, but do not restart cog
check_error(r)
' this method relies on the SD file using contiguous sectors !!!
r := sd.readSDCard(sdsector, @buff, count) 'get a block (ensure buff big enough!)
check_error(r)
sdsector := ioControl[noparse][[/noparse]0] 'get the next sectorno
r := sd.stopSDCard 'stop SD card, but do not stop cog
check_error(r)
This also starts and stops the sd card, but only every 8k instead of every 128 bytes. ie 64x less often.
@buff reads 8k into an array. The odds are pretty good when reading a file that the sectors are going to be in order.
I wonder, what would happen if every read to a sector first checked if that sector already existed in the block array?
* Get sector number
* Get current block number
* is sector number in block - yes, get it. no, read in new block.
Writes would use existing code. Maybe writes might be a bit slower than they are currently.
But reads, could they be sped up using this method? I wonder, how much of the time needed for a sector read is spent initialising and shutting down the sd card vs actually reading the data?
Andrew: Interesting idea vis the meta-data. At the moment the DPB's for hard drives held in the emulator (ZiCog or SIMH) and passed to the CBIOS when requested. Perhaps we could just change that so that the emulator first gets the DPB from the meta data.
I'm not sure if the Prop is up to zip type compression (but the Prop allways comes up with surprises though). If you know a nice simple memory efficient compression algorithm we could implement in spin it might be worth a thought.
CP/M may not care about physical sector sizes at the higher level. But our SIMH Altair Z80 compatible CBIOSX certainly does.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Clusso: When you say "4 x 128 will be faster.", are you meaning:
a) 4 times 128 byte sectors in one 512 byte SD block.
b) 4 times 128 byte sectors in four 512 byte SD block.
I start to think the only way to see if any speed gains are worth the effort in real world use is to try it out !
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Thanks Andrew. Getting to understand·more each day.
In case anyone wants to try the packed sector format, attached is a zip file which contains the packed 4x128 byte CPM sectors into 1x512 byte SD sector. The floppy files are now 1MB long and have the 128 bytes extracted from the 137 bytes. The hard drive is now 8MB long. It is in fact the identical i.dsk (unmodified) SIMH file as it required no modification.
Now to respond...
We demand contiguous sectors for the files. If we do a lot of file manipulation on the SD drive we may fail with this requirement. I don't think we want to verify each disk/file when we open it (means reading every sector!! ouch).
Therefore, I amagainst the idea of unzipping on the SD drive.
I do not like the idea of embedding the hdp within the file. However, there is another way because we just have to set a basis for the hdp calculations. FAT16 can determine the size of the file. That is all we need to know to work out the hdp parameters. Currently, we already know the default size of the 8B hard disks, so it is only a matter of deciding what we can use for a large disk and standardise on it. (see answer to James below as this is a possibility.
Floppies - Andrew, you raise an interesting point. You treat a floppy as a hard drive. That would be great for speed!!! Perhaps you and heater can discuss this as I know heater does not want to go away from SIMH compatibility.
James:
The 8KB block reads are only done to load up the SRAM and are a remnant from when there was no SD. And yes most of it can be removed. In fact, the only bit of code that needs to be read is the CPM boot code which is loaded into $FF00.
So, what if we had a config file which contained the $FF00 boot code and hdp parameters and anything else we may want to put into our emulations???
The sd.initSDcard and sd.stopSDcard are just commands to tell the SD code to release the SD card. They do not actually do any connection to the SD. But it is within here that the extra time is being·wasted because of the temporary fix to force the release of D0. So, these calls are a requirement of the TriBlade (and RamBlade) to share the bus with the SD and SRAM.
There are lots of things that can be done to speed up the emulation. But these are not a priority yet. We need to get everything working properly first. For instance, spin calls can be replaced directly in pasm.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ Links to other interesting threads:
Heater: I am pretty certain if we do the deblocking in the prop and use 4x128 per 512 SD sector it will be faster, but of course much more complicated. Writes will take a heavy hit but reads will be quite a bit faster so as there are on balance many more reads than writes, it will in fact be faster. I am just not sure that we should do it for rev 1.0.
James:
re the comment on the other thread on SDdrivers. Certainly you can try and replace the driver. Heater is using a different driver which does not have FAT16 support. It still has the bug that I require fixing for the TriBlade.
I am sure you can help in better ways as this will only take me an evening when I get my hardware - and it will take me time to explain what is happening in the TriBlade hardware. Just believe me when I say the sd.initSDcard and sd.stopSDcard are required and when working properly (new driver) it will fix the insane delay being imposed on the card now. If you want to see the results of the new driver compared with the old driver (without my required fix), see the thread http://forums.parallax.com/forums/default.aspx?f=25&m=368563
James, it would be really helpful if you could see if you can test and get the banking on CPM3 working ???
I will be retiring shortly. Driving to Qld tomorrow, so don't know if I will be on toorrow evening. Certainly will not be Tuesday daytime.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ Links to other interesting threads:
heater said...
Andrew: Interesting idea vis the meta-data. At the moment the DPB's for hard drives held in the emulator (ZiCog or SIMH) and passed to the CBIOS when requested. Perhaps we could just change that so that the emulator first gets the DPB from the meta data.
I'm not sure if the Prop is up to zip type compression (but the Prop allways comes up with surprises though). If you know a nice simple memory efficient compression algorithm we could implement in spin it might be worth a thought.
CP/M may not care about physical sector sizes at the higher level. But our SIMH Altair Z80 compatible CBIOSX certainly does.
Hi! OK, I understand the Propeller is going to need some simple compression algorithm if you want to pursue it. I recommend this approach as a quick and easy fix; compress based on 128 byte blocks using RLE encoding. The wikipedia article includes a sample algorithm in Java which should be portable to Spin.
My suggestion is to set a metadata bit at the front of each block indicating if it is compressed or not. Use RLE for any compressed blocks. Perform compression on all 128 byte blocks and if the resulting compressed block is smaller than the original, keep it and mark the metadata as a compressed block. If not smaller, keep the original data and mark its metadata as uncompressed. GNU TAR uses a similar algorithm. As with all compression, you may consider some sort of CRC or checksum for data integrity
Any of the CRC algorithms would probably be sufficient and if you'd like example source it is available on the wiki. A quick boost may be the GNU cksum utility
Notionally, the resulting file would be a group of packets with headers containing the bit for compression status, length of data, and checksum
An uncompressed block packet would look like this
00,7F,AA,[noparse][[/noparse]data bytes 00-7F] AA being the checksum or CRC
a compressed block packet would look like this
FF,55,AA,[noparse][[/noparse]data bytes variable length, 55 in this case]
Using block packaging, RLE encoding, and XOR8 checksums would be relatively lightweight processing and minimal code size and a fair compression ratio with some data integrity.
Guess I am not allowed to sleep anymore - the world changes fast here. After reading everything let me try to simplify or add to the confusion. Here are my goals
I would like:
1) have complete interoperability between ZiCog-triblade, N8VEM, and SIMH
You may think this is crazy but it can be done. It will take a few tweaks of all three. What we would loose that it is a pure altairz80. Getting 3 and loosing one seems pretty good to me.
What does this mean: A common conin, conout ports and a common disk emulation, and common way of doing bank select. The only one of these that is fixed at the moment is the bank select latch in the N8VEM. I want to be able to boot from the boot tracks of the disk image so this is why we have to have the above. That way we can build CPM with CPM and use the standard tools that CPM does to do this.
2) I like the packed blocks and I think we do this in spin code and not in CBIOS as I think it is much simpler to do there.
3) Because of 1 we should load a minimum monitor and make a selection from there. Already done on N8VEM - just need to change what is loaded on Triblade. SIMH will have to take a look at but should probably be straightforward there too.
What all does this mean
I am considering N8VEM to be N8VEM + propIO. The eprom becomes only the monitor and we boot CPM from uSD. No more drives in eprom in this config. We have banked memory and possibly a ram disk with it as well. We need to investigate how CPM3 uses banked memory as I recall it is a buffer for disk so in effect it is a ramdisk of sorts already. With Cluso's new ramblade we only have 512M of memory so a ram disk might only be temporary anyways.
On the triblade there is not much change. We can either move propIO conin, conout to match triblade or vice versa. I have no strong feelings here - as long as the ports are open on the N8VEM I/O space.
Just change triblade to load minimal monitor initially instead of CPM image.
SIMH - Change to same I/O space.
CBIOS - Ifdef to the new I/O space. This will only require 4 places CONIN, CONOUT. Memory Banking, and the Disk controller area. We may be able to reduce to 2 if I can move propIO to same CONIN, CONOUT addresses. I will investigate that.
We need to define DPH/DPB for the disks we want. I think they are 1MB, 8MB and a large disk which I think CPM3 supports 512MB (that should be plenty large)
On the N8VEM I think there will be 2 streams anyways. Those that use the propIO will go down this stream and those that use the Floppy/IDE board will go another path. I am not envisioning people using both at the same time - that is a problem I don't want to solve and will leave to someone else.
So if we do correctly then a disk image generated on SIMH can be booted on SIMH, ZiCiog-Triblade, and NV8EM + propIO which I would say would be a great accomplishment.
These are my thoughts - I hope I have cleared up things some but probably have made it more confusing.
Yep the pockeTerm is/was a starting point for the terminal. It is a great piece of code and I fully acknowledge your work in the code as we discussed. I have a PockeTerm as well and it is a great little board.
Dave
Yoda, sorry for the delay, I was bumping elbows with Parallax hot shots at UPENE yesterday [noparse]:)[/noparse] I hope the code is of help, case statements would be the ideal way to go with reguards to the code.
OBC had already started the VT-100 code when I got started. Our combined effort made the VT-100 possible. When I talked with OBC yesterday about that, he asked "did you clean up that code". My reply was no! If you guys get stuck somewhere, let me know, I'll fire up my N8VEM.
I have a feeling our Spin emulation could handle both at the same tiime ! There is lots of free space in the I/O map.
Dr_A has already sneaked an N8VEM port in to ZiCog to enable the N8VEM xmodem program to work.
* Common bank ??
Still no one has explained to me why this is an issue here. I realize N8VEM is bound by it's hardware. ZiCog/TriBlade is bound by its software, strangely enough. We have implemented SIMH compatible banking which as I said before is also very minimal in its implementation in the Z80 emulation. We need to keeo close to that for speed. Bank checking has to be done on every memory access after all.
* Packed blocks OK
Just this evening I am attempting to implement at least reads of packed blocks in floppies.
* Monitor program OK
Well again I don't see this as a compatibility issue between SIMH, N8VEM and ZiCog. Perhaps I am missing a point.
SIMH and ZiCog could load a monitor program into RAM prior to starting the emulation very easily if they choose.
ZiCog could just read the monitor from an SD file and "POKE" it into RAM.
SIMH could introduce a new driver to do it. The floppy and hard disk drivers in SIMH do exactly with bootrom code if you configure to boot from them. This monitor soounds like that but without any emulated hardware device with it.
Not sure how a monitor would get from the Prop to the N8VEM though.
From my brief reading around CP/M 3 and banked memory it is not really used like a RAM disk. As far as I can tell the basic bank switching puts most of the CCP, BDOS and BIOS into a bank along with disk buffers and such. The application area, TPA, is then in another bank. The TPA can now be much bigger because most of CP/M is out of the way. There is a "resident" area at the top of memory for the bits of CP/M required to actually get entry into it etc. In SIMH the resident area is 16K, banks are 48K. This is what we have implemented in TriBlade.
Use of other banks appears to be entirely down to the application program to organize for itself.
Things get a bit smarter in MP/M where there is effectively many TPAs at the same time.
I'd like to stick with regular SIMH DPH/DPB for the 1MB floppies and 8MB hard drives. The Cluso mega drive has yet to be determined.
My view on SIMH compatibility.
Firstly: for ZiCog to use stock CP/M files (CNIOS, BOOT code...) with no or very minor changes. I have no desire to be BIOS writing or maintaining a wildly different version. If we end up with a few little diffs to patch into new versions from Peter Schorn we can live with that.
Secondly: For media compatibility. Sadly this is not really possible with the crazy Altair 137 byte sector floppies. But again if the transformation from Altair to ZiCog can be described in one paragraph and performed in a dead simple script or BASIC then it's OK. But no changing of sectors per track, number of track, skew, etc etc.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Hi! I agree with Dave. He pretty much has it captured I think. The only thing I would add is that I don't see the N8VEM PropIO and DiskIO as mutually exclusive. I believe They are complimentary and reinforce each other. PropIO adds VGA display, PS/2 keyboard, and uSD data storage. DiskIO adds IDE hard drive, ATAPI (CD-ROM, LS120, ZIP, etc), compact flash, and floppy interface. They both do forms of mass storage which compliment the other.
The issues in developing the DiskIO mirror those same steps in PropIO development and vice versa. I believe the tools created for DiskIO FDC CBIOS support are directly applicable to the PropIO CBIOS development. Tools like ECB bus monitor, DSKY, and LODCBIOS allow an SBC which boots CP/M and/or RAM monitor from its built in ROM drive to take on temporarily the "personality" of another CBIOS for debugging and testing.
I am very glad we are discussing this *beforehand* rather than trying to back into a compatible solution *after* the decisions are already made. This is a very healthy and wise discussion. My hats off to all. Please let me know what I can do to help!
Thank you very much and have a nice day!
Andrew Lynch, 73 de N8VEM
PS, what ever happened to Mike Green's CP/M SBC for Propeller project? last I heard he had selected a Zilog SBC of some sort (ez80?) and was struggling to get it to boot or something like that. I sure wish he'd chime in.
PPS, after PropIO and DiskIO are both settled, I have another N8VEM Propeller project in the works. This would be a Propeller based ECB prototyping board. Not much exists on it other than a mental image. Basically it is an ECB prototyping board with Propeller chip on it and standardized connectors to access its pins.
I think we are all pretty much on the same page now.
Heater: You asked how the monitor gets from the propIO to the N8VEM - it doesn't - it is in eprom on the N8VEM - that would be the only think in the eprom. I think you are trying to separate the propIO from the N8VEM. I consider them as one unit in this discussion of how to make things compatible. Some things are fixed because of the N8VEM (like it boots from eprom and its memory bank switching is in hardware), the propIO is where we have the freedom to address compatibility as we can do it in software and not restricted by the hardware.
I am happy with the 1MB and 8MB drives and will copy the DPH/DPB's for now. Once I get the Disks working on the propIO, I plan to take the same CBIOS and CPM that is used on SIMH and ZiCog and use that as base for N8VEM + propIO so that we have common code.
Heater has done the depacking for reading (on he ZiCog thread) but I will answer here.
I have a VB6 program that I modified in 15 minutes to create the new format packed floppy files which took seconds to do all 4. The hard drive did not require any chnages from SIMH. I can post that as an exe file for conversion, and heater has a python script for conversion. So we have conversions covered, and we can release the converted files zipped anyway (which I did last night).
So packed 4x128 per 512 SD sector it is (unless speed impact - heater will see shortly).
Lets keep the hdp seperate for now as the disk files are simple. We have time to change but I think we shouldn't.
I am interested to see if we can make the floppies perform like hard drives (block instead of character I/O) like Andrew has done - but for a later time.
Cannot wait to see the progress while I am off-air for next 1-2 days, and then I will have hardware again
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔ Links to other interesting threads:
Wow, so much happening so quickly. Like Cluso, I need some more 'hardware' time. Too much time with 3min gaps at work to log in and dream about things, and not enough time to actually test them. I haven't managed to get to testing that ramdrive code from a few days back but it is on the list.
Great to hear the packed hard drive worked. That means that binary image would be the same as the binary image for the N8VEM rom - just a different size. So that is moving towards compatability.
Re the floppy, interesting to see how little Python code is needed. vb6 can do it too. I'm more comfortable with vb6 - but then not everyone might have it. Python is freely available, right? And on several platforms?
Did heater get it to boot from a hard drive? I'm thinking that if you can read 8 x8k blocks off a floppy image to fill 64k of working z80 space and then start running, that ought to work from a hard drive too? Just do syscpm2 on a hard drive - I must test that.
Then - you could have a bootup message asking which drive to boot from. Or even change it to work like DOS, where it searches for drives and boots off the first one that has a system on the system tracks. It could be a pretty crude system, eg maybe even look for E5 on track 0, sector 0. A blank formatted disk will have E5 there. A disk with an OS will have something else (? a C3 for a jump instruction). That could be just a little bit of spin code. So then, it wouldn't matter if you didn't have any floppy disk images. Still keeps with the simh compatability eg 8 floppies and 2-4 hard drives. But you could leave out floppies if you wanted. Then the vb6/python code isn't critical.
Yoda said...
I think we are all pretty much on the same page now.
Heater: You asked how the monitor gets from the propIO to the N8VEM - it doesn't - it is in eprom on the N8VEM - that would be the only think in the eprom. I think you are trying to separate the propIO from the N8VEM. I consider them as one unit in this discussion of how to make things compatible. Some things are fixed because of the N8VEM (like it boots from eprom and its memory bank switching is in hardware), the propIO is where we have the freedom to address compatibility as we can do it in software and not restricted by the hardware.
I am happy with the 1MB and 8MB drives and will copy the DPH/DPB's for now. Once I get the Disks working on the propIO, I plan to take the same CBIOS and CPM that is used on SIMH and ZiCog and use that as base for N8VEM + propIO so that we have common code.
Dave
Hi Dave! Thanks! As you probably already know, I have DPH/DPBs for 1MB and 8MB drives in the N8VEM CBIOS already. They work great and you are free to use them as much as you like. So can anyone else for that matter! The 8MB units are at maximum size for CP/M 2.2
Thanks and have a nice day!
Andrew Lynch
PS, I will be traveling this week so I may be a bit slow responding! Its not that I don't care (I do) its just my family needs to eat!
Could you post, or direct us to, any block diagrams or schematics of your N8VEM/PropIO set up?
I'm still wondering how bank switching affects the Propeller end as far as I can see it does not.
If you are willing to adopt the SIMH CBIOS then we are well on the way to compatibility[noparse]:)[/noparse]
Sounds like we have a great plan here.
Cluso:
Looks like packed disks will be fine. I was just thinking, on writes we have always assumed fetching a block, updating a sector within it and then writing it out. Terrible slow. But now we see for writes to consecutive sectors only one fetch need be done for each 4 sectors written (still have 4 writes without lazy writes). So this feels better all the time.
I'm confident that we could hack the CBIOS to use the HD driver for all floppies. Can we leave that for a while?
Dr_A:
All VBers should migrate to Python. Uses indentation for delimiting blocks. All Spin users should love it.
Yes its free, for windows go here
http://www.python.org/download/windows/
Booting off the hard drive requires a little extra boot code to be used HDSKBOOT.COM.
It requires more RAM than I have on my DemoBoard so can not test it here.
HDSKBOOT.COM needs I little mod for TriBlade, I'll put it up here soon.
To make a hard drive bootable, just do syscpm2 on drive A: as normal and then do:
A> BOOTGEN CPMBOOT.COM I:
As for the monitor loading I have a Proposal...
Proposal:
I assume N8VEM will have monitor in its PROM which runs on boot and then allows selection of boot drive.
I propose that for ZiCog, if we decide to have a monitor at all, that we get the emulation to do likewise.
So on start up:
1) Emulator reads a boot program from a file in the FAT file system.
2) Places that code into the Z80 RAM area.
3) Starts the emulation.
4) The monitor runs and user can now select boot device or it times out and boots a default disk.
The would mean:
1) Same boot code, at least source used in both TriBlade and N8VEM
2) Not having to have any special CP/M disks with boot loader on.
3) Updates, changes, enhancements to the monitor are easy to drop into FAT.
Yoda, Am I correct inmy assumption above?
Comments?
By the way... I think I'm not very keen on meta information in our disk images after all. And not compression either.
If we have only a hand full of disk formats 1M, 8M, 32M and Cluso HUGE then I don't see the pressing need for it.
I'm all for simplicity and the Pure CP/Mness of simple packed disks.
Besides for the little PropDemoBoard demo set up there is no room for much more code.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Been thinking about the discussions and I think we may still be a little different. What I would like is to be able to use the same boot image across SIMH, triblade, N8VEM + propIO, and eventually even the morpheus board. I want to be able to do more then just read the drives (which is pretty doable), I want to boot from a common disk image. It is a little work but attainable. It will take tweaks on all sides. We could have one set of ifdefs in the BIOS to support all 4 platforms. I think that would be very good for everyone to expand the usage.
Thoughts??
Dave
BTW I checked CONIN,CONSTAT, and CONOUT on the SIMH/Altair Bios - I can probably move there but will be tricky. I only have 4 I/O addresses on the propIO, so we will need a simpler disk controller interface. The SIMH/Altair is using 8 addresses I believe.
For the simh, would the 'standard' boot image (at least for starters) be the cpm2.dsk (8 floppies, 2 hd) that has had DO SYSCPM2 run on it to create the boot tracks?
If yes, then that ought to be possible to use as the boot image for the zicog too. eg no DRIVE__1M. I don't know how to do it as I don't fully understand heater's comments above nor do I understand how the data is stored on a boot sector, but this is just another little code to crack!
But - booting on a N8VEM might be tricky. Presumably that means a board where the propeller boots up and takes over the ram chip on the N8VEM, preloads it with data, then does a reset on the Z80 and lets the Z80 run. I spent an hour last night lying in bed thinking about this. It would involve a new design of the N8VEM board. But no problem there as I've already gone through 8 revisions (my shed is full of boards!). The propeller would replace the eprom and the uart and a few glue chips eg the rom select chips. And it would need some latches. It would be a hybrid propeller/Z80 board, and would still have the ECB backplane.
But - the zicog/triblade may yet make a hybrid z80/propeller unnecessary, especially if the disk access speed can get up to N8VEM speeds.
So - for the moment that leaves us with the N8VEM booting off eprom. So it doesn't really matter what the N8VEM boot system is, as long as it gets to an A> prompt. CP/M is then already loaded, and so it has enough smarts to talk to the sd card via the propeller. So to go back to the top of this post, you would not be booting from a common disk image but at least all systems would be able to read a common disk image.
I'll keep thinking about this, but it is a very fluid situation and new ideas keep appearing on this forum every few hours!
Post Edited (Dr_Acula) : 8/25/2009 12:16:29 AM GMT
Yes but we would have to tweak simh a little to be compatible.
no, no. All I want in the eprom on the N8VEM is the monitor. I want the triblade to boot to the monitor as well. The SIMH has a similar concept already - booting to a loader. It is kind of transparent like you have made N8VEM in the past boot directly to CPM. I don't want CPM in ROM anymore (maybe as a backup situation). What you would do is something from the monitor like b <drive> and it would load the OS from the boot tracks of the selected drive. It would be classic CPM and things like DO SYSCPM2 would work on all the platforms. For example, you could have CPM2 on drive A boot track, CPM3 on drive B boot track and MPM on drive C boot track. Could also send a command to propeller to swap drive letters logically.
SIMH already has the notion of supporting different platforms like altair, cromemco, different disk controllers and even n8vem. We would just need to tweak one of the modules like n8vem.c (call it something else) that would give us the common emulation between SIMH, N8VEM + propIO, ZiCog + triblade, and even ZiCog + Morpheus (if we are lucky). Done right the same set of disk images would boot on all platforms and have a common CBIOS that would be different than the Altair CBIOS by just a few ifdefs.
Ok, let me get this clear. The N8VEM has an eprom. It boots up a small program which asks 'what would you like to load?' and maybe gives you a menu. You select 'cpm2 from drive A of the propeller'
What does it do next? I have't studied the prop I/O schematic closely, but can you take over the bus and write to the ram chip?
But if not, there still could be ways to exchange information back and forth with the eprom boot program (written in assembly I presume) doing OUT and IN instructions to ports that the prop IO can trap and recognise. If you can send a byte and receive a byte, then you can port CP/M over and populate the memory.
Then, like you say, you can load different versions of CP/M directly from simh images. Is this on the right track?
I have to digest this a little while. Common boot disks would be ideal.
Yes SIMH can handle different format/geometry disks but only for hard disks. The CPM/2 , CPM/3, MP/M and other bootable disks from the SIMH site are all Altair 1M floppies disks. I don't see being able to drop them into SIMH, PropIO, Morpeus, ZiCog unless they are all Altair compatible. At least as far as floppy drive A:
I don't see Peter wanting to move away from Altair emulation, that the whole point is it not.
Dr_A: As I said that DRIVE__1M file is not really a disk image, it does not have boot tracks, directories, etc. It just holds the content of 64K RAM as we would like it to be after Z80 reset. So basically it's all zeros accept for the BOOTROM code at FF00. Or it could contain a complete image of CP/M ready to run.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Comments
Yoda & banked memory: TriBlade simulates banked memory in software, N8VEM presumably must have it in all in hardware. Hopefully, CPM3 will remain identical. TriBlade can certainly emulate the N8VEM but depending on how N8VEM implements it, there could be time impacts or not.
Yoda & monitor program: cannot comment???
Cluso & 128/256512 sector size: Yes, let CPM use 128 byte sectors and we will only read/write 128 bytes (from the packed 512 byte SD sectors). Later, if there is a speed improvement by letting CPM3 deblock (I now think that's unlikely as we can do it faster), we can do it without changing the SD files.
Cluso & Drive letters: True
Cluso & SIMH Banked Memory: True
Heater's Summary: YES
I will post the disk images and code changes required shortly
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Post Edited (Cluso99) : 8/23/2009 12:23:06 PM GMT
Re file compatability with N8VEM, take the 1meg eprom image for instance. That gets created in a simh and then you just change the name from .dsk to .bin, and import it into an eprom programmer program (or not even change the name if you like), and it is a disk. No need for vb6 programs along the way. Maybe the 137 byte thing needs a vb6 program but not the hard drive images.
Re heater "4) Hopefully we can drop an N8VEM console port emulation into some free I/O addresses." = already done. Working for the last week.
Re dpbs, this it the dpb table for the 1meg eprom. It is commented out in my version as I replaced it with the 32k eprom. Some tweaks to a few variables (I'm using a smaller eprom as I have the 4dsystems uDrive for mass storage. Which will soon be superceded by zicog).
When cluso posts the new code I might go through and see how close it is to the 1meg dpb.
From the N8VEM
;DPBLK0: ;DISK PARAMETER BLOCK (ROMDISK 1MB) ** RESERVED FOR FUTURE EXPANSION * ; SPT .DW 256 ; 256 SECTORS OF 128 BYTES PER 32K TRACK ; BSH .DB 2 ; BLOCK SHIFT FACTOR (SIZE OF ALLOCATION BLOCK) ; BLM .DB 15 ; PART OF THE ALLOCATION BLOCK SIZE MATH ; EXM .DB 1 ; DEFINES SIZE OF EXTENT (DIRECTORY INFO) ; DSM .DW 495 ; BLOCKSIZE [noparse][[/noparse]2048] * NUMBER OF BLOCKS +1 =DRIVE SIZE ; DRM .DW 255 ; NUMBER OF DIRECTORY ENTRIES ; AL0 .DB 11110000 ; BIT MAP OF SPACE ALLOCATED TO DIRECTORY ; AL1 .DB 00000000 ; DIRECTORY CAN HAVE UP TO 16 BLOCKS ALLOCATED ; CKS .DW 0 ; SIZE OF DIRECTORY CHECK [noparse][[/noparse]0 IF NON REMOVEABLE] ; OFF .DW 1 ; 1 TRACK RESERVED [noparse][[/noparse]FIRST 32K OF ROM] DPBLK0: ; DISK PARAMETER BLOCK (ROMDISK 32KB WITH 16 2K TRACKS, 22K usable) SPT_0: .DW 16 ; 16 SECTORS OF 128 BYTES PER 2K TRACK BSH_0: .DB 3 ; BLOCK SHIFT FACTOR (SIZE OF ALLOCATION BLOCK) BLM_0: .DB 7 ; PART OF THE ALLOCATION BLOCK SIZE MATH EXM_0: .DB 0 ; DEFINES SIZE OF EXTENT (DIRECTORY INFO) DSM_0: .DW 31 ; BLOCKSIZE [noparse][[/noparse]1024] * NUMBER OF BLOCKS + 1 = DRIVE SIZE DRM_0: .DW 31 ; NUMBER OF DIRECTORY ENTRIES AL0_0: .DB 10000000 ; BIT MAP OF SPACE ALLOCATED TO DIRECTORY AL1_0: .DB 00000000 ; DIRECTORY CAN HAVE UP TO 16 BLOCKS ALLOCATED CKS_0: .DW 0 ; SIZE OF DIRECTORY CHECK [noparse][[/noparse]0 IF NON REMOVEABLE] OFF_0: .DW 5 ; FIRST 5 TRACKS TRACKS RESERVED (10K FOR SYSTEM) ; SYSTEM IS ROM LOADER, CCP, BDOS, CBIOS, AND MONITORI'm looking forward to speed tests on the new code. Meanwhile, back to check that ramdisk code...
There must be a worthwhile speed up by packing 4 times 128 bytes sectors into 512 byte SD blocks. provided the CP/M file system is not to badly fragmented.
Lets say we want to read consecutive sectors (at boot up say):
We get the command to read the first sector. Fetch a 512 bytes SD block. Deliver the first 128 bytes of it as the result.
Then comes the command to read the next sector. We have remembered that we already have the SD block required in our buffer, so deliver the second 128 bytes as the result with no SD read.
And so on for the other two sectors.
So 4 sectors read but only one SD block read. sounds good to me.
No speed gain on writes unless we use delayed writes to SD block. I would skip delayed writes for now.
So I'm all for backing 4 sectors to one SD block. Not to mention the space and download time saving.
This is great looks like we have agreed a plan.
Comments Yoda ?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
I just started modifying the ZiCog demo code to support 128 byte sectors within the 512 byte sectors on the SD card. I recalled that I use the low level block access where I pass the address and length to the SD driver and it returns the 128 bytes only. I thought the address was converted to a byte offset on the SD card BUT this is not correct. It in fact uses the sector offset on the SD card.
So,·the driver only reads the first 128 bytes from the SD card (or at least only passes 128 bytes to the upper driver), which of course is faster. When you write the first 128 bytes to the SD card, it pads the remainder of the sector with zeros. Likewise faster.
However, if we convert to 4 x 128 byte sectors per SD sector, the SD low level driver requires the whole 512 bytes to be delivered to the upper driver and it take care of the unpacking and packing. This is going to have a severe impact on performance.
Since we are not short of space on the SD card, I am going to reverse my recomendation and stick with the current method·and use·only the first 128 bytes of each SD sector. However, I recommend we trim the floppy files to 8MB (=1MB usable). Trimming the files in fact·has no change on any software, except Heater's Prop Demo Board since he uses a single contiguous file which is already incompatible. Heater - I presume you will just concatenate the new files and ditch the 137 byte sectors????
Comments please ??? Sorry I goofed !!!
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Post Edited (Cluso99) : 8/23/2009 10:29:59 AM GMT
What is wrong with my description of reads of consecutive 128 byte sectors when packed 4 at a time into 512 byte SD blocks? For each 4 sectors to be read only one SD block needs to fetched. Sounds like a 4 times speed gain over what we do now to me.
Is it writes that are the problem? With packed sectors for each write we have to:
1) Fetch the 512 byte sector
2) Update on of the 4 "quarters" of it.
3) Write the thing out again.
Looks here like we have halved the write speed. (Unless "lazy writes" are implemented).
So 4 times gain on read, half speed loss on writes.
In real world use, compiling say, what is the perceived net effect?
Boot up will and program loading will all be faster.
Copying from drive to drive may still approach double our current speed. (4 times read speed) * (0.5 times write speed).
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Writes however, will be slower. This is because with only 1 x 128 byte sector there is no need to read first. Only 128 bytes are transferred and they are written followed by 384 nulls. Though, writes are much much less than reads.
So, while more complex than I thought, 4 x 128 will be faster. And CPM will not even know because it is outside it's knowledge.
I guess, I just thought it was an extremely simple matter, when in fact it is not quite so.
We have other things to do this week, so lets fix them and at the same time maybe we can try some tests with both before we decide. I will have hardware tomorrow evening but do not expect time till Tuesday evening or late, depending on Tuesday meeting.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Hi! I am with Dave on this. In my *opinion* the PropIO could easily handle one or both of the packed formats for a disk image. The Propeller and CBIOS hides all the implementation details from CP/M. If the CBIOS supports a 128 byte sector being passed from the PropIO, how the Propeller extracts the 128 bytes out of a 512 byte physical sector is an implementation. Single CP/M sector to single physical sector (loose) or 4 CP/M sector to single physical sector is entirely arbitrary and I think the Propeller could easily do both. The key would be to put some metadata in the disk image file setting up the rules so the disk images are interpreted correctly.
Alternatively, the Propeller could just pass 512 byte physical sectors to the CBIOS and let it handle the conversion effort. The N8VEM PropIO CBIOS would need two scenarios to decode the CP/M to physical mapping but assuming it can identify which disk image is being used at the moment, that is certainly doable.
Dave's point is 100% correct though; we should agree on a standard and take the time to do this so the agreement leverages the synergy, not adds to the confusion. I think we can agree on a common format or at least support each others works. I also agree that the N8VEM should boot to the RAM monitor but that is personal preference. Some people like to boot directly to CP/M and there are ROM images which do this very thing.
How about this; can we agree on a disk image format with metadata in the header to capture the DPH/DPB parameters *AND* the CP/M to Physical mapping construct? Then when the Propeller or ZiCog or PropIO or N8VEM CBIOS mounts one of these disk images, it goes through a metadata parsing algorithm and implements the correct decoding logic based on the situation it is facing?
One point of confusion that keeps popping up is that there is *NO* distinction between floppy drives and hard drives in CP/M itself. It only recognizes character IO (CONIN, CONOUT) and block devices (READ, WRITE). CP/M doesn't know or care about physical sector size nor DMA. All those details are abstracted into the CBIOS implementation. The N8VEM design leverages the abstraction by treating all block devices the same (ROM, RAM, IDE, ATAPI, and now floppy disks). Adding PropIO and uSD support is just another block device handled in the CBIOS and/or PropIO.
BTW, the N8VEM CBIOS is absolutely stuffed with ifdefs already as James can attest. This situation is about to get even more complicated because in parallel with massive developments on the PropIO (Dave), another builder (Dan) has taken up the challenge of writing a FDC CBIOS for true floppy drive support. He is doing great and true floppy drive support in the CBIOS is so close you can practically taste it from here;
http://groups.google.com/group/n8vem/browse_thread/thread/5460b493eac078cc/d67329112c7b8d5c?hl=en&
This is a fabulous development but I also see two major parallel developments coming together in one massive merge in the not too distant future. Fortunately, the CBIOS changes so far to support PropIO are fairly minimal (character mode only) but that is about to change and add another block device to support uSD from the PropIO.
From my perspective, the ZiCog/Triblade/N8VEM PropIO have far more in common than differences. What differences there are can be addressed in disk image file metadata, Propeller code and/or CBIOS code, and generous use of ifdefs in the source code.
I hope this helps. I appreciate all the hard work you guys are doing. Just having this discussion makes me happy.
Thanks and have a nice day!
Andrew Lynch
Hi! Let me try on the points;
1. Speed is not really a big concern to me. Reliability and portability is more important. I prefer the 4 to 1 mapping but I don't see 1 to 1 mapping being an issue. This is translation and can even be handled dynamically. (ie, load the 4 to 1 mapped disk image and the Propeller "inflates" it to 1 to 1 for performance sake)
2. There is no standard for drives in CP/M other than the initial IBM 3740 8" SSSD FM formatted 128 byte physical sector disks Gary Kildall used back in primordial days. I recommend using metadata to capture DPH/DPB and physical mapping parameters to abstract the data because the variations are almost unlimited.
3. Haven't made the jump to CP/M 3 yet but it carries over a lot of the CP/M 2.2 structure.
4. console support is good. It is virtual on ZiCog but has real implementations on N8VEM. There are the UART, James keyboard & LCD interface, PropIO VGA & keyboard, my VDU will add another, and so on and so forth. The N8VEM CBIOS is going to have some serious ifdefs to support everything.
Regarding transfer time of disk images, if you'll accept my idea of DPH/DPB and mapping parameter metadata in the header of the file, how about adding a compression layer as well? Use a standardized compression format (gzip .gz or .zip) to squash the mostly empty disk image datafile. Then when the Propeller "mounts" the file, the disk image parameters are passed to the Propeller code and the compressed file expands as needed depending on context. The disk image file would be created dynamically as needed and address any sector mapping issues on the fly. I believe that would be in the "mount" command for disk images.
Thanks and have a nice day!
Andrew Lynch
Ok, I just tried a little hack to see if I understood something. At bootup there are lots of .....
Found where the code is
UART.str(string("Loading SRAM... ")) crlf repeat ReadSDblock 'read from SD WriteSRAMblock 'write to sram blockno++ UART.tx(".") while blockno < 128 'eof? (all 1MB)A 'block' is 8k. And there are 128 of them which makes 1meg. So, what this is doing is loading the entire floppy drive image into ram. Only thing is, I only have 512k of ram. And no ram disk. So - I tried dropping the 128 back to just 8 blocks to load up just the 64k. And it still seems to work fine. Much faster bootup too.
Then I did some timing on those 8k blocks. If we accept, philosophically, that a computer does a lot more reads than writes, and maybe accept slower writes in return for faster reads it might lead to some different possibilities. The time between those '.'s is very short. Loading MBASIC is only 4 ., ie 4x8k. Yet, loading MBASIC takes a lot longer. Why is this so?
Next point is that the creation of a CP/M disk is probably going to involve a blank disk and then a series of PIP commands or R commands. Most of the files are not going to be fragmented at all.
I think reading sector by sector may be taking longer due to this code in the in_dskread
r := sd.initSDcard(spiDO,spiClk,spiDI,spiCS) 'init the SD CheckError(r) ' this method relies on the SD file using contiguous sectors !!! Only the first 128 bytes of 512 bytes used sd_block := (drive_base[noparse][[/noparse]altair_drive]) + (altair_track[noparse][[/noparse]altair_drive] * 32) + altair_sector err := sd.readSDCard(sd_block, @disk_buff, 128) 'read a sector ' CheckError(err) r := sd.stopSDcard 'stop the SD (free up pins, but keep cog running) CheckError(r)So that is starting and stopping the sd card for every sector.
Going back to the boot code, there is this
This also starts and stops the sd card, but only every 8k instead of every 128 bytes. ie 64x less often.
@buff reads 8k into an array. The odds are pretty good when reading a file that the sectors are going to be in order.
I wonder, what would happen if every read to a sector first checked if that sector already existed in the block array?
* Get sector number
* Get current block number
* is sector number in block - yes, get it. no, read in new block.
Writes would use existing code. Maybe writes might be a bit slower than they are currently.
But reads, could they be sped up using this method? I wonder, how much of the time needed for a sector read is spent initialising and shutting down the sd card vs actually reading the data?
I'm not sure if the Prop is up to zip type compression (but the Prop allways comes up with surprises though). If you know a nice simple memory efficient compression algorithm we could implement in spin it might be worth a thought.
CP/M may not care about physical sector sizes at the higher level. But our SIMH Altair Z80 compatible CBIOSX certainly does.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
a) 4 times 128 byte sectors in one 512 byte SD block.
b) 4 times 128 byte sectors in four 512 byte SD block.
I start to think the only way to see if any speed gains are worth the effort in real world use is to try it out !
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
In case anyone wants to try the packed sector format, attached is a zip file which contains the packed 4x128 byte CPM sectors into 1x512 byte SD sector. The floppy files are now 1MB long and have the 128 bytes extracted from the 137 bytes. The hard drive is now 8MB long. It is in fact the identical i.dsk (unmodified) SIMH file as it required no modification.
Now to respond...
James:
There are lots of things that can be done to speed up the emulation. But these are not a priority yet. We need to get everything working properly first. For instance, spin calls can be replaced directly in pasm.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
James:
re the comment on the other thread on SDdrivers. Certainly you can try and replace the driver. Heater is using a different driver which does not have FAT16 support. It still has the bug that I require fixing for the TriBlade.
I am sure you can help in better ways as this will only take me an evening when I get my hardware - and it will take me time to explain what is happening in the TriBlade hardware. Just believe me when I say the sd.initSDcard and sd.stopSDcard are required and when working properly (new driver) it will fix the insane delay being imposed on the card now. If you want to see the results of the new driver compared with the old driver (without my required fix), see the thread http://forums.parallax.com/forums/default.aspx?f=25&m=368563
James, it would be really helpful if you could see if you can test and get the banking on CPM3 working ???
I will be retiring shortly. Driving to Qld tomorrow, so don't know if I will be on toorrow evening. Certainly will not be Tuesday daytime.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Hi! OK, I understand the Propeller is going to need some simple compression algorithm if you want to pursue it. I recommend this approach as a quick and easy fix; compress based on 128 byte blocks using RLE encoding. The wikipedia article includes a sample algorithm in Java which should be portable to Spin.
http://en.wikipedia.org/wiki/Run-length_encoding
My suggestion is to set a metadata bit at the front of each block indicating if it is compressed or not. Use RLE for any compressed blocks. Perform compression on all 128 byte blocks and if the resulting compressed block is smaller than the original, keep it and mark the metadata as a compressed block. If not smaller, keep the original data and mark its metadata as uncompressed. GNU TAR uses a similar algorithm. As with all compression, you may consider some sort of CRC or checksum for data integrity
http://en.wikipedia.org/wiki/Cyclic_redundancy_check
Any of the CRC algorithms would probably be sufficient and if you'd like example source it is available on the wiki. A quick boost may be the GNU cksum utility
http://en.wikipedia.org/wiki/Cksum
A checksum would also work for error detection. Using the XOR8 would be very quick to implement
http://en.wikipedia.org/wiki/List_of_checksum_algorithms
Notionally, the resulting file would be a group of packets with headers containing the bit for compression status, length of data, and checksum
An uncompressed block packet would look like this
00,7F,AA,[noparse][[/noparse]data bytes 00-7F] AA being the checksum or CRC
a compressed block packet would look like this
FF,55,AA,[noparse][[/noparse]data bytes variable length, 55 in this case]
Using block packaging, RLE encoding, and XOR8 checksums would be relatively lightweight processing and minimal code size and a fair compression ratio with some data integrity.
I hope this helps! Thanks and have a nice day!
Andrew Lynch
I would like:
1) have complete interoperability between ZiCog-triblade, N8VEM, and SIMH
You may think this is crazy but it can be done. It will take a few tweaks of all three. What we would loose that it is a pure altairz80. Getting 3 and loosing one seems pretty good to me.
What does this mean: A common conin, conout ports and a common disk emulation, and common way of doing bank select. The only one of these that is fixed at the moment is the bank select latch in the N8VEM. I want to be able to boot from the boot tracks of the disk image so this is why we have to have the above. That way we can build CPM with CPM and use the standard tools that CPM does to do this.
2) I like the packed blocks and I think we do this in spin code and not in CBIOS as I think it is much simpler to do there.
3) Because of 1 we should load a minimum monitor and make a selection from there. Already done on N8VEM - just need to change what is loaded on Triblade. SIMH will have to take a look at but should probably be straightforward there too.
What all does this mean
I am considering N8VEM to be N8VEM + propIO. The eprom becomes only the monitor and we boot CPM from uSD. No more drives in eprom in this config. We have banked memory and possibly a ram disk with it as well. We need to investigate how CPM3 uses banked memory as I recall it is a buffer for disk so in effect it is a ramdisk of sorts already. With Cluso's new ramblade we only have 512M of memory so a ram disk might only be temporary anyways.
On the triblade there is not much change. We can either move propIO conin, conout to match triblade or vice versa. I have no strong feelings here - as long as the ports are open on the N8VEM I/O space.
Just change triblade to load minimal monitor initially instead of CPM image.
SIMH - Change to same I/O space.
CBIOS - Ifdef to the new I/O space. This will only require 4 places CONIN, CONOUT. Memory Banking, and the Disk controller area. We may be able to reduce to 2 if I can move propIO to same CONIN, CONOUT addresses. I will investigate that.
We need to define DPH/DPB for the disks we want. I think they are 1MB, 8MB and a large disk which I think CPM3 supports 512MB (that should be plenty large)
On the N8VEM I think there will be 2 streams anyways. Those that use the propIO will go down this stream and those that use the Floppy/IDE board will go another path. I am not envisioning people using both at the same time - that is a problem I don't want to solve and will leave to someone else.
So if we do correctly then a disk image generated on SIMH can be booted on SIMH, ZiCiog-Triblade, and NV8EM + propIO which I would say would be a great accomplishment.
These are my thoughts - I hope I have cleared up things some but probably have made it more confusing.
Dave
OBC had already started the VT-100 code when I got started. Our combined effort made the VT-100 possible. When I talked with OBC yesterday about that, he asked "did you clean up that code". My reply was no! If you guys get stuck somewhere, let me know, I'll fire up my N8VEM.
Vince
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Briel Computers
http://www.brielcomputers.com
* Common conin OK
* Common conout OK
I have a feeling our Spin emulation could handle both at the same tiime ! There is lots of free space in the I/O map.
Dr_A has already sneaked an N8VEM port in to ZiCog to enable the N8VEM xmodem program to work.
* Common bank ??
Still no one has explained to me why this is an issue here. I realize N8VEM is bound by it's hardware. ZiCog/TriBlade is bound by its software, strangely enough. We have implemented SIMH compatible banking which as I said before is also very minimal in its implementation in the Z80 emulation. We need to keeo close to that for speed. Bank checking has to be done on every memory access after all.
* Packed blocks OK
Just this evening I am attempting to implement at least reads of packed blocks in floppies.
* Monitor program OK
Well again I don't see this as a compatibility issue between SIMH, N8VEM and ZiCog. Perhaps I am missing a point.
SIMH and ZiCog could load a monitor program into RAM prior to starting the emulation very easily if they choose.
ZiCog could just read the monitor from an SD file and "POKE" it into RAM.
SIMH could introduce a new driver to do it. The floppy and hard disk drivers in SIMH do exactly with bootrom code if you configure to boot from them. This monitor soounds like that but without any emulated hardware device with it.
Not sure how a monitor would get from the Prop to the N8VEM though.
From my brief reading around CP/M 3 and banked memory it is not really used like a RAM disk. As far as I can tell the basic bank switching puts most of the CCP, BDOS and BIOS into a bank along with disk buffers and such. The application area, TPA, is then in another bank. The TPA can now be much bigger because most of CP/M is out of the way. There is a "resident" area at the top of memory for the bits of CP/M required to actually get entry into it etc. In SIMH the resident area is 16K, banks are 48K. This is what we have implemented in TriBlade.
Use of other banks appears to be entirely down to the application program to organize for itself.
Things get a bit smarter in MP/M where there is effectively many TPAs at the same time.
I'd like to stick with regular SIMH DPH/DPB for the 1MB floppies and 8MB hard drives. The Cluso mega drive has yet to be determined.
My view on SIMH compatibility.
Firstly: for ZiCog to use stock CP/M files (CNIOS, BOOT code...) with no or very minor changes. I have no desire to be BIOS writing or maintaining a wildly different version. If we end up with a few little diffs to patch into new versions from Peter Schorn we can live with that.
Secondly: For media compatibility. Sadly this is not really possible with the crazy Altair 137 byte sector floppies. But again if the transformation from Altair to ZiCog can be described in one paragraph and performed in a dead simple script or BASIC then it's OK. But no changing of sectors per track, number of track, skew, etc etc.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
The issues in developing the DiskIO mirror those same steps in PropIO development and vice versa. I believe the tools created for DiskIO FDC CBIOS support are directly applicable to the PropIO CBIOS development. Tools like ECB bus monitor, DSKY, and LODCBIOS allow an SBC which boots CP/M and/or RAM monitor from its built in ROM drive to take on temporarily the "personality" of another CBIOS for debugging and testing.
I am very glad we are discussing this *beforehand* rather than trying to back into a compatible solution *after* the decisions are already made. This is a very healthy and wise discussion. My hats off to all. Please let me know what I can do to help!
Thank you very much and have a nice day!
Andrew Lynch, 73 de N8VEM
PS, what ever happened to Mike Green's CP/M SBC for Propeller project? last I heard he had selected a Zilog SBC of some sort (ez80?) and was struggling to get it to boot or something like that. I sure wish he'd chime in.
PPS, after PropIO and DiskIO are both settled, I have another N8VEM Propeller project in the works. This would be a Propeller based ECB prototyping board. Not much exists on it other than a mental image. Basically it is an ECB prototyping board with Propeller chip on it and standardized connectors to access its pins.
http://n8vem-sbc.pbworks.com/New-Peripheral-Concepts
Heater: You asked how the monitor gets from the propIO to the N8VEM - it doesn't - it is in eprom on the N8VEM - that would be the only think in the eprom. I think you are trying to separate the propIO from the N8VEM. I consider them as one unit in this discussion of how to make things compatible. Some things are fixed because of the N8VEM (like it boots from eprom and its memory bank switching is in hardware), the propIO is where we have the freedom to address compatibility as we can do it in software and not restricted by the hardware.
I am happy with the 1MB and 8MB drives and will copy the DPH/DPB's for now. Once I get the Disks working on the propIO, I plan to take the same CBIOS and CPM that is used on SIMH and ZiCog and use that as base for N8VEM + propIO so that we have common code.
Dave
Heater has done the depacking for reading (on he ZiCog thread) but I will answer here.
I have a VB6 program that I modified in 15 minutes to create the new format packed floppy files which took seconds to do all 4. The hard drive did not require any chnages from SIMH. I can post that as an exe file for conversion, and heater has a python script for conversion. So we have conversions covered, and we can release the converted files zipped anyway (which I did last night).
So packed 4x128 per 512 SD sector it is (unless speed impact - heater will see shortly).
Lets keep the hdp seperate for now as the disk files are simple. We have time to change but I think we shouldn't.
I am interested to see if we can make the floppies perform like hard drives (block instead of character I/O) like Andrew has done - but for a later time.
Cannot wait to see the progress while I am off-air for next 1-2 days, and then I will have hardware again
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Great to hear the packed hard drive worked. That means that binary image would be the same as the binary image for the N8VEM rom - just a different size. So that is moving towards compatability.
Re the floppy, interesting to see how little Python code is needed. vb6 can do it too. I'm more comfortable with vb6 - but then not everyone might have it. Python is freely available, right? And on several platforms?
Did heater get it to boot from a hard drive? I'm thinking that if you can read 8 x8k blocks off a floppy image to fill 64k of working z80 space and then start running, that ought to work from a hard drive too? Just do syscpm2 on a hard drive - I must test that.
Then - you could have a bootup message asking which drive to boot from. Or even change it to work like DOS, where it searches for drives and boots off the first one that has a system on the system tracks. It could be a pretty crude system, eg maybe even look for E5 on track 0, sector 0. A blank formatted disk will have E5 there. A disk with an OS will have something else (? a C3 for a jump instruction). That could be just a little bit of spin code. So then, it wouldn't matter if you didn't have any floppy disk images. Still keeps with the simh compatability eg 8 floppies and 2-4 hard drives. But you could leave out floppies if you wanted. Then the vb6/python code isn't critical.
Hi Dave! Thanks! As you probably already know, I have DPH/DPBs for 1MB and 8MB drives in the N8VEM CBIOS already. They work great and you are free to use them as much as you like. So can anyone else for that matter! The 8MB units are at maximum size for CP/M 2.2
Thanks and have a nice day!
Andrew Lynch
PS, I will be traveling this week so I may be a bit slow responding! Its not that I don't care (I do) its just my family needs to eat!
PPS, Great job on the VT-100 support for PropIO!
Post Edited (lynchaj) : 8/24/2009 2:31:00 AM GMT
Could you post, or direct us to, any block diagrams or schematics of your N8VEM/PropIO set up?
I'm still wondering how bank switching affects the Propeller end as far as I can see it does not.
If you are willing to adopt the SIMH CBIOS then we are well on the way to compatibility[noparse]:)[/noparse]
Sounds like we have a great plan here.
Cluso:
Looks like packed disks will be fine. I was just thinking, on writes we have always assumed fetching a block, updating a sector within it and then writing it out. Terrible slow. But now we see for writes to consecutive sectors only one fetch need be done for each 4 sectors written (still have 4 writes without lazy writes). So this feels better all the time.
I'm confident that we could hack the CBIOS to use the HD driver for all floppies. Can we leave that for a while?
Dr_A:
All VBers should migrate to Python. Uses indentation for delimiting blocks. All Spin users should love it.
Yes its free, for windows go here
Booting off the hard drive requires a little extra boot code to be used HDSKBOOT.COM.
It requires more RAM than I have on my DemoBoard so can not test it here.
HDSKBOOT.COM needs I little mod for TriBlade, I'll put it up here soon.
To make a hard drive bootable, just do syscpm2 on drive A: as normal and then do:
A> BOOTGEN CPMBOOT.COM I:
As for the monitor loading I have a Proposal...
Proposal:
I assume N8VEM will have monitor in its PROM which runs on boot and then allows selection of boot drive.
I propose that for ZiCog, if we decide to have a monitor at all, that we get the emulation to do likewise.
So on start up:
1) Emulator reads a boot program from a file in the FAT file system.
2) Places that code into the Z80 RAM area.
3) Starts the emulation.
4) The monitor runs and user can now select boot device or it times out and boots a default disk.
The would mean:
1) Same boot code, at least source used in both TriBlade and N8VEM
2) Not having to have any special CP/M disks with boot loader on.
3) Updates, changes, enhancements to the monitor are easy to drop into FAT.
Yoda, Am I correct inmy assumption above?
Comments?
By the way... I think I'm not very keen on meta information in our disk images after all. And not compression either.
If we have only a hand full of disk formats 1M, 8M, 32M and Cluso HUGE then I don't see the pressing need for it.
I'm all for simplicity and the Pure CP/Mness of simple packed disks.
Besides for the little PropDemoBoard demo set up there is no room for much more code.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Everything sounds great. Maybe I will get time tomorrow evening - we will see.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBladeProp, RamBlade, TwinBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: Micros eg Altair, and Terminals eg VT100 (Index) ZiCog (Z80) , MoCog (6809)
· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBladeProp is: www.bluemagic.biz/cluso.htm
Been thinking about the discussions and I think we may still be a little different. What I would like is to be able to use the same boot image across SIMH, triblade, N8VEM + propIO, and eventually even the morpheus board. I want to be able to do more then just read the drives (which is pretty doable), I want to boot from a common disk image. It is a little work but attainable. It will take tweaks on all sides. We could have one set of ifdefs in the BIOS to support all 4 platforms. I think that would be very good for everyone to expand the usage.
Thoughts??
Dave
BTW I checked CONIN,CONSTAT, and CONOUT on the SIMH/Altair Bios - I can probably move there but will be tricky. I only have 4 I/O addresses on the propIO, so we will need a simpler disk controller interface. The SIMH/Altair is using 8 addresses I believe.
For the simh, would the 'standard' boot image (at least for starters) be the cpm2.dsk (8 floppies, 2 hd) that has had DO SYSCPM2 run on it to create the boot tracks?
If yes, then that ought to be possible to use as the boot image for the zicog too. eg no DRIVE__1M. I don't know how to do it as I don't fully understand heater's comments above nor do I understand how the data is stored on a boot sector, but this is just another little code to crack!
But - booting on a N8VEM might be tricky. Presumably that means a board where the propeller boots up and takes over the ram chip on the N8VEM, preloads it with data, then does a reset on the Z80 and lets the Z80 run. I spent an hour last night lying in bed thinking about this. It would involve a new design of the N8VEM board. But no problem there as I've already gone through 8 revisions (my shed is full of boards!). The propeller would replace the eprom and the uart and a few glue chips eg the rom select chips. And it would need some latches. It would be a hybrid propeller/Z80 board, and would still have the ECB backplane.
But - the zicog/triblade may yet make a hybrid z80/propeller unnecessary, especially if the disk access speed can get up to N8VEM speeds.
So - for the moment that leaves us with the N8VEM booting off eprom. So it doesn't really matter what the N8VEM boot system is, as long as it gets to an A> prompt. CP/M is then already loaded, and so it has enough smarts to talk to the sd card via the propeller. So to go back to the top of this post, you would not be booting from a common disk image but at least all systems would be able to read a common disk image.
I'll keep thinking about this, but it is a very fluid situation and new ideas keep appearing on this forum every few hours!
Post Edited (Dr_Acula) : 8/25/2009 12:16:29 AM GMT
Yes but we would have to tweak simh a little to be compatible.
no, no. All I want in the eprom on the N8VEM is the monitor. I want the triblade to boot to the monitor as well. The SIMH has a similar concept already - booting to a loader. It is kind of transparent like you have made N8VEM in the past boot directly to CPM. I don't want CPM in ROM anymore (maybe as a backup situation). What you would do is something from the monitor like b <drive> and it would load the OS from the boot tracks of the selected drive. It would be classic CPM and things like DO SYSCPM2 would work on all the platforms. For example, you could have CPM2 on drive A boot track, CPM3 on drive B boot track and MPM on drive C boot track. Could also send a command to propeller to swap drive letters logically.
SIMH already has the notion of supporting different platforms like altair, cromemco, different disk controllers and even n8vem. We would just need to tweak one of the modules like n8vem.c (call it something else) that would give us the common emulation between SIMH, N8VEM + propIO, ZiCog + triblade, and even ZiCog + Morpheus (if we are lucky). Done right the same set of disk images would boot on all platforms and have a common CBIOS that would be different than the Altair CBIOS by just a few ifdefs.
Would be very compelling I think
Dave
What does it do next? I have't studied the prop I/O schematic closely, but can you take over the bus and write to the ram chip?
But if not, there still could be ways to exchange information back and forth with the eprom boot program (written in assembly I presume) doing OUT and IN instructions to ports that the prop IO can trap and recognise. If you can send a byte and receive a byte, then you can port CP/M over and populate the memory.
Then, like you say, you can load different versions of CP/M directly from simh images. Is this on the right track?
Yes SIMH can handle different format/geometry disks but only for hard disks. The CPM/2 , CPM/3, MP/M and other bootable disks from the SIMH site are all Altair 1M floppies disks. I don't see being able to drop them into SIMH, PropIO, Morpeus, ZiCog unless they are all Altair compatible. At least as far as floppy drive A:
I don't see Peter wanting to move away from Altair emulation, that the whole point is it not.
Dr_A: As I said that DRIVE__1M file is not really a disk image, it does not have boot tracks, directories, etc. It just holds the content of 64K RAM as we would like it to be after Z80 reset. So basically it's all zeros accept for the BOOTROM code at FF00. Or it could contain a complete image of CP/M ready to run.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Post Edited (heater) : 8/25/2009 6:34:31 AM GMT