Stand-alone Programmer?

Seairth

I'm looking for a stand-alone programmer (preferably console-based) that will take a .binary or .eeprom file and load it. Most importantly, I don't want it to validate the image at all. Just upload it. Does anything like that already exist? If not, I don't suppose there's documentation (other than reading booter.src) for the programming protocol.

SRLM

Does propeller-load validate? If it doesn't, then that seems to fit your requirements.

Seairth

SRLM wrote: »

Does propeller-load validate? If it doesn't, then that seems to fit your requirements.

I hadn't tried it because the documentation indicated that an ELF image was required. If that's not the case, I'll download it and try it.

Cluso99

There are a few open source loaders...
Catalina's Payload
Heater recompiled a PropLoader for Raspberry Pi and also the Routers based on OpenWRT.
IIRC there is also a Prop Loader written in VB that Parallax released
There is also a Propeller PASM example.

I also have modules in my Propeller OS that will take a .BIN or .EEP image and write to the EEPROM or load into RAM.

SRLM

The propeller-load tool supports .binary:

[ -s ]            write a spin .binary file for use with the Propeller Tool

Seairth

SRLM wrote: »

The propeller-load tool supports .binary:

[ -s ]            write a spin .binary file for use with the Propeller Tool

Right, but the Propeller Tool verifies the image. I need a tool that will load binary without verifying the image.

SRLM

I'm sorry, I was wrong: the -s file appears to convert the .elf file to a .binary file. propeller-load can take the .binary file directly without any problems. There's no Propeller Tool involved.

$ propeller-load -r -t main.beta2.binary

Cluso99

Seairth wrote: »

Right, but the Propeller Tool verifies the image. I need a tool that will load binary without verifying the image.

Cannot recall if the P1 bootloader verifies the image in hub ram with the checksum in hub $0005.
Would a pc program to "touch" the checksum do the job? Think I may have done this when I was laying around with using an 4/8KB EEPROM

Ale

There was a python based loader posted ages ago... here the code... (I stumbled upon it some days ago...)
No idea if and how it works

#!/usr/bin/env python
"""\
Parallax Propeller code uploader
Copyright (C) 2007 Remy Blank

This file is part of PropTools.

This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation, version 2.

This program is distributed in the hope that it will be useful, but 
WITHOUT ANY WARRANTY; without even the implied warranty of 
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General 
Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA
"""

import optparse
import os
import time
import serial

# Processor constants
lfsrRequestLen = 250
lfsrReplyLen = 250
lfsrSeed = ord("P")
cmdShutdown = 0
cmdLoadRamRun = 1
cmdLoadEeprom = 2
cmdLoadEepromRun = 3

# Platform defaults
defSerial = {
    "posix": "/dev/ttyUSB0",
    "nt": "COM1",
}


def lfsr(seed):
    """Generate bits from 8-bit LFSR with taps at 0xB2."""
    while True:
        yield seed & 0x01
        seed = ((seed << 1) & 0xfe) | (((seed >> 7) ^ (seed >> 5) ^ (seed >> 4) ^ (seed >> 1)) & 1)


def encodeLong(value):
    """Encode a 32-bit long as short/long pulses."""
    result = []
    for i in range(10):
        result.append(chr(0x92 | (value & 0x01) | ((value & 2) << 2) | ((value & 4) << 4)))
        value >>= 3
    result.append(chr(0xf2 | (value & 0x01) | ((value & 2) << 2)))
    return "".join(result)


def doNothing(msg):
    """Do nothing progress callback."""
    pass
    

class LoaderError(Exception): pass


class Loader(object):
    """Propeller code uploader."""
    eepromSize = 32768
    
    def __init__(self, port):
        self.serial = serial.Serial(baudrate=115200, timeout=0)
        self.serial.port = port
        
    # High-level functions
    def getVersion(self, progress=doNothing):
        """Connect to the Propeller and return its version."""
        self.open()
        try:
            version = self.connect()
            self.writeLong(cmdShutdown)
            time.sleep(0.010)
            self.reset()
            return version
        finally:
            self.close()
        
    def upload(self, code=None, path=None, eeprom=False, run=True, progress=doNothing):
        """Connect to the Propeller and upload code to RAM or EEPROM."""
        if path is not None:
            f = open(path, "rb")
            try:
                code = f.read()
            finally:
                f.close() 
        self.open()
        try:
            version = self.connect()
            progress("Connected (version=%d)" % version)
            self.sendCode(code, eeprom, run, progress)
        finally:
            self.close()
    
    # Low-level functions
    def open(self):
        self.serial.open()
    
    def close(self):
        self.serial.close()
        
    def reset(self):
        self.serial.flushOutput()
        self.serial.setDTR(1)
        time.sleep(0.025)
        self.serial.setDTR(0)
        time.sleep(0.090)
        self.serial.flushInput()
        
    def calibrate(self):
        self.writeByte(0xf9)
        
    def connect(self):
        self.reset()
        self.calibrate()
        seq = []
        for (i, value) in zip(range(lfsrRequestLen + lfsrReplyLen), lfsr(lfsrSeed)):
            seq.append(value)
        self.serial.write("".join(chr(each | 0xfe) for each in seq[0:lfsrRequestLen]))
        self.serial.write(chr(0xf9) * (lfsrReplyLen + 8))
        for i in range(lfsrRequestLen, lfsrRequestLen + lfsrReplyLen):
            if self.readBit(False, 0.100) != seq[i]:
                raise LoaderError("No hardware found")
        version = 0
        for i in range(8):
            version = ((version >> 1) & 0x7f) | ((self.readBit(False, 0.050) << 7))
        return version

    def binToEeprom(self, code):
        if len(code) > self.eepromSize - 8:
            raise LoaderError("Code too long for EEPROM (max %d bytes)" % (self.eepromSize - 8))
        dbase = ord(code[0x0a]) + (ord(code[0x0b]) << 8)
        if dbase > self.eepromSize:
            raise LoaderError("Invalid binary format")
        code += "".join(chr(0x00) * (dbase - 8 - len(code)))
        code += "".join(chr(each) for each in [0xff, 0xff, 0xf9, 0xff, 0xff, 0xff, 0xf9, 0xff])
        code += "".join(chr(0x00) * (self.eepromSize - len(code)))
        return code
        
    def sendCode(self, code, eeprom=False, run=True, progress=doNothing):
        if len(code) % 4 != 0:
            raise LoaderError("Invalid code size: must be a multiple of 4")
        if eeprom and len(code) < self.eepromSize:
            code = self.binToEeprom(code)
        checksum = reduce(lambda a, b: a + b, (ord(each) for each in code))
        if not eeprom:
            checksum += 2 * (0xff + 0xff + 0xf9 + 0xff)
        checksum &= 0xff
        if checksum != 0:
            raise LoaderError("Code checksum error: 0x%.2x" % checksum)
        command = [cmdShutdown, cmdLoadRamRun, cmdLoadEeprom, cmdLoadEepromRun][eeprom * 2 + run]
        self.writeLong(command)
        if not eeprom and not run:
            return
        self.writeLong(len(code) // 4)
        progress("Sending code (%d bytes)" % len(code))
        i = 0
        while i < len(code): 
            self.writeLong(ord(code[i]) | (ord(code[i + 1]) << 8) | (ord(code[i + 2]) << 16) | (ord(code[i + 3]) << 24))
            i += 4
        if self.readBit(True, 8) == 1:
            raise LoaderError("RAM checksum error")
        if eeprom:
            progress("Programming EEPROM")
            if self.readBit(True, 5) == 1:
                raise LoaderError("EEPROM programming error")
            progress("Verifying EEPROM")
            if self.readBit(True, 2.5) == 1:
                raise LoaderError("EEPROM verification error")

    # Lowest-level functions
    def writeByte(self, value):
        self.serial.write(chr(value))
        
    def writeLong(self, value):
        self.serial.write(encodeLong(value))
        
    def readBit(self, echo, timeout):
        start = time.time()
        while time.time() - start < timeout:
            if echo:
                self.writeByte(0xf9)
                time.sleep(0.025)
            c = self.serial.read(1)
            if c:
                if c in (chr(0xfe), chr(0xff)):
                    return ord(c) & 0x01
                else:
                    raise LoaderError("Bad reply")
        raise LoaderError("Timeout error")


def upload(serial, path, eeprom=False, run=True, progress=doNothing):
    """Upload file on given serial port."""
    loader = Loader(serial)
    progress("Uploading %s" % path)
    loader.upload(path=path, eeprom=eeprom, run=run, progress=progress)
    progress("Done")

    
def watchUpload(serial, path, delay, eeprom=False, run=True, progress=doNothing):
    """Upload file on given serial port, and keep watching for changes and uploading."""
    loader = Loader(serial)
    firstLoop = True
    mtime = None
    while True:
        try:
            prevMTime = mtime
            try:
                mtime = os.stat(path).st_mtime
            except OSError:
                mtime = None
            if (mtime is not None) and (mtime != prevMTime):
                if not firstLoop:
                    progress("File change detected")
                    time.sleep(delay)
                progress("Uploading %s" % path)
                loader.upload(path=path, eeprom=eeprom, run=run, progress=progress)
                progress("Done\n")
            else:
                time.sleep(1)
        except LoaderError, e:
            progress(str(e) + "\n")
        firstLoop = False


class HelpFormatter(optparse.IndentedHelpFormatter):
    """Slightly customized option help formatter"""
    def format_usage(self, usage):
        return "Usage: %s\n" % usage
    
    def format_heading(self, heading):
        if heading == "options":
            heading = "Options"
        return optparse.IndentedHelpFormatter.format_heading(self, heading)
        
    def format_description(self, description):
        if not description:
            return ""
        return description


def printStatus(msg):
    """Print status messages."""
    print msg
    

def main(argv):
    """Execute command-line program."""
    import sys
    
    parser = optparse.OptionParser(
        prog=os.path.basename(argv[0]),
        usage="%prog [options] path",
        description="Parallax Propeller uploader\n",
        formatter=HelpFormatter(),
        add_help_option=False,
    )
    try:
        import PropTools
        parser.description += """\
This program is part of %(project)s %(version)s (%(date)s)
%(copyright)s
""" % PropTools._metadata_.__dict__
        parser.version = "%prog " + PropTools._metadata_.version
    except ImportError:
        parser.description += """\
Copyright (C) 2007 Remy Blank
"""

    parser.add_option("-d", "--delay", dest="delay", nargs=1, type="float", metavar="N", default=1.0,
        help="In watch mode, wait N seconds after detecting a file change before uploading. The default is %default.")
    parser.add_option("-e", "--eeprom", action="store_true", dest="eeprom", default=None,
        help="Program device EEPROM. The default is to program the EEPROM only if the path ends with '.eeprom'.")
    parser.add_option("-h", "--help", action="help", 
        help="Show this help message and exit.")
    parser.add_option("-n", "--no-run", action="store_false", dest="run", default=True,
        help="Don't run the code after upload.")
    parser.add_option("-r", "--ram", action="store_false", dest="eeprom",
        help="Program device RAM. The default is to program the RAM except if the path ends with '.eeprom'.")
    parser.add_option("-s", "--serial", dest="serial", nargs=1, type="string", metavar="DEVICE", default=defSerial.get(os.name, "none"),
        help="Select the serial port device. The default is %default.")
    parser.add_option("",   "--version", action="version",
        help="Show the program version and exit.")
    parser.add_option("-w", "--watch", action="store_true", dest="watch", default=False,
        help="Continuously watch the file and upload it if it changes.")

    (options, args) = parser.parse_args(argv[1:])
    if len(args) != 1:
        sys.stderr.write("Invalid number of arguments\n")
        parser.print_help(sys.stderr)
        return 2
    path = args[0]
    if options.eeprom is None:
        options.eeprom = path.endswith(".eeprom")
    
    try:
        if options.watch:
            watchUpload(options.serial, path, options.delay, options.eeprom, options.run, printStatus)
        else:
            upload(options.serial, path, options.eeprom, options.run, printStatus)
    except (SystemExit, KeyboardInterrupt):
        return 3
    except Exception, e:
        sys.stderr.write(str(e) + "\n")
        return 1


if __name__ == "__main__":
    import sys
    sys.exit(main(sys.argv))

Seairth

Ale wrote: »

There was a python based loader posted ages ago... here the code... (I stumbled upon it some days ago...)
No idea if and how it works

#!/usr/bin/env python
"""\
Parallax Propeller code uploader
Copyright (C) 2007 Remy Blank

This file is part of PropTools.

This program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the
Free Software Foundation, version 2.

This program is distributed in the hope that it will be useful, but 
WITHOUT ANY WARRANTY; without even the implied warranty of 
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General 
Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software Foundation,
Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA
"""

import optparse
import os
import time
import serial

# Processor constants
lfsrRequestLen = 250
lfsrReplyLen = 250
lfsrSeed = ord("P")
cmdShutdown = 0
cmdLoadRamRun = 1
cmdLoadEeprom = 2
cmdLoadEepromRun = 3

# Platform defaults
defSerial = {
    "posix": "/dev/ttyUSB0",
    "nt": "COM1",
}


def lfsr(seed):
    """Generate bits from 8-bit LFSR with taps at 0xB2."""
    while True:
        yield seed & 0x01
        seed = ((seed << 1) & 0xfe) | (((seed >> 7) ^ (seed >> 5) ^ (seed >> 4) ^ (seed >> 1)) & 1)


def encodeLong(value):
    """Encode a 32-bit long as short/long pulses."""
    result = []
    for i in range(10):
        result.append(chr(0x92 | (value & 0x01) | ((value & 2) << 2) | ((value & 4) << 4)))
        value >>= 3
    result.append(chr(0xf2 | (value & 0x01) | ((value & 2) << 2)))
    return "".join(result)


def doNothing(msg):
    """Do nothing progress callback."""
    pass
    

class LoaderError(Exception): pass


class Loader(object):
    """Propeller code uploader."""
    eepromSize = 32768
    
    def __init__(self, port):
        self.serial = serial.Serial(baudrate=115200, timeout=0)
        self.serial.port = port
        
    # High-level functions
    def getVersion(self, progress=doNothing):
        """Connect to the Propeller and return its version."""
        self.open()
        try:
            version = self.connect()
            self.writeLong(cmdShutdown)
            time.sleep(0.010)
            self.reset()
            return version
        finally:
            self.close()
        
    def upload(self, code=None, path=None, eeprom=False, run=True, progress=doNothing):
        """Connect to the Propeller and upload code to RAM or EEPROM."""
        if path is not None:
            f = open(path, "rb")
            try:
                code = f.read()
            finally:
                f.close() 
        self.open()
        try:
            version = self.connect()
            progress("Connected (version=%d)" % version)
            self.sendCode(code, eeprom, run, progress)
        finally:
            self.close()
    
    # Low-level functions
    def open(self):
        self.serial.open()
    
    def close(self):
        self.serial.close()
        
    def reset(self):
        self.serial.flushOutput()
        self.serial.setDTR(1)
        time.sleep(0.025)
        self.serial.setDTR(0)
        time.sleep(0.090)
        self.serial.flushInput()
        
    def calibrate(self):
        self.writeByte(0xf9)
        
    def connect(self):
        self.reset()
        self.calibrate()
        seq = []
        for (i, value) in zip(range(lfsrRequestLen + lfsrReplyLen), lfsr(lfsrSeed)):
            seq.append(value)
        self.serial.write("".join(chr(each | 0xfe) for each in seq[0:lfsrRequestLen]))
        self.serial.write(chr(0xf9) * (lfsrReplyLen + 8))
        for i in range(lfsrRequestLen, lfsrRequestLen + lfsrReplyLen):
            if self.readBit(False, 0.100) != seq[i]:
                raise LoaderError("No hardware found")
        version = 0
        for i in range(8):
            version = ((version >> 1) & 0x7f) | ((self.readBit(False, 0.050) << 7))
        return version

    def binToEeprom(self, code):
        if len(code) > self.eepromSize - 8:
            raise LoaderError("Code too long for EEPROM (max %d bytes)" % (self.eepromSize - 8))
        dbase = ord(code[0x0a]) + (ord(code[0x0b]) << 8)
        if dbase > self.eepromSize:
            raise LoaderError("Invalid binary format")
        code += "".join(chr(0x00) * (dbase - 8 - len(code)))
        code += "".join(chr(each) for each in [0xff, 0xff, 0xf9, 0xff, 0xff, 0xff, 0xf9, 0xff])
        code += "".join(chr(0x00) * (self.eepromSize - len(code)))
        return code
        
    def sendCode(self, code, eeprom=False, run=True, progress=doNothing):
        if len(code) % 4 != 0:
            raise LoaderError("Invalid code size: must be a multiple of 4")
        if eeprom and len(code) < self.eepromSize:
            code = self.binToEeprom(code)
        checksum = reduce(lambda a, b: a + b, (ord(each) for each in code))
        if not eeprom:
            checksum += 2 * (0xff + 0xff + 0xf9 + 0xff)
        checksum &= 0xff
        if checksum != 0:
            raise LoaderError("Code checksum error: 0x%.2x" % checksum)
        command = [cmdShutdown, cmdLoadRamRun, cmdLoadEeprom, cmdLoadEepromRun][eeprom * 2 + run]
        self.writeLong(command)
        if not eeprom and not run:
            return
        self.writeLong(len(code) // 4)
        progress("Sending code (%d bytes)" % len(code))
        i = 0
        while i < len(code): 
            self.writeLong(ord(code[i]) | (ord(code[i + 1]) << 8) | (ord(code[i + 2]) << 16) | (ord(code[i + 3]) << 24))
            i += 4
        if self.readBit(True, 8) == 1:
            raise LoaderError("RAM checksum error")
        if eeprom:
            progress("Programming EEPROM")
            if self.readBit(True, 5) == 1:
                raise LoaderError("EEPROM programming error")
            progress("Verifying EEPROM")
            if self.readBit(True, 2.5) == 1:
                raise LoaderError("EEPROM verification error")

    # Lowest-level functions
    def writeByte(self, value):
        self.serial.write(chr(value))
        
    def writeLong(self, value):
        self.serial.write(encodeLong(value))
        
    def readBit(self, echo, timeout):
        start = time.time()
        while time.time() - start < timeout:
            if echo:
                self.writeByte(0xf9)
                time.sleep(0.025)
            c = self.serial.read(1)
            if c:
                if c in (chr(0xfe), chr(0xff)):
                    return ord(c) & 0x01
                else:
                    raise LoaderError("Bad reply")
        raise LoaderError("Timeout error")


def upload(serial, path, eeprom=False, run=True, progress=doNothing):
    """Upload file on given serial port."""
    loader = Loader(serial)
    progress("Uploading %s" % path)
    loader.upload(path=path, eeprom=eeprom, run=run, progress=progress)
    progress("Done")

    
def watchUpload(serial, path, delay, eeprom=False, run=True, progress=doNothing):
    """Upload file on given serial port, and keep watching for changes and uploading."""
    loader = Loader(serial)
    firstLoop = True
    mtime = None
    while True:
        try:
            prevMTime = mtime
            try:
                mtime = os.stat(path).st_mtime
            except OSError:
                mtime = None
            if (mtime is not None) and (mtime != prevMTime):
                if not firstLoop:
                    progress("File change detected")
                    time.sleep(delay)
                progress("Uploading %s" % path)
                loader.upload(path=path, eeprom=eeprom, run=run, progress=progress)
                progress("Done\n")
            else:
                time.sleep(1)
        except LoaderError, e:
            progress(str(e) + "\n")
        firstLoop = False


class HelpFormatter(optparse.IndentedHelpFormatter):
    """Slightly customized option help formatter"""
    def format_usage(self, usage):
        return "Usage: %s\n" % usage
    
    def format_heading(self, heading):
        if heading == "options":
            heading = "Options"
        return optparse.IndentedHelpFormatter.format_heading(self, heading)
        
    def format_description(self, description):
        if not description:
            return ""
        return description


def printStatus(msg):
    """Print status messages."""
    print msg
    

def main(argv):
    """Execute command-line program."""
    import sys
    
    parser = optparse.OptionParser(
        prog=os.path.basename(argv[0]),
        usage="%prog [options] path",
        description="Parallax Propeller uploader\n",
        formatter=HelpFormatter(),
        add_help_option=False,
    )
    try:
        import PropTools
        parser.description += """\
This program is part of %(project)s %(version)s (%(date)s)
%(copyright)s
""" % PropTools._metadata_.__dict__
        parser.version = "%prog " + PropTools._metadata_.version
    except ImportError:
        parser.description += """\
Copyright (C) 2007 Remy Blank
"""

    parser.add_option("-d", "--delay", dest="delay", nargs=1, type="float", metavar="N", default=1.0,
        help="In watch mode, wait N seconds after detecting a file change before uploading. The default is %default.")
    parser.add_option("-e", "--eeprom", action="store_true", dest="eeprom", default=None,
        help="Program device EEPROM. The default is to program the EEPROM only if the path ends with '.eeprom'.")
    parser.add_option("-h", "--help", action="help", 
        help="Show this help message and exit.")
    parser.add_option("-n", "--no-run", action="store_false", dest="run", default=True,
        help="Don't run the code after upload.")
    parser.add_option("-r", "--ram", action="store_false", dest="eeprom",
        help="Program device RAM. The default is to program the RAM except if the path ends with '.eeprom'.")
    parser.add_option("-s", "--serial", dest="serial", nargs=1, type="string", metavar="DEVICE", default=defSerial.get(os.name, "none"),
        help="Select the serial port device. The default is %default.")
    parser.add_option("",   "--version", action="version",
        help="Show the program version and exit.")
    parser.add_option("-w", "--watch", action="store_true", dest="watch", default=False,
        help="Continuously watch the file and upload it if it changes.")

    (options, args) = parser.parse_args(argv[1:])
    if len(args) != 1:
        sys.stderr.write("Invalid number of arguments\n")
        parser.print_help(sys.stderr)
        return 2
    path = args[0]
    if options.eeprom is None:
        options.eeprom = path.endswith(".eeprom")
    
    try:
        if options.watch:
            watchUpload(options.serial, path, options.delay, options.eeprom, options.run, printStatus)
        else:
            upload(options.serial, path, options.eeprom, options.run, printStatus)
    except (SystemExit, KeyboardInterrupt):
        return 3
    except Exception, e:
        sys.stderr.write(str(e) + "\n")
        return 1


if __name__ == "__main__":
    import sys
    sys.exit(main(sys.argv))

That;s fantastic! I hope this works (will check shortly). As it happens to be, I'm writing an assembler in Python, so this will fit nicely! Thanks!

Ale

I wrote recently a pasm assembler in python ;-)... I can post it if you want

Seairth

Ale wrote: »

There was a python based loader posted ages ago...

Here's an updated version of that script, which works with Python 3 (make sure to install PySerial). I've tested it with both the Propeller Demo Board and the P1V on the DE0-Nano. Note that I have changed the command-line a bit. You can:

• Get the version of the attached Propeller device
• Upload a binary file to RAM and run code
• Upload a binary or EEPROM to a 32K eeprom

NOTE: I'd really like to thank Remy Blank for doing the hard work on this. My contribution is minor compared to his.

#!/usr/bin/env python

# PyNut is free software: you can redistribute it and/or modify it under the terms
# of the GNU General Public License as published by the Free Software Foundation,
# either version 3 of the License, or (at your option) any later version. The
# software is distributed WITHOUT ANY WARRANTY; without even the implied warranty
# of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General
# Public License for more details.
# 
# You should have received a copy of the GNU General Public License along with
# the software.  If not, see <http://www.gnu.org/licenses/>.

# The following code is based on the "Parallax Propeller code uploader", written
# by Remy Blank.  The original code was release under GNU General Pulic License,
# version 2.  A copy of the original code may be found at
# http://forums.parallax.com/showthread.php/157773-Stand-alone-Programmer?p=1298179

import os
import time
import serial

# Processor constants
lfsrRequestLen = 250
lfsrReplyLen = 250
lfsrSeed = ord("P")

cmdShutdown = 0
cmdLoadRamRun = 1
cmdLoadEeprom = 2
cmdLoadEepromRun = 3

# Platform defaults
defSerial = {
    "posix": "/dev/ttyUSB0",
    "nt": "COM1",
}

def _lfsr(seed):
    """Generate bits from 8-bit LFSR with taps at 0xB2."""
    while True:
        yield seed & 0x01
        seed = ((seed << 1) & 0xfe) | (((seed >> 7) ^ (seed >> 5) ^ (seed >> 4) ^ (seed >> 1)) & 1)

def encode_long(value):
    """Encode a 32-bit long as short/long pulses."""
    result = bytearray()
    for i in range(10):
        result.append(0x92 | (value & 0x01) | ((value & 2) << 2) | ((value & 4) << 4))
        value >>= 3
    result.append(0xf2 | (value & 0x01) | ((value & 2) << 2))
    return result


def do_nothing(msg):
    """Default progress callback."""
    pass
    

class LoaderError(Exception): pass


class Loader(object):
    """Propeller code uploader."""
    eepromSize = 32768
    
    def __init__(self, port):
        self.serial = serial.Serial(baudrate=115200, timeout=0)
        self.serial.port = port
        
    # High-level functions
    def get_version(self, progress=do_nothing):
        """Connect to the Propeller and return its version."""
        self._open()
        try:
            version = self._connect()
            self._write_long(cmdShutdown)
            time.sleep(0.010)
            self._reset()
            return version
        finally:
            self._close()
        
    def upload(self, code=None, path=None, eeprom=False, run=True, progress=do_nothing):
        """Connect to the Propeller and upload code to RAM or EEPROM."""
        
        if path is not None:
            progress("Uploading {}".format(path))
            with open(path, "rb") as f:
                code = f.read()

        if len(code) % 4 != 0:
            raise LoaderError("Invalid code size: must be a multiple of 4")

        if eeprom and len(code) < self.eepromSize:
            code = self._bin_to_eeprom(code)

        checksum = sum(code)
        
        if not eeprom:
            checksum += 2 * (0xff + 0xff + 0xf9 + 0xff)

        checksum &= 0xff

        if checksum != 0:
            raise LoaderError("Code checksum error: 0x{:0>2x}".format(checksum))

        self._open()
        try:
            version = self._connect()
            progress("Connected (version={})".format(version))
            self._send_code(code, eeprom, run, progress)
        finally:
            self._close()
    
    # Low-level functions
    def _open(self):
        self.serial.open()
    
    def _close(self):
        self.serial.close()
        
    def _reset(self):
        self.serial.flushOutput()
        self.serial.setDTR(1)
        time.sleep(0.025)
        self.serial.setDTR(0)
        time.sleep(0.090)
        self.serial.flushInput()
        
    def _calibrate(self):
        self._write_byte(0xf9)
        
    def _connect(self):
        self._reset()
        self._calibrate()
        
        seq = []
        
        for (i, value) in zip(range(lfsrRequestLen + lfsrReplyLen), _lfsr(lfsrSeed)):
            seq.append(value)
        
        self.serial.write(bytes((each | 0xfe) for each in seq[0:lfsrRequestLen]))
        self.serial.write(bytes((0xf9,) * (lfsrReplyLen + 8)))
        
        for i in range(lfsrRequestLen, lfsrRequestLen + lfsrReplyLen):
            if self._read_bit(False, 0.100) != seq[i]:
                raise LoaderError("No hardware found")
        
        version = 0
        for i in range(8):
            version = ((version >> 1) & 0x7f) | ((self._read_bit(False, 0.050) << 7))
        
        return version

    def _bin_to_eeprom(self, code):
        if len(code) > self.eepromSize - 8:
            raise LoaderError("Code too long for EEPROM (max {} bytes)".format(self.eepromSize - 8))
        
        dbase = code[0x0a] + (code[0x0b] << 8)
        
        if dbase > self.eepromSize:
            raise LoaderError("Invalid binary format")
        
        eeprom = bytearray(code)
        eeprom += bytearray([0x00] * (dbase - 8 - len(code)))
        eeprom += bytearray([0xff, 0xff, 0xf9, 0xff] * 2)
        eeprom += bytearray([0x00] * int(self.eepromSize - len(code)))
        
        return eeprom
        
    def _send_code(self, code, eeprom=False, run=True, progress=do_nothing):
        command = [cmdShutdown, cmdLoadRamRun, cmdLoadEeprom, cmdLoadEepromRun][eeprom * 2 + run]
        
        self._write_long(command)
        
        if not eeprom and not run:
            return
        
        self._write_long(len(code) // 4)
        progress("Sending code ({} bytes)".format(len(code)))
        
        i = 0
        while i < len(code): 
            self._write_long(code[i] | (code[i + 1] << 8) | (code[i + 2] << 16) | (code[i + 3] << 24))
            i += 4
        
        if self._read_bit(True, 8) == 1:
            raise LoaderError("RAM checksum error")
        
        if eeprom:
            progress("Programming EEPROM")
            if self._read_bit(True, 5) == 1:
                raise LoaderError("EEPROM programming error")
        
            progress("Verifying EEPROM")
            if self._read_bit(True, 2.5) == 1:
                raise LoaderError("EEPROM verification error")

    # Lowest-level functions
    def _write_byte(self, value):
        self.serial.write(bytes((value,)))
        
    def _write_long(self, value):
        self.serial.write(encode_long(value))
        
    def _read_bit(self, echo, timeout):
        start = time.time()
        while time.time() - start < timeout:
            if echo:
                self._write_byte(0xf9)
                time.sleep(0.025)
            c = self.serial.read(1)
            if c:
                if c[0] in (0xfe, 0xff):
                    return c[0] & 0x01
                else:
                    raise LoaderError("Bad reply")
        raise LoaderError("Timeout error")

def get_version(serial):
    """Get the version of the connected Propeller chip."""

    loader = Loader(serial)
    print(loader.get_version())

def upload(serial, path, eeprom=False, run=True, progress=do_nothing):
    """Upload file on given serial port."""

    loader = Loader(serial)
    loader.upload(path=path, eeprom=eeprom, run=run, progress=progress)
    progress("Done")

def printStatus(msg):
    """Print status messages."""
    print(msg)
    
def _action_get_version(args):
    get_version(args.serial)

def _action_upload(args):
    path = args.filename

    if path.endswith(".eeprom"):
        args.destination = "EEPROM"
    else:
        args.destination = args.destination.upper()
    
    try:
        upload(args.serial, path, (args.destination == "EEPROM"), args.run, printStatus)
    except (SystemExit, KeyboardInterrupt):
        return 3
    except Exception as e:
        sys.stderr.write(str(e) + "\n")
        return 1

if __name__ == "__main__":
    import sys
    import argparse

    parser = argparse.ArgumentParser()

    parser.add_argument("-v",   "--version", action="version", version="%(prog)s 0.1",
                        help="Show the program version and exit.")

    subparsers = parser.add_subparsers()

    parser_v = subparsers.add_parser("version")
    parser_v.set_defaults(action=_action_get_version)
    parser_v.add_argument("-s", "--serial", dest="serial", type=str, metavar="DEVICE", default=defSerial.get(os.name, "none"),
                          help="Select the serial port device. The default is %(default)s.")

    parser_u = subparsers.add_parser("upload")
    parser_u.set_defaults(action=_action_upload)
    parser_u.add_argument("filename", type=str,
                          help="Binary file to be uploaded.")
    parser_u.add_argument("-d", "--destination", type=str, default="RAM", choices=["RAM", "EEPROM"],
                          help="Upload to RAM or to EEPROM.  The default is %(default)s.")
    parser_u.add_argument("-n", "--no-run", action="store_false", dest="run", default=True,
                          help="Don't run the code after upload.")
    parser_u.add_argument("-s", "--serial", dest="serial", type=str, metavar="DEVICE", default=defSerial.get(os.name, "none"),
                          help="Select the serial port device. The default is %(default)s.")

    args = parser.parse_args()

    if hasattr(args, "action"):
        exit(args.action(args))

    parser.print_usage()

T Chap

I use an app with bstl in it that works fine on windows or mac.

Seairth

T Chap wrote: »

I use an app with bstl in it that works fine on windows or mac.

I hadn't heard of BST. Here's the link for anyone else that may be interested: http://www.fnarfbargle.com/bst.html

T Chap

Heck, I just realized this was not the regular propeller forum. I have no ideas on verilog use.

Seairth

T Chap wrote: »

Heck, I just realized this was not the regular propeller forum. I have no ideas on verilog use.

That's okay. It may still be useful. I haven't looked into it yet to see how customizable it is (if at all).

The reason I asked for stand-alone programmers in the first place is because I wanted something that was more liberal than the Parallax Propeller Tool with respect to what could be uploaded, thereby allowing an easier time of changing the P1V code in ways that would result in incompatible binaries with the current Propeller.

Seairth

While reworking the python code above, I encountered a little gem in the way that the serial communications is implemented. For those who are interested in these sorts of things, I wrote it up in a blog post.

jmg

Seairth wrote: »

While reworking the python code above, I encountered a little gem in the way that the serial communications is implemented. For those who are interested in these sorts of things, I wrote it up in a blog post.

Nicely done.

What is the upper Baud limit, for given Prop Clocks, P1 and P1V ?

I think a P1 boots from RC, (slow, and needs margins) but a P1V can boot faster ?

Most modern USB-serial devices allow many more actual BAUD choice than generic values - commonly seen is a 12M/N baud granularity.
I guess Python can also do this, as almost PC SW will pass baud as the value, & the driver works out how close it can get.

Seairth

jmg wrote: »

Nicely done.

What is the upper Baud limit, for given Prop Clocks, P1 and P1V ?

I think a P1 boots from RC, (slow, and needs margins) but a P1V can boot faster ?

Most modern USB-serial devices allow many more actual BAUD choice than generic values - commonly seen is a 12M/N baud granularity.
I guess Python can also do this, as almost PC SW will pass baud as the value, & the driver works out how close it can get.

Propeller starts up with RCFAST, which can range between 8MHz and 20MHz (typ. 12MHz). The P1V, on the other hand, behaves a bit differently, I think. Based on tim.v, the P1V initially runs at 10MHz (which is roughly close to 12MHz). You can change this by modifying hub.v to set the initial value of "cfg" on reset.

Based on the code in loader.src, it looks like the limiting factor is that you need at least 64 clock cycles from the beginning of the stop bit to the beginning of the next start bit to ensure accurate timing measurement. Going from this number, the max baud rates would be:

Propeller
8MHz (RCFAST worst case) 125,000 bps
12MHz (RCFAST typical case): 187,500 bps
20MHz (RCFAST best case): 312,500 bps

P1V
10MHz (simulated RCFAST) : 156,250 bps
80MHz (modified hub.v): 1,250,000 bps

The PropPlug can handle speeds up to 3Mbps, so a modified hub.v would allow for a 1.25Mbps transfer rate. Note, however, that loader.src would also have to be modified to adjust the timeouts, or else the loader would time out in 1/4 the current time.


All this aside, though, I think you could safely use this technique in your own code for speeds up to 1Mbps (with effective data throughput of just under 1/3 the baud rate). And if you have a bit of extra registers, you might be able to unroll the receive code, which would give you a significant bump in speed. This assumes, of course, that the begin-of-stop-bit-to-begin-of-start-bit time is still the most computationally intensive part.

In the end, is it practical to use this "1N1" technique? Probably only under limited conditions: you only have access to serial (e.g. communication through the PropPlug), you can't run actual full duplex serial with 8 bits of data per frame, you need a vers small block of PASM, etc.

Cluso99

Thanks Seairth, fantastic explanation.

Here is an idea that would extend the timing window, and perhaps could enable overall faster downloading for larger programs...

Download a short "boot" program which
1. Launches, enables the xtal, and waits for a new download protocol at >= 115,200 baud.
The download program then sends data faster using this new protocol. A better CRC could be used.
Even at 115,200 baud, the second stage could download a full 8 bits per byte which is 11/4 = 2.75 times faster.

The loader could be smart and enable this 2 stage download only if the code length is above a certain size.

This would help when downloading over wireless which tends to use lower rates.

The other issue is using cheap USB-Serial dongles which either do not have DTR or RTS, and/or just don't have the transistor reset circuit.

jmg

Thanks for the numbers.

Seairth wrote: »

In the end, is it practical to use this "1N1" technique? Probably only under limited conditions: you only have access to serial (e.g. communication through the PropPlug), you can't run actual full duplex serial with 8 bits of data per frame, you need a vers small block of PASM, etc.

I think a key element of the 1N1 is around the clock tolerance - this design even allows for significant CLK drift after the calibrate phase.
Most 8N1 Auto-baud designs send a 'U' or similar, and then expect a reasonably stable clock from the calibrate char.


A benefit of sticking with 1N1 is it works on all Props, but pushing the peak baud up as high as is practical/reliable gives a simple single number to change.
At the highest BAUD rates, on parts like FT232R, the 'sticker' value is not quite attainable, as more stop bits are inserted.
HS-USB parts can deliver the throughput, but they are in larger packages, more $$, and need a crystal.

pik33

jmg wrote: »

Nicely done.



I think a P1 boots from RC, (slow, and needs margins) but a P1V can boot faster ?

P1V can boot directly from its RAM with predefined content and this is the fastest way to boot it.

Seairth

pik33 wrote: »

P1V can boot directly from its RAM with predefined content and this is the fastest way to boot it.

As long as you don't mind the multi-minute compile and programming in Quartus every time you want to change that code.

pik33

The Quartus has option - update memory files - which run in seconds. The problem is it will update only mif/hex files, and not the $readmemh ones, but it is only conversion/syntax problem.

And the boot time is only matter when you are using the system and not debugging it.