Shop OBEX P1 Docs P2 Docs Learn Events
A Propeller-WebTool Framework for Compiled Languages? - Page 6 — Parallax Forums

A Propeller-WebTool Framework for Compiled Languages?

13468921

Comments

  • TappermanTapperman Posts: 319
    edited 2014-01-05 14:55
    Cluso99 wrote: »
    BTW what did it cist?

    Amazon shows it for $99.00

    ... Tim
  • max72max72 Posts: 1,155
    edited 2014-01-05 15:14
  • Heater.Heater. Posts: 21,230
    edited 2014-01-05 16:20
    Cluso.
    Are there any tools for the SSD to find out how many writes have occurred or other statistics?
    I would presume there is some form of algorithm used for wear levelling like on SD cards.
    Yes and yes.

    Strangely enough I was just investigating this. SSD's support the same SMART diagnostics as regular hard drives. Under Linux there is a package called smartmontools which contains a command line program to pull diagnostics off the drive. There is all kinds of interesting information in there including "Wear_Leveling_Count" which is shown as a percentage and drops down from 100 to zero as the possibilities for wear levelling diminish. There is also a general drive health estimate.

    The smartmontools also contains a background process that can check drive health at regular intervals and notify you, email or whatever, if something starts to look bad.
    I also presume that the sw is smart enough to not rewrite is no changes have been done.
    Perhaps. I would not be so sure. An updated file will not be written back to the same logical blocks it came from. The SSD has no idea about files anyway.

    That 120GB SSD cost 122 Euros. About ten times more per GB than a hard drive.

    Still we cannot rely on those diagnostics. We have to be ready for catastrophic failure at any time. Backup, backup, backup!

    I just wanted to get a handle on expected life span due to the wear issue alone. The question is: "Is there actually much benefit in moving swap and logs etc to a regular disk or is it just hassle for nothing?

    Here is the health summary from smartctl:
    # smartctl -H /dev/sda
    smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-4-amd64] (local build)
    Copyright (C) 2002-11 by Bruce Allen, http://smartmontools.sourceforge.net
    
    === START OF READ SMART DATA SECTION ===
    SMART overall-health self-assessment test result: PASSED
    
    Here is the diagnostic data, note the "Wear_Leveling_Count":
    smartctl 5.41 2011-06-09 r3365 [x86_64-linux-3.2.0-4-amd64] (local build)
    Copyright (C) 2002-11 by Bruce Allen, [URL]http://smartmontools.sourceforge.net[/URL]
    
    === START OF INFORMATION SECTION ===
    Device Model:     Samsung SSD 840 EVO 120GB
    Serial Number:    S1D7NEADB04170R
    LU WWN Device Id: 5 002538 8500c3b0e
    Firmware Version: EXT0BB0Q
    User Capacity:    120,034,123,776 bytes [120 GB]
    Sector Size:      512 bytes logical/physical
    Device is:        Not in smartctl database [for details use: -P showall]
    ATA Version is:   8
    ATA Standard is:  ATA-8-ACS revision 4c
    Local Time is:    Mon Jan  6 01:33:56 2014 EET
    SMART support is: Available - device has SMART capability.
    SMART support is: Enabled
    
    === START OF READ SMART DATA SECTION ===
    SMART overall-health self-assessment test result: PASSED
    
    General SMART Values:
    Offline data collection status:  (0x00)    Offline data collection activity
                        was never started.
                        Auto Offline Data Collection: Disabled.
    Self-test execution status:      (   0)    The previous self-test routine completed
                        without error or no self-test has ever 
                        been run.
    Total time to complete Offline 
    data collection:         ( 4200) seconds.
    Offline data collection
    capabilities:              (0x53) SMART execute Offline immediate.
                        Auto Offline data collection on/off support.
                        Suspend Offline collection upon new
                        command.
                        No Offline surface scan supported.
                        Self-test supported.
                        No Conveyance Self-test supported.
                        Selective Self-test supported.
    SMART capabilities:            (0x0003)    Saves SMART data before entering
                        power-saving mode.
                        Supports SMART auto save timer.
    Error logging capability:        (0x01)    Error logging supported.
                        General Purpose Logging supported.
    Short self-test routine 
    recommended polling time:      (   2) minutes.
    Extended self-test routine
    recommended polling time:      (  70) minutes.
    SCT capabilities:            (0x003d)    SCT Status supported.
                        SCT Error Recovery Control supported.
                        SCT Feature Control supported.
                        SCT Data Table supported.
    
    SMART Attributes Data Structure revision number: 1
    Vendor Specific SMART Attributes with Thresholds:
    ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
      5 Reallocated_Sector_Ct   0x0033   100   100   010    Pre-fail  Always       -       0
      9 Power_On_Hours          0x0032   099   099   000    Old_age   Always       -       28
     12 Power_Cycle_Count       0x0032   099   099   000    Old_age   Always       -       12
    [B]177 Wear_Leveling_Count     0x0013   100   100   000    Pre-fail  Always       -       0[/B]
    179 Used_Rsvd_Blk_Cnt_Tot   0x0013   100   100   010    Pre-fail  Always       -       0
    181 Program_Fail_Cnt_Total  0x0032   100   100   010    Old_age   Always       -       0
    182 Erase_Fail_Count_Total  0x0032   100   100   010    Old_age   Always       -       0
    183 Runtime_Bad_Block       0x0013   100   100   010    Pre-fail  Always       -       0
    187 Reported_Uncorrect      0x0032   100   100   000    Old_age   Always       -       0
    190 Airflow_Temperature_Cel 0x0032   070   063   000    Old_age   Always       -       30
    195 Hardware_ECC_Recovered  0x001a   200   200   000    Old_age   Always       -       0
    199 UDMA_CRC_Error_Count    0x003e   099   099   000    Old_age   Always       -       1
    235 Unknown_Attribute       0x0012   099   099   000    Old_age   Always       -       6
    241 Total_LBAs_Written      0x0032   099   099   000    Old_age   Always       -       149522384
    
    SMART Error Log Version: 1
    No Errors Logged
    
    SMART Self-test log structure revision number 1
    No self-tests have been logged.  [To run self-tests, use: smartctl -t]
    
    SMART Selective self-test log data structure revision number 1
     SPAN  MIN_LBA  MAX_LBA  CURRENT_TEST_STATUS
        1        0        0  Not_testing
        2        0        0  Not_testing
        3        0        0  Not_testing
        4        0        0  Not_testing
        5        0        0  Not_testing
    Selective self-test flags (0x0):
      After scanning selected spans, do NOT read-scan remainder of disk.
    If Selective self-test is pending on power-up, resume after 0 minute delay.
    
  • Heater.Heater. Posts: 21,230
    edited 2014-01-05 16:24
    OK, That's wierd. I have two google-chrome browsers installed now. Both version: 31.0.1650.63.
    The one packaged by Debian removes all line feeds from posts which makes life impossible.
    The one from Google inserts two new lines for every one I enter!
  • jazzedjazzed Posts: 11,803
    edited 2014-01-05 18:00
    Heater. wrote: »
    OK, That's wierd. I have two google-chrome browsers installed now. Both version: 31.0.1650.63.
    The one packaged by Debian removes all line feeds from posts which makes life impossible.
    The one from Google inserts two new lines for every one I enter!
    I have the second problem. I always thought it was just the forum editor.

    The other annoying editor bit has been pasting text .... until recently. I noticed that there is a right-click "paste as plain text" option now. Maybe it was there all along, I have no idea. Changing text colors is troublesome because I use the "Parallax dark" profile to ease the strain on my eyes.
  • TorTor Posts: 2,010
    edited 2014-01-06 03:36
    Heater. wrote: »
    [SSD]What worries me now is wearing the thing out. I think I'd like to move my swap partition, log files etc to a regular hard drive. Or do I worry too much?
    I'm not too worried about wear on SSD drives, what is worrying is what happens when they fail: You lose everything. Not like regular hard drives which often start to show symptoms before they fail, and even when they fail suddenly it's usually possible to extract at least a good part of the content. When SSD and other flash memory fail then all is lost. And there is no warning.
    The other day a Windows tablet managed to destroy my 32GB USB3.0 stick.. first time that ever happened to me. I can't get a single byte out of it. Fortunately I'm in the habit of backing up (in triple) these days. But even so, it's frustrating. You kind of know where you are with HDDs.. even though these things also fail.

    -Tor
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 05:16
    Tor,

    You lose everything. Not like regular hard drives which often start to show symptoms before they fail...
    I think the key word here is "often".

    I have had hard drives fail on me in all kinds of ways. Some times with some warning like starting to make strange noises, or odd errors occurring. Some times just instant death. Like when a controller fails.

    Back in the day it was easy to corrupt a Linux file system, by a sudden power loss say, and have it become totally unusable. Even if the disk itself was perfectly OK and just needed reformatting to use again. Recovering data from such a corrupt FS is not a simple task.

    So that "often" means you have to be prepared for the sudden death situation with hard drives anyway. Having this as the major failure mode on SSD's does not change much.

    That and the fact that a major case of data loss is user error, over writing a file or "rm -rf *" and the need for regular backups is there anyway. Often those "soft" corruptions are more insidious as you may not even notice it has happened until you have backed up the corrupted files a few times!

    My gut instinct is that wear caused by writing swap, logs, browser cache, etc might be an issue. But I'm going to go against my gut and just run the thing as it is. Let's see what happens!
  • rabaggettrabaggett Posts: 96
    edited 2014-01-06 06:07
    [QUOTE=Tor;1232969
    The other day a Windows tablet managed to destroy my 32GB USB3.0 stick.. first time that ever happened to me. I can't get a single byte out of it.

    -Tor[/QUOTE]

    Agree with your backup plan.

    Just thought I'd mention, I've saved the data from several sticks belonging to people less diligent than you by transplanting the flash chips onto an identical stick. The file system was still lost in a couple of instances, but all files still recovered with PhotoRec. It seems the controllers in those sticks are somewhat fragile.. I've never encountered one where the flash was actually 'worn out'.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 06:29
    rabaggett,

    That sounds like a very fine a delicate operation. Well done.

    I wonder what happens to those controllers. I mean chips is chips, right, they generally run for decades unless they are subject to something catastrophic. Do they get zapped by static on the USB connections or what? Why would an SSD controller die, it's in a nice safe environment?
  • prof_brainoprof_braino Posts: 4,313
    edited 2014-01-06 06:30
    i have an early ssd on notebook and same drive on desktop. neither is noticable slower after 2 or 3 years. but i clean install xp every 6 months or so.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 06:38
    braino,

    What's all this "noticeable slow after 2 or 3 years"?

    I haven't read around on SSDs much yet but I get the idea that slowness can happen when a) Your SSD is getting full and/or b) the OS has not told it that the blocks used by deleted files are now actually free for use in wear levelling.

    a) We can't do much about except delete files or get more space.
    b) Should be taken care of by the OS issuing TRIM commands to the drive that tells it what blocks the files system is now treating as free.

    Unlike old hard drives and MS file systems fragmentation is not an issue that grows over time. Modern operating systems don't suffer from file fragmentation much and the SSD has no seek time over head anyway.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 07:16
    I just managed to get everything reinstalled and set up on my machine to the point I can build the openspin.js again.

    Amazingly I was hitting the compile button on the openspin_web_demo2.html page many times and it just kept working. Nice an quickly too.

    What? Did the memory leaking go away? Do I have a better version of Clamg or Emscripten now?

    Turns out it had actually eaten all my RAM and 1.6GB of swap space. Just as before.

    That SSD is so fast I did not notice!

    The odd thing here is that the memory allowed for the openspin.js compiler is limited to 64 MB and it never complains of running out as it would if it did not free it's memory. Seems the browser is keeping the memory even if the app gives it back.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 07:20
    Jazzed,

    What was the idea with your demo. Does openspin.js work well enough for a proof of concept demo?
  • jazzedjazzed Posts: 11,803
    edited 2014-01-06 08:53
    Hi Heater.

    Having openspin.js is key to having a ubiquitous solution. Can you post the version that doesn't leak? The last one leaked for me in Chrome and Firefox.

    I'm considering several demos and have been working on a specific one that is http://localhost:8080 hosted for now.

    Eventual example configurations* mentioned in another post, but restated here for clarity:
    Phone or Tablet + Wifi enabled ActivityBot or other Propeller board.
    Phone or Tablet + Hosted ActivityBot or other Propeller board.
    Hosted ActivityBot or other Propeller board - like today's Propeller-Tool or SimpleIDE.
    The configuration determines the software used.

    The most useful aspect of openspin.js would be: run in any browser (-IE). That means any Wifi enabled phone, pad, or other device can be used to develop and use SPIN apps without much extra burden. Having an IE capable version would simplify all configurations (like it or not).

    Other solutions not using an emscripten port such as Propeller-GCC (compiled with llvm on Mac, but not practical for emscripten) can use a traditional computer (PC or RPi) to provide compiler services. One could also have a "cloud" solution, but i fear that is not scalable unless the "cloud server" is massively parallel for handling compiler jobs.

    Having the option to run a simple server on a Hosted Linux/Mac/Windows PC (or Phone/Tablet?) without needing internet access is very desirable and possible (I.E. http://localhost:8080/editor.html).

    Configuration notes*
    Phone is Android, iPhone, or other.
    Tablet is Android, iPad, or other non-PC device.
    Hosted is by traditional OS like Linux, Mac, or Windows.

    A Propeller-WebTool IDE would have an editor, compiler, loader, and terminal. The editor and compiler could use a separate web address from the loader and terminal if necessary.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 09:49
    Jazzed,

    OK that sounds cool.

    Just now my view is that everthing is browser based and I don't care what machine or OS that browser is running on. Where "everything" is at least an editor and Spin compiler, the loader is a server process except perhaps on Chrome books that can get access to serial I/O.

    Some clever person will have to think about the user interface and how it needs tweaking, if it does, for large screen PCs and small screen tabs.

    This should run on IE. It compiles to regular JS, I don't know what the issue there is yet. I'm still amazed that it works at all!

    I don't understand the statement: "Having an IE capable version would simplify all configurations" I'm not sure how IE affects anything apart from the usual tweaks for browser differences.

    Sadly I don't have a version without memory leak. the demo2 version I posted is best so far. My latest effort here got worse!
  • jazzedjazzed Posts: 11,803
    edited 2014-01-06 10:10
    Heater. wrote: »
    I don't understand the statement: "Having an IE capable version would simplify all configurations" I'm not sure how IE affects anything apart from the usual tweaks for browser differences.
    It just means that I'd like the solution to work on all platforms without asking a user to install anything special. There will be some tweaking required regardless especially for locally hosted HTML/Javascript.

    I'll put together one prototype package and post it here once I have more parts in place.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-06 10:51
    This memory leak thing is strange.

    I went back to basics and built a hello world program and put it in the same HTML as we have for the compiler.

    Every time I hit compile it prints "hello world" and the Chrome task manager shows 64MB RAM gets used. Hit compile again and there goes another 64MB.
    64MB happens to be the memory size I configured for it to run in.

    Now, sometimes after a few seconds of inactivity Chrome shows that memory is freed and we get back to the starting level.

    Sometimes it does not. It just gets stuck there. Very odd.
  • msrobotsmsrobots Posts: 3,709
    edited 2014-01-06 15:27
    Another small update for the Editor.htm

    I added this function
    [SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]function[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] getQueryParam(n){[/SIZE]
    [SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]var[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] half = location.search.split(n+[/SIZE][SIZE=2][COLOR=#a31515][SIZE=2][COLOR=#a31515]'='[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2])[1];[/SIZE]
    [SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]return[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] half? decodeURIComponent(half.split([/SIZE][SIZE=2][COLOR=#a31515][SIZE=2][COLOR=#a31515]'&'[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2])[0]):[/SIZE][SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]null[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2];[/SIZE]
    [SIZE=2] }    
    [/SIZE]
    

    and I added this to the end of the bodyonload() function
    ...
    [SIZE=2]    editor.focus();
     [/SIZE][SIZE=2]    [/SIZE][SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]var[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] file=getQueryParam([/SIZE][SIZE=2][COLOR=#a31515][SIZE=2][COLOR=#a31515]'a'[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]);[/SIZE]
    [SIZE=2]    [/SIZE][SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]if[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2] (file!=[/SIZE][SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]null[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2])[/SIZE]
    [SIZE=2]     {
     [/SIZE][SIZE=2]        document.getElementById([/SIZE][SIZE=2][COLOR=#a31515][SIZE=2][COLOR=#a31515]"filename"[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2]).value=file;[/SIZE]
    [SIZE=2]         btloadClick();
         }
     [/SIZE][SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]return [/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]true[/COLOR][/SIZE][/COLOR][/SIZE][SIZE=2];[/SIZE]
    

    And now you can call the Editor.htm with a filename as parameter like

    http://<yourserver>Editor.htm?a=index.htm to load file index.htm and edit it.

    Enjoy!

    Mike


  • potatoheadpotatohead Posts: 10,261
    edited 2014-01-06 21:09
    Re: SSD getting slower

    XP does not support trim. Win 7 does, and it does not always get turned on.

    Excessive moves to avoid multiple bad areas can slow the SSD controller and this is mostly older, slower drives.

    Firmware may be a cause, but if it works well, changing that can be trouble.

    Just checked mine. It is a big 500GB one, and the larger ones have more blocks to work with / unit of filesystem space. Says "excellent" but I have no real metrics to feel good about yet.

    The big scare comes from a significant filesystem size change apparently. Once that happens, things are worn and time is short.

    I have large CAD and virtual machines on mine. It is a 3X speed up overall. Worth it, and I just back up to a fat, slow RPM drive.
  • potatoheadpotatohead Posts: 10,261
    edited 2014-01-06 21:36
    This is looking very interesting guys... That's it. Just reading with interest and thought I would let you know.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-07 01:39
    potatohead,

    Thanks for the comments on SSDs. I won't be getting one for the XP box we have here then. Makes me glad I was not an early adopter. I'm going to be paranoically watching the output of the SMART diagnostics for months now...

    Sounds like Jazzed is rapidly cooking something interesting.
  • jazzedjazzed Posts: 11,803
    edited 2014-01-07 10:45
    I'm working on the Hosted solution with a USB attached ActivityBot.

    Status:
    AJAX serial terminal receive work in progress.
    Only tested with Chrome on Windows so far.
    • Variant of Phil's Codemirror 2 posted by msrobots
    • Uses "suggest" to filter, select, and load an existing SPIN file
    • Can save SPIN file as a given file name
    • Compiles SPIN program
    • Can program Propeller
    Python server (http://localhost:8080) running on my PC supports:
    • TERMSEND (char)
    • TERMSTART (port)
    • TERMSTOP
    • COMPILE (spin file with compiled openspin)
    • PROGRAM (spin file/binary with propeller-load)
    • IDENTIFY (first propeller port with pyserial & propeller-load)
    • SCANSERIAL (find all valid propeller ports with pyserial & propeller-load)
    • SUGGEST (files that can be loaded)
    • BAUDRATE (baud for terminal)
    Serial Terminal (http://localhost:8080/terminal.html):
    • Connect to first Propeller found (or port text)
    • Uses Identify and "suggest" to allow choosing port
    • Can send chars to Propeller via selected port
    • Receive is work in progress
  • Heater.Heater. Posts: 21,230
    edited 2014-01-07 13:38
    I think I have a solution to my dilemma regarding an SSD wearing out due to heavy hammering to the swap file, logs, caches etc.
    I'm going to buy a second SSD and put all those heavily used files on there :)

    Jazzed,

    I can't quite reach your localhost from here :(
  • ctwardellctwardell Posts: 1,716
    edited 2014-01-07 13:45
    Heater. wrote: »
    I think I have a solution to my dilemma regarding an SSD wearing out due to heavy hammering to the swap file, logs, caches etc.
    I'm going to buy a second SSD and put all those heavily used files on there :)

    Please keep us informed on how this goes.

    I'd like to do the same on my workstation in the next few months.

    C.W.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2014-01-07 13:46
    heater wrote:
    Jazzed,

    I can't quite reach your localhost from here
    Did you try a traceroute to see where it's getting hung up? :)

    -Phil
  • Heater.Heater. Posts: 21,230
    edited 2014-01-07 13:52
    Phil,
    $ ping localhost
    PING localhost (127.0.0.1) 56(84) bytes of data.
    64 bytes from localhost (127.0.0.1): icmp_req=1 ttl=64 time=0.060 ms
    64 bytes from localhost (127.0.0.1): icmp_req=2 ttl=64 time=0.062 ms
    

    I don't understand it. As you see the connection is fine, fast too. :)
  • jazzedjazzed Posts: 11,803
    edited 2014-01-07 13:52
    Geez.

    I tried very hard to get this stupid forum editor not to make links to localhost which is totally invalid for anyone but me in this case.

    Alas, I edited the post and forgot to turn off automatically parse links. Will fix that now.
  • Heater.Heater. Posts: 21,230
    edited 2014-01-07 13:58
    Can I suggest you use websockets for the terminal connection server to browser. It really easy to do with socket.io. http://socket.io/
  • prof_brainoprof_braino Posts: 4,313
    edited 2014-01-07 13:59
    potatohead wrote: »
    Re: SSD getting slower

    XP does not support trim.

    Re #161 - XP has been just fine after years on an early 80G SSD. Way faster than a spinning drive in any case. I don't have space for lots of movies but I can keep those on a USB flob if I need to, (which I don't). No trim, no wear leveling, no nothin' aside from stock install. YMMV.
  • jazzedjazzed Posts: 11,803
    edited 2014-01-07 14:26
    Can we please move the SSD discussions to another thread?

    Thanks for the socket.io link. I'll look into it further if I find my current solution lacking.
Sign In or Register to comment.