Shop OBEX P1 Docs P2 Docs Learn Events
Automatic Spin Program Documenter (Online version now available) — Parallax Forums

Automatic Spin Program Documenter (Online version now available)

Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
edited 2014-07-13 11:51 in Propeller 1
I love writing programs, but I hate writing documentation. One problem with writing program documentation is that it's so redundant [whine, snivel]. If you've already commented your code, why do it all over again? Another issue I have with documenting code is having to to it with Microsoft Word (which I detest) and then converting it to PDF. This is a problem because such documents are page-oriented, while the subject matter is not. For example, how many times have you had to leaf back and forth in a multi-page source listing, while referring to a description of how it works on yet another page? On the other hand, hypertext formats are just made for documenting code, and that's the route I chose to take.

So, faced with documenting the S2 object I've worked on for several months, loving to program, and hating to document, the solution was obvious: write a program to document the code for me! So that's what I did.

Fortunately, Chip and Jeff had the foresight to include a strategy within Spin itself for self-documenting code. The various commenting layers make it possible to separate document comments from code comments. What Spin provides only sketchily, however, is a presentation layer for the document comments so that the documentation can be viewed in a way that turns them them into a useful reference. That's the gap that I have tried to fill.

The Perl program I've written takes in a Spin file and spits out hyperlinked HTML. This is not something that's new or unique, however. Perl programmers have had such a facility for a long time with Perl's POD (plain old documention) markup language and the pod2html conversion program. Anyone who's used CPAN or ActiveState's Perl will appreciate the difference that nicely-formatted HTML can make in a document.

POD can be a bit intrusive, though, when sprinkled inline amongst various Perl subroutines and viewed as part of the source code. Consequently, POD sections are usually relegated to the end of the file. I wanted something that would look presentable both within the Spin source and also post-conversion. The idea was that most of the markup would consist of things that people do anyway, with one or two discreet additions for formatting and highlighting the HTML. So here's what I came up with.

Spin's documentation commenting consists of block comments, enclosed in double braces, and one-liners that begin with double apostrophe's:
{{ This is a block comment. 
   It can go on for many lines.
}}

'' This is a one-line comment.
The program I've written looks for documentation comments sprinkled amongst the source code and manipulates their contents to produce an HTML file, while also including the code itself.

The {{ block comments }} are used to demarcate free-formatted paragraphs. Generally, text within block comments will be formatted to fit the browser page width, regardless of how wide the lines are in the Spin program. And they produce an easy-to-read, proportionally spaced font stripped of the extra spaces that may appear in the Spin source code. Formatting of block comments for conversion purposes is a stricter than it is in Spin, however. The program will only recognize them if the opening {{ are the first non-blank characters in a given line and if the closing }} are the last non-blank characters in a line.

'' One-line comments mark lines that are also documented as separate lines. They're still presented with a proportional font, but leading spaces and extra embedded spaces are preserved. One-line comments are only recognized when the two apostrophes '' appear as the first non-blank characters in a given line.

''' The triple-apostrophe begins a special form of one-line comments. These are used to demarcate a single line wherein the text gets output with a monospaced font. If you have the Parallax font installed (recommended) it will use the Parallax font. These lines are typically used to draw diagrams with Spin's special graphics characters (which get converted to UTF-8 for the HTML output).

{{{ Monospaced block comments }} (note just two ending }s) work like ''' comments, but without the necessity for the prefix at the beginning of each line.

{{{... Monospaced sample code }} (note just two ending }s) provide a way to insert sample code within the object itself without it being compiled with the object. When converted to a document, the contents of this block are displayed in the same way other source code is displayed (i.e. in a drop-down box triggered by clicking the SOURCE CODE... link. The difference here, though, is that clicking anywhere in the displayed source selects all of it, so that it can be copied and pasted into the Propeller Tool.

Finally, one-line comments can be embedded within block comments to format individual lines within a paragraph, say.

The program supports several kinds of headings, shown below. They can be embedded within both block and line comments, with about the same effect. All have the same syntax: <formatting character(s)>[ Heading Text ]<formatting character(s)>. This is in line with the way headings are usually shown in Spin (and even PBASIC) programs anyway, so it's not somehting that will be jarring to see on one's program. In the final document only the Heading Text is shown, not the surrounding brackets and formatting characters. The only cue the conversion program looks for is one formatting character on either side of the brackets. The rest are redundant. For example, -[ Heading Text ]- is sufficient to demarcate a subheading. Here are the heading types:
'':::::::[ Title Heading ]:::::::::::::::::::::::::::::::::::::::::::::::::::::

'' One title heading should be used at the very top of your program. The text
'' gets displayed centered, with a Table of Contents entry labeled, "Preface".
'' It is used ahead of any copyright and version information that typically
'' appears before the program's CONstant section.

''=======[ Major Section Heading ]=============================================

'' The text between brackets in this kind of heading appears in both the
'' presentation text and in the Table of contents.

''=======[ Major Section Heading... ]==========================================

'' Any section heading ending with an ellipsis (...) is shown in the Table of
'' Contents with the eliipsis, but in the text without. In the ToC, this has a
'' special effect, in that all subsequent, lesser ToC entries are hidden until
'' the entry with the ellipsis is clicked. A subsequent click will again hide
'' the subentries below it, helping to keep the ToC tidy and small.

'' #######[ Major Section Heading ]############################################

'' This is the same as =[ Major Section heading ]=, except that no ToC entry is
'' made. Think of the pound signs as equal signs with cross-hatches that say,
'' "No ToC entry!"

''-------[ Minor Section Heading ]---------------------------------------------

'' The text between brackets in this kind of heading appears in both the
'' presentation text and in the Table of contents.

''-------[ Minor Section Heading... ]------------------------------------------

'' Any section heading ending with an ellipsis (...) is shown in the Table of
'' Contents with the eliipsis, but in the text without. In the ToC, this has a
'' special effect, in that all subsequent, lesser ToC entries are hidden until
'' the entry with the ellipsis is clicked. A subsequent click will again hide
'' the subentries below it, helping to keep the ToC tidy and small. The only
'' entries lesser than a minor heading are method names.

'' +++++++[ Major Section Heading ]++++++++++++++++++++++++++++++++++++++++++++

'' This is the same as -[ Minor Section heading ]-, except that no ToC entry is
'' made. Think of the plus signs as hyphens with cross-hatches that say,
'' "No ToC entry!"
In addition to the above formatting, the only other feature of note is highlighting. Any single word immediately preceded by a backtick (e.g. `boldface) will be shown in boldface.

In most program sections, document text and source code are shown in the order in which they appear. The exceptions are PUB and PRI sections. In these, any document comments appearing within the section (i.e. not at the bottom) are gathered for presentation at the top, after the method heading, but before the source code link. This is in keeping with the practice of the PUB or PRI method line appearing in the source code, then the document comments pertaining to that method, followed by the rest of the method's source code.

I could go on with this long-winded description. But it might be easier just to exhibit an example that encompasses all the features I've mentioned. Attached is the latest source for the S2 Spin object. In it, you can see examples of the markup cues I've mentioned here. Here is a link to the HTML code that my program generated from it:

All the ToC entries and "SOURCE CODE..." tags are hyperlinks that you can play with. I've tested the HTML with Opera (WinXP, OS/X), Firefox (WinXP), and Safari (OS/X). All seem to behave themselves. IE6 was usable, but a disaster from a presentation standpoint. (I don't use IE, so I haven't installed any later versions.) Opera was by far the easiest and most forgiving to develop for; and, IMO, has the best font rendering of all of them.

Anyway, if there are anomalies that you find too distracting, let me know. Meanwhile, I plan to make the spin2html program available as a web service. My web host has to move my site to a different server first to accommodate Perl 5.8, which supports unicode. That should be done by sometime tomorrow.

-Phil

Addendum: The online documenter is up and running at: http://www.phipi.com/spin2html/.
Addendum (2010.11.26): Added a description (above) regarding the new sample code formatting and updated the S2 object to demonstrate its use.
s2.spin 280.4K
«1

Comments

  • Cluso99Cluso99 Posts: 18,066
    edited 2010-11-17 19:03
    This is simply BRILLIANT Phil. Congratulations :yeah::yeah::yeah::yeah::yeah:
  • RossHRossH Posts: 5,336
    edited 2010-11-17 19:10
    If we combine Parallax's Propeller tool, Cluso's zero footprint debugger and now Phil's program documenter, would we end up with programs that compile, debug and then document themselves?

    Ross.
  • Heater.Heater. Posts: 21,230
    edited 2010-11-17 19:12
    Very nice but this unicode stuff makes s2.spin unintelligible in my browsers default text viewer, kate. Hope it still works after I have cleaned it up with iconv:)
  • HarleyHarley Posts: 997
    edited 2010-11-17 19:34
    @ Phil,

    I am using Google's Chrome, instead of Safari, and it appears great. Not using Safari, I don't have anything else to compare it to, but I'd guess it is proper.

    Next to 'peek' at your source file to understand how the 'original' document appeared.
  • prof_brainoprof_braino Posts: 4,313
    edited 2010-11-17 20:57
    This is a really cool idea. And it looks good.

    Just curious: Do you write documentation so that you can read it yourself, or so that other can read it?

    How often do you read you own documentation?
    Any ideas how often other folks read your code?

    I tend to document just to organize my thoughts, and the better my documentation, the better my results. Although I seldom read it myself until somebody asks a question.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-17 21:12
    All,

    Thanks for the kind words!

    Heater,

    Yeah, Kate is probably not the best viewer for Spin source code (UTF-16). BTW, I tried viewing the HTML in Konqueror (3.5.6). Everything looked okay, but the ToC links didn't work. Firefox under the latest Linux Mint (Gnome) worked fine, though.

    Prof_braino,

    It depends on who the program is for. If it's just for me, I always comment assembly code, but that's about it. If it's for a wider audience, I add more documentation. In any event, it takes about a month of sitting unread, and then coming back to it, before I can tell whether the documentation is effective. I've had cases, when reading my own documentation months own the road, where I've thought, "How could anyone understand that? Heck, I don't understand it, and I wrote it!"

    -Phil
  • HumanoidoHumanoido Posts: 5,770
    edited 2010-11-17 22:43
    Phil, excellent work! It makes a very neat and unique web page for easy viewing of documentation and code. I would hope that more programs in the OBEX end up with great documentation like this. Phil, it's always exciting to see your programming projects. Hope you have many more to come.
  • TubularTubular Posts: 4,620
    edited 2010-11-17 23:00
    Excellent work Phil. I have to admit I have thought many times about autodocumenting systems, attempted it once with a Pharma company (whose systems have ridiculous redundancy of documentation, and corresponding opportunity for errors). I couldn't persuade them to join the revolution, unfortunately.

    One of my peeves is the lack of quality documentation (ie Manual) that comes with modern software.

    For the S2, why not solve this by pressing a secret button combo. Then just load in a texta/marker, and it can write out its (very easy to read large text size) manual :)
  • StefanL38StefanL38 Posts: 2,292
    edited 2010-11-18 01:27
    Hi Phil,

    this is really great! I can't wait to get a possability to create this
    html-docs!

    Please tell us an estimated date when it will be available.

    @Parallax: to encourage submitters of objects to the OBEX
    how about some kind of credits that the submitter can achieve
    if his object contains a full documentation in this standard?

    credits like a percentage for a lower price on parallax products

    Of course somebody at parallax would have to read (and understand it).

    To keep the time low the bar to jump over has to be hanging high.
    and some criteria to check it quickly


    Fully documented? EVERY section of the program has a comment?
    MIT licence added? small part of the credits

    by the way: @PhiPi: how about adding a feature to you software that tells the user which sections, PUBS or PRIs still havent a documenation?


    If you read it once and understand it it will get the big part of the credits

    If the documenation is not added within in a month by the author anybody
    else can claim a comment-right for a week. If a FULLY commented version is submitted within a week THIS submitter will get the credits

    These are only spontanious ideas. Maybe some (or all) are not really practical. The main intention of my post is to make parallax think about how can parallax support a high quality documenation of the objetcs?

    IMAGINE:
    ALL objects would be commented like S2.SPIN!
    The propeller and SPIN would have a reputation of

    "very easy to understand because it's so well documented"

    best regards

    Stefan
  • Heater.Heater. Posts: 21,230
    edited 2010-11-18 01:37
    Poor old kate.

    The more serious issue was that things like diff don't work on such Spin files. Fortunately kdiff3 does.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-18 08:00
    Heater,

    Kate can work with Unicode Spin files okay. After you've loaded the file, go to Tools->Encoding, and select "Unicode (utf16)".

    -Phil
  • jazzedjazzed Posts: 11,803
    edited 2010-11-18 08:11
    This is useful work and the program output is nice.

    Just FYI: The well known industry standard alternative is doxygen which is mature, multi-platform, multi-language, supports .html/.rtf, and has a configuration wizard.

    Doxygen requires adding a C style function declaration (or other recognized language) for each method output to work nicely. Short and simple standardized tags like javadoc @param are used and easy to remember.

    Phil's program supports SPIN syntax by default, so it should be easier to use.
  • Heater.Heater. Posts: 21,230
    edited 2010-11-18 08:50
    Phil,

    Thanks.

    Don't know how I missed that setting. Guess I stopped looking down the list of languages after Japanese, Korean or such, not thinking that UTF-16 was a "language".
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-18 21:46
    I've gotten the online version working now. Check the top post for the address.

    -Phil
  • rosco_pcrosco_pc Posts: 449
    edited 2010-11-18 23:41
    Changing your resize function to the following will make it work under IE and also all other browser I've tested sofar :D
            function resize() {
              var myWidth = 0, myHeight = 0;
              if(typeof(window.innerWidth) == 'number') {
                //Non-IE
                myWidth = window.innerWidth;
                myHeight = window.innerHeight;
              } else if(document.documentElement && (document.documentElement.clientWidth || document.documentElement.clientHeight)) {
                //IE 6+ in 'standards compliant mode'
                myWidth = document.documentElement.clientWidth;
                myHeight = document.documentElement.clientHeight;
              } else if(document.body && (document.body.clientWidth || document.body.clientHeight)) {
                //IE 4 compatible
                myWidth = document.body.clientWidth;
                myHeight = document.body.clientHeight;
              }
    
              document.getElementById("text").style.height = (myHeight - 10) + "px";
              document.getElementById("text").style.width = (Math.min(myWidth - 250, 800)) + "px";
              document.getElementById("toc").style.height = (myHeight - 10) + "px";
            }
    
  • Cluso99Cluso99 Posts: 18,066
    edited 2010-11-18 23:44
    Found a few problems or in msoft speak unintended features :smilewinkgrin:

    The tabs do not expand correctly in the first file (interpreter) and the second file (document only) nothing is displayed (perhaps because there is no spin code?)

    Of course, neither of these conform to your specifications. I just thought I would run a couple of programs through to see what they looked like.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-19 00:37
    rosco_pc,

    Thanks! That took care of auto-resizing the two panes in IE6. IE6 still has a lot of issues, though, including not handling inline images or understanding the max-height style. But it's a lot better than it was! Thanks again!

    Cluso99,

    I don't have any trouble with the ToC or SOURCE CODE... links expanding with your code. Any lines that require monospaced fonts, though, need to be either in {{{ }} blocks or ' ' ' lines. This includes anything with Spin special graphics characters, such as box outlines. (See the modified heading in the attached.)

    -Phil
  • Cluso99Cluso99 Posts: 18,066
    edited 2010-11-19 02:01
    Thanks Phil. I just thought it would be worthwhile running the code as is through your program to see the output.

    Now I will have to find the time to go through my code - Ah the joys of finding new stuff!
  • rosco_pcrosco_pc Posts: 449
    edited 2010-11-19 04:32
    You're welcome,

    And yeah inline images can not be handled by IE. I can have a look at the max-height style and see if I can come up with a solution.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-19 11:27
    I guess I should add an alt attribute to the <img> tag, then, so IE has something to display.

    -Phil
  • Cluso99Cluso99 Posts: 18,066
    edited 2010-11-19 13:21
    Phil: I have no problems that I can see using IE8 & XP2
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-19 14:07
    Cluso99,

    Does the Parallax logo show up in the upper lefthand corner?

    rosco_pc,

    It looks like max-height is supported in IE, beginning with IE7. So I think I won't worry about that one.

    -Phil
  • rosco_pcrosco_pc Posts: 449
    edited 2010-11-19 15:09
    And data URLs are supported from IE8 onwards, but I think I may have found a solution for IE6,7 that does not require too much aditional javascript (playing around with it at the moment). I only use IE at work, running Linux or OpenBSD on all my computers at home. But it annoys me enough that it does not work properly AND I like the challenge and I keep learning something new every time :)
  • StefanL38StefanL38 Posts: 2,292
    edited 2010-11-20 00:56
    Hi Phil,

    the doc-tool is really great. documenting the parts of a program helps a lot.
    For newbees there are still questions left.

    I want to a make a suggestion for another feature:
    Source code is shown in a non-proportional font.
    How about adding another special keyword "DEMO-PRG" where all lines
    are shown in this non-proportional font as one piece without interuptions between different methods CON, VAR, DAT-sections?

    So that the user can do a mark, copy & paste to extract a compilable and executable demo-prg?

    best regards

    Stefan
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-20 09:34
    Stefan,

    What you're suggesting can be done now:
    ''==[ Demo Program ]==
    {{{
    CON
      ...
    OBJ
      ...
    PUB start
      ...
    }}
    
    The demo code won't appear in a drop-down code box, though, but that's something I could add.

    -Phil
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2010-11-26 11:50
    I took Stefan's suggestion to heart and added a facility for displaying sample code in a way that's more useful. See the first post for details and a revised S2 object that demonstrates its use.

    -Phil
  • Cluso99Cluso99 Posts: 18,066
    edited 2010-11-27 23:00
    Phil: Sorry to take so long. Yes the Parallax logo is intact.
    1280 x 800 - 148K
  • msrobotsmsrobots Posts: 3,701
    edited 2012-07-12 23:36
    Since I stumbled about this Thred in another one getting hijacked - I bump this one!

    This is another fine piece of work, Phil did and nobody nows about today. This is really nice, integrates EASY with the standart PropTool comments and produces REALLY nice output.

    Great work Phil. Good to hear about that newer version you are working on, integrating into the PropTool - waiting with patience, but starting to re-comment my SD-server and SD-client project. V.1.5 its almost ready for the OBEX and I will include your comment-tags. Simple brilliant again. Thank you.

    Enjoy!

    Mike
  • kuronekokuroneko Posts: 3,623
    edited 2012-07-13 00:03
    @Phil: The online version rejects valid SPIN files with more than one dot in the name. I see this as an unnecessary restriction.
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2012-07-13 00:15
    kuroneko,

    I agree. I guess I didn't have any multi-dotted Spin files to test it with. :) It will have to do for the time being, though. The online program will eventually be replaced by a derivative of the standalone program that I integrated with the Prop Tool. Any corrections I make will be made there first. But none of this will happen without a nod from Parallax. I put a ton of time into this program, but it doesn't make sense to release it if the Gold Standard takes a separate path.

    -Phil
Sign In or Register to comment.