Automatic Spin Program Documenter (Online version now available)
Phil Pilgrim (PhiPi)
Posts: 23,514
I love writing programs, but I hate writing documentation. One problem with writing program documentation is that it's so redundant [whine, snivel]. If you've already commented your code, why do it all over again? Another issue I have with documenting code is having to to it with Microsoft Word (which I detest) and then converting it to PDF. This is a problem because such documents are page-oriented, while the subject matter is not. For example, how many times have you had to leaf back and forth in a multi-page source listing, while referring to a description of how it works on yet another page? On the other hand, hypertext formats are just made for documenting code, and that's the route I chose to take.
So, faced with documenting the S2 object I've worked on for several months, loving to program, and hating to document, the solution was obvious: write a program to document the code for me! So that's what I did.
Fortunately, Chip and Jeff had the foresight to include a strategy within Spin itself for self-documenting code. The various commenting layers make it possible to separate document comments from code comments. What Spin provides only sketchily, however, is a presentation layer for the document comments so that the documentation can be viewed in a way that turns them them into a useful reference. That's the gap that I have tried to fill.
The Perl program I've written takes in a Spin file and spits out hyperlinked HTML. This is not something that's new or unique, however. Perl programmers have had such a facility for a long time with Perl's POD (plain old documention) markup language and the pod2html conversion program. Anyone who's used CPAN or ActiveState's Perl will appreciate the difference that nicely-formatted HTML can make in a document.
POD can be a bit intrusive, though, when sprinkled inline amongst various Perl subroutines and viewed as part of the source code. Consequently, POD sections are usually relegated to the end of the file. I wanted something that would look presentable both within the Spin source and also post-conversion. The idea was that most of the markup would consist of things that people do anyway, with one or two discreet additions for formatting and highlighting the HTML. So here's what I came up with.
Spin's documentation commenting consists of block comments, enclosed in double braces, and one-liners that begin with double apostrophe's:
The {{ block comments }} are used to demarcate free-formatted paragraphs. Generally, text within block comments will be formatted to fit the browser page width, regardless of how wide the lines are in the Spin program. And they produce an easy-to-read, proportionally spaced font stripped of the extra spaces that may appear in the Spin source code. Formatting of block comments for conversion purposes is a stricter than it is in Spin, however. The program will only recognize them if the opening {{ are the first non-blank characters in a given line and if the closing }} are the last non-blank characters in a line.
'' One-line comments mark lines that are also documented as separate lines. They're still presented with a proportional font, but leading spaces and extra embedded spaces are preserved. One-line comments are only recognized when the two apostrophes '' appear as the first non-blank characters in a given line.
''' The triple-apostrophe begins a special form of one-line comments. These are used to demarcate a single line wherein the text gets output with a monospaced font. If you have the Parallax font installed (recommended) it will use the Parallax font. These lines are typically used to draw diagrams with Spin's special graphics characters (which get converted to UTF-8 for the HTML output).
{{{ Monospaced block comments }} (note just two ending }s) work like ''' comments, but without the necessity for the prefix at the beginning of each line.
{{{... Monospaced sample code }} (note just two ending }s) provide a way to insert sample code within the object itself without it being compiled with the object. When converted to a document, the contents of this block are displayed in the same way other source code is displayed (i.e. in a drop-down box triggered by clicking the SOURCE CODE... link. The difference here, though, is that clicking anywhere in the displayed source selects all of it, so that it can be copied and pasted into the Propeller Tool.
Finally, one-line comments can be embedded within block comments to format individual lines within a paragraph, say.
The program supports several kinds of headings, shown below. They can be embedded within both block and line comments, with about the same effect. All have the same syntax: <formatting character(s)>[ Heading Text ]<formatting character(s)>. This is in line with the way headings are usually shown in Spin (and even PBASIC) programs anyway, so it's not somehting that will be jarring to see on one's program. In the final document only the Heading Text is shown, not the surrounding brackets and formatting characters. The only cue the conversion program looks for is one formatting character on either side of the brackets. The rest are redundant. For example, -[ Heading Text ]- is sufficient to demarcate a subheading. Here are the heading types:
In most program sections, document text and source code are shown in the order in which they appear. The exceptions are PUB and PRI sections. In these, any document comments appearing within the section (i.e. not at the bottom) are gathered for presentation at the top, after the method heading, but before the source code link. This is in keeping with the practice of the PUB or PRI method line appearing in the source code, then the document comments pertaining to that method, followed by the rest of the method's source code.
I could go on with this long-winded description. But it might be easier just to exhibit an example that encompasses all the features I've mentioned. Attached is the latest source for the S2 Spin object. In it, you can see examples of the markup cues I've mentioned here. Here is a link to the HTML code that my program generated from it:
All the ToC entries and "SOURCE CODE..." tags are hyperlinks that you can play with. I've tested the HTML with Opera (WinXP, OS/X), Firefox (WinXP), and Safari (OS/X). All seem to behave themselves. IE6 was usable, but a disaster from a presentation standpoint. (I don't use IE, so I haven't installed any later versions.) Opera was by far the easiest and most forgiving to develop for; and, IMO, has the best font rendering of all of them.
Anyway, if there are anomalies that you find too distracting, let me know. Meanwhile, I plan to make the spin2html program available as a web service. My web host has to move my site to a different server first to accommodate Perl 5.8, which supports unicode. That should be done by sometime tomorrow.
-Phil
Addendum: The online documenter is up and running at: http://www.phipi.com/spin2html/.
Addendum (2010.11.26): Added a description (above) regarding the new sample code formatting and updated the S2 object to demonstrate its use.
So, faced with documenting the S2 object I've worked on for several months, loving to program, and hating to document, the solution was obvious: write a program to document the code for me! So that's what I did.
Fortunately, Chip and Jeff had the foresight to include a strategy within Spin itself for self-documenting code. The various commenting layers make it possible to separate document comments from code comments. What Spin provides only sketchily, however, is a presentation layer for the document comments so that the documentation can be viewed in a way that turns them them into a useful reference. That's the gap that I have tried to fill.
The Perl program I've written takes in a Spin file and spits out hyperlinked HTML. This is not something that's new or unique, however. Perl programmers have had such a facility for a long time with Perl's POD (plain old documention) markup language and the pod2html conversion program. Anyone who's used CPAN or ActiveState's Perl will appreciate the difference that nicely-formatted HTML can make in a document.
POD can be a bit intrusive, though, when sprinkled inline amongst various Perl subroutines and viewed as part of the source code. Consequently, POD sections are usually relegated to the end of the file. I wanted something that would look presentable both within the Spin source and also post-conversion. The idea was that most of the markup would consist of things that people do anyway, with one or two discreet additions for formatting and highlighting the HTML. So here's what I came up with.
Spin's documentation commenting consists of block comments, enclosed in double braces, and one-liners that begin with double apostrophe's:
{{ This is a block comment.
It can go on for many lines.
}}
'' This is a one-line comment.
The program I've written looks for documentation comments sprinkled amongst the source code and manipulates their contents to produce an HTML file, while also including the code itself.The {{ block comments }} are used to demarcate free-formatted paragraphs. Generally, text within block comments will be formatted to fit the browser page width, regardless of how wide the lines are in the Spin program. And they produce an easy-to-read, proportionally spaced font stripped of the extra spaces that may appear in the Spin source code. Formatting of block comments for conversion purposes is a stricter than it is in Spin, however. The program will only recognize them if the opening {{ are the first non-blank characters in a given line and if the closing }} are the last non-blank characters in a line.
'' One-line comments mark lines that are also documented as separate lines. They're still presented with a proportional font, but leading spaces and extra embedded spaces are preserved. One-line comments are only recognized when the two apostrophes '' appear as the first non-blank characters in a given line.
''' The triple-apostrophe begins a special form of one-line comments. These are used to demarcate a single line wherein the text gets output with a monospaced font. If you have the Parallax font installed (recommended) it will use the Parallax font. These lines are typically used to draw diagrams with Spin's special graphics characters (which get converted to UTF-8 for the HTML output).
{{{ Monospaced block comments }} (note just two ending }s) work like ''' comments, but without the necessity for the prefix at the beginning of each line.
{{{... Monospaced sample code }} (note just two ending }s) provide a way to insert sample code within the object itself without it being compiled with the object. When converted to a document, the contents of this block are displayed in the same way other source code is displayed (i.e. in a drop-down box triggered by clicking the SOURCE CODE... link. The difference here, though, is that clicking anywhere in the displayed source selects all of it, so that it can be copied and pasted into the Propeller Tool.
Finally, one-line comments can be embedded within block comments to format individual lines within a paragraph, say.
The program supports several kinds of headings, shown below. They can be embedded within both block and line comments, with about the same effect. All have the same syntax: <formatting character(s)>[ Heading Text ]<formatting character(s)>. This is in line with the way headings are usually shown in Spin (and even PBASIC) programs anyway, so it's not somehting that will be jarring to see on one's program. In the final document only the Heading Text is shown, not the surrounding brackets and formatting characters. The only cue the conversion program looks for is one formatting character on either side of the brackets. The rest are redundant. For example, -[ Heading Text ]- is sufficient to demarcate a subheading. Here are the heading types:
'':::::::[ Title Heading ]::::::::::::::::::::::::::::::::::::::::::::::::::::: '' One title heading should be used at the very top of your program. The text '' gets displayed centered, with a Table of Contents entry labeled, "Preface". '' It is used ahead of any copyright and version information that typically '' appears before the program's CONstant section. ''=======[ Major Section Heading ]============================================= '' The text between brackets in this kind of heading appears in both the '' presentation text and in the Table of contents. ''=======[ Major Section Heading... ]========================================== '' Any section heading ending with an ellipsis (...) is shown in the Table of '' Contents with the eliipsis, but in the text without. In the ToC, this has a '' special effect, in that all subsequent, lesser ToC entries are hidden until '' the entry with the ellipsis is clicked. A subsequent click will again hide '' the subentries below it, helping to keep the ToC tidy and small. '' #######[ Major Section Heading ]############################################ '' This is the same as =[ Major Section heading ]=, except that no ToC entry is '' made. Think of the pound signs as equal signs with cross-hatches that say, '' "No ToC entry!" ''-------[ Minor Section Heading ]--------------------------------------------- '' The text between brackets in this kind of heading appears in both the '' presentation text and in the Table of contents. ''-------[ Minor Section Heading... ]------------------------------------------ '' Any section heading ending with an ellipsis (...) is shown in the Table of '' Contents with the eliipsis, but in the text without. In the ToC, this has a '' special effect, in that all subsequent, lesser ToC entries are hidden until '' the entry with the ellipsis is clicked. A subsequent click will again hide '' the subentries below it, helping to keep the ToC tidy and small. The only '' entries lesser than a minor heading are method names. '' +++++++[ Major Section Heading ]++++++++++++++++++++++++++++++++++++++++++++ '' This is the same as -[ Minor Section heading ]-, except that no ToC entry is '' made. Think of the plus signs as hyphens with cross-hatches that say, '' "No ToC entry!"In addition to the above formatting, the only other feature of note is highlighting. Any single word immediately preceded by a backtick (e.g. `boldface) will be shown in boldface.
In most program sections, document text and source code are shown in the order in which they appear. The exceptions are PUB and PRI sections. In these, any document comments appearing within the section (i.e. not at the bottom) are gathered for presentation at the top, after the method heading, but before the source code link. This is in keeping with the practice of the PUB or PRI method line appearing in the source code, then the document comments pertaining to that method, followed by the rest of the method's source code.
I could go on with this long-winded description. But it might be easier just to exhibit an example that encompasses all the features I've mentioned. Attached is the latest source for the S2 Spin object. In it, you can see examples of the markup cues I've mentioned here. Here is a link to the HTML code that my program generated from it:
All the ToC entries and "SOURCE CODE..." tags are hyperlinks that you can play with. I've tested the HTML with Opera (WinXP, OS/X), Firefox (WinXP), and Safari (OS/X). All seem to behave themselves. IE6 was usable, but a disaster from a presentation standpoint. (I don't use IE, so I haven't installed any later versions.) Opera was by far the easiest and most forgiving to develop for; and, IMO, has the best font rendering of all of them.
Anyway, if there are anomalies that you find too distracting, let me know. Meanwhile, I plan to make the spin2html program available as a web service. My web host has to move my site to a different server first to accommodate Perl 5.8, which supports unicode. That should be done by sometime tomorrow.
-Phil
Addendum: The online documenter is up and running at: http://www.phipi.com/spin2html/.
Addendum (2010.11.26): Added a description (above) regarding the new sample code formatting and updated the S2 object to demonstrate its use.
Comments
Ross.
I am using Google's Chrome, instead of Safari, and it appears great. Not using Safari, I don't have anything else to compare it to, but I'd guess it is proper.
Next to 'peek' at your source file to understand how the 'original' document appeared.
Just curious: Do you write documentation so that you can read it yourself, or so that other can read it?
How often do you read you own documentation?
Any ideas how often other folks read your code?
I tend to document just to organize my thoughts, and the better my documentation, the better my results. Although I seldom read it myself until somebody asks a question.
Thanks for the kind words!
Heater,
Yeah, Kate is probably not the best viewer for Spin source code (UTF-16). BTW, I tried viewing the HTML in Konqueror (3.5.6). Everything looked okay, but the ToC links didn't work. Firefox under the latest Linux Mint (Gnome) worked fine, though.
Prof_braino,
It depends on who the program is for. If it's just for me, I always comment assembly code, but that's about it. If it's for a wider audience, I add more documentation. In any event, it takes about a month of sitting unread, and then coming back to it, before I can tell whether the documentation is effective. I've had cases, when reading my own documentation months own the road, where I've thought, "How could anyone understand that? Heck, I don't understand it, and I wrote it!"
-Phil
One of my peeves is the lack of quality documentation (ie Manual) that comes with modern software.
For the S2, why not solve this by pressing a secret button combo. Then just load in a texta/marker, and it can write out its (very easy to read large text size) manual
this is really great! I can't wait to get a possability to create this
html-docs!
Please tell us an estimated date when it will be available.
@Parallax: to encourage submitters of objects to the OBEX
how about some kind of credits that the submitter can achieve
if his object contains a full documentation in this standard?
credits like a percentage for a lower price on parallax products
Of course somebody at parallax would have to read (and understand it).
To keep the time low the bar to jump over has to be hanging high.
and some criteria to check it quickly
Fully documented? EVERY section of the program has a comment?
MIT licence added? small part of the credits
by the way: @PhiPi: how about adding a feature to you software that tells the user which sections, PUBS or PRIs still havent a documenation?
If you read it once and understand it it will get the big part of the credits
If the documenation is not added within in a month by the author anybody
else can claim a comment-right for a week. If a FULLY commented version is submitted within a week THIS submitter will get the credits
These are only spontanious ideas. Maybe some (or all) are not really practical. The main intention of my post is to make parallax think about how can parallax support a high quality documenation of the objetcs?
IMAGINE:
ALL objects would be commented like S2.SPIN!
The propeller and SPIN would have a reputation of
"very easy to understand because it's so well documented"
best regards
Stefan
The more serious issue was that things like diff don't work on such Spin files. Fortunately kdiff3 does.
Kate can work with Unicode Spin files okay. After you've loaded the file, go to Tools->Encoding, and select "Unicode (utf16)".
-Phil
Just FYI: The well known industry standard alternative is doxygen which is mature, multi-platform, multi-language, supports .html/.rtf, and has a configuration wizard.
Doxygen requires adding a C style function declaration (or other recognized language) for each method output to work nicely. Short and simple standardized tags like javadoc @param are used and easy to remember.
Phil's program supports SPIN syntax by default, so it should be easier to use.
Thanks.
Don't know how I missed that setting. Guess I stopped looking down the list of languages after Japanese, Korean or such, not thinking that UTF-16 was a "language".
-Phil
function resize() { var myWidth = 0, myHeight = 0; if(typeof(window.innerWidth) == 'number') { //Non-IE myWidth = window.innerWidth; myHeight = window.innerHeight; } else if(document.documentElement && (document.documentElement.clientWidth || document.documentElement.clientHeight)) { //IE 6+ in 'standards compliant mode' myWidth = document.documentElement.clientWidth; myHeight = document.documentElement.clientHeight; } else if(document.body && (document.body.clientWidth || document.body.clientHeight)) { //IE 4 compatible myWidth = document.body.clientWidth; myHeight = document.body.clientHeight; } document.getElementById("text").style.height = (myHeight - 10) + "px"; document.getElementById("text").style.width = (Math.min(myWidth - 250, 800)) + "px"; document.getElementById("toc").style.height = (myHeight - 10) + "px"; }The tabs do not expand correctly in the first file (interpreter) and the second file (document only) nothing is displayed (perhaps because there is no spin code?)
Of course, neither of these conform to your specifications. I just thought I would run a couple of programs through to see what they looked like.
Thanks! That took care of auto-resizing the two panes in IE6. IE6 still has a lot of issues, though, including not handling inline images or understanding the max-height style. But it's a lot better than it was! Thanks again!
Cluso99,
I don't have any trouble with the ToC or SOURCE CODE... links expanding with your code. Any lines that require monospaced fonts, though, need to be either in {{{ }} blocks or ' ' ' lines. This includes anything with Spin special graphics characters, such as box outlines. (See the modified heading in the attached.)
-Phil
Now I will have to find the time to go through my code - Ah the joys of finding new stuff!
And yeah inline images can not be handled by IE. I can have a look at the max-height style and see if I can come up with a solution.
-Phil
Does the Parallax logo show up in the upper lefthand corner?
rosco_pc,
It looks like max-height is supported in IE, beginning with IE7. So I think I won't worry about that one.
-Phil
the doc-tool is really great. documenting the parts of a program helps a lot.
For newbees there are still questions left.
I want to a make a suggestion for another feature:
Source code is shown in a non-proportional font.
How about adding another special keyword "DEMO-PRG" where all lines
are shown in this non-proportional font as one piece without interuptions between different methods CON, VAR, DAT-sections?
So that the user can do a mark, copy & paste to extract a compilable and executable demo-prg?
best regards
Stefan
What you're suggesting can be done now:
''==[ Demo Program ]== {{{ CON ... OBJ ... PUB start ... }}The demo code won't appear in a drop-down code box, though, but that's something I could add.-Phil
-Phil
This is another fine piece of work, Phil did and nobody nows about today. This is really nice, integrates EASY with the standart PropTool comments and produces REALLY nice output.
Great work Phil. Good to hear about that newer version you are working on, integrating into the PropTool - waiting with patience, but starting to re-comment my SD-server and SD-client project. V.1.5 its almost ready for the OBEX and I will include your comment-tags. Simple brilliant again. Thank you.
Enjoy!
Mike
I agree. I guess I didn't have any multi-dotted Spin files to test it with.
-Phil