Code as you think

I was in the middle of deciding how to proceed with my current side project when I realized that I'd not yet created an architectural description for it. Since I already have an architecture description tool that I wrote, I have no excuse for not writing one out. So I stopped to think why I didn't create an architecture definition first. Ans: Because it would stop me from doing what I was doing right then, which was to decide:

• how to separate out the current spike of the language from its sample code and actions.
• how this will be deployed in and of itself, ie how to dogfood it?
• how to move from the spike folder to the actual folder
• how to move from bitbucket(where I'm keeping all this stuff till its public-worthy) to github or a public bitbucket repo (where I expect this to be shared with the world)
• ...all in the middle of adding commit messages for changes
• while juggling code between the PC and the mac
• ... and so on.

As I'm writing this, more thoughts are streaming through, and they're all jumbled together: code ideas, architecture ideas, documentation, behind-the-scenes reasons, journal entries - all in one single stream of consciousness. Stopping now to create an architecture definition would not just slow me down, it would take me down a totally different path. And I don't want to go down that path because the IDEAS ARE JUST FLOWING!

Another reason I didn't want to context switch into "architecture definition mode" was the high cost of that switch : create a new document, decide where to save it, type in the actual content, optionally see how it looks once the markdown becomes html and so forth. IMO, this is also why documentation is out-of-date with code, especially expository documentation like architecture and design docs. The comment-above-code kind of documentation may still be in touch with the reality of the code, but higher level documentation like READMEs and user guides quickly become disconnected from the code they're talking about.

That's when it hit me: What we need are tools and languages that understand the stream-of-consiousness way of building software: code snippets flying into the system from the human mind along with documentation and random thoughts and todos and reorganizations of such thoughts - all of which the system will record in sequence and generate meaningful views out of. 

Imagine an editor that:

• Allows creation of content without the need to pick a name or a location for it.
• Knows enough about the content to provide appropriate editing affordances - syntax highlighting, preview, whatever.
• ... and autosaves it for you - no separate save step required
• ... then allows tagging it as being "of a particular type"... whatever that means to you
• ... then allows you to seamlessly retag it differently
• ... including being able to arrange it in hierarchies (if that made sense) or graphs (ie nodes and links - if that made better sense)
• Presents all existing content and its associated "tag graph"
• Allows linking content with each other
• ... or even parts of content with each other.
• Shows the links visibly
• Allows tagging parts of a document as being of a different tag, but still housed in the source document. Content tagged thus will be displayed in both tagged content, but the second display will be a reference to the original.
• Tracks all changes to the content graph in a stream-of-consciousness journal that's visible and accessible at all times so that you can see what you did before in the order that you did it.
• Allows you to add your own manual entries to the journal to track your thoughts at that point in time.

Such an editor could be used to make the context switches mentioned above as seamless as possible. The only bit of automation it promises is the decidedly "big brother"-like journal, but in my opinion that's something the computer should do for us - track what we're doing automatically so we can look back and "see the trace of our thoughts from before in our actions". Features like seamless switching allows easy creation of design, code, comments  and code in back-n-forth style, for example; while the ability to link content allows for todos to be linked to the point in code where change is required, but still can be maintained as a separate list of todos for easy review.

To allow easy adoption, such an editor should add the following:

• Treat an existing directory structure as the basis for its tags: folders become tags, folder structures become hierarchical tags, files become fully described hierarchical tags. If starting from a blank folder, however, allow for the initial tagged content to remain as files until the user discovers the "natural order of things" for himself.
• Seamlessly convert files to folders when the user renames them.
• Save the changes to the underlying folder as soon as possible so that there's no "last minute unsaved content change"
• Allow for some scripting/automation triggered by content changes.

UI Design Notes:

• Start the editor from a particular dir, which we'll call <dir> in these notes. The startup sequence allows choosing the default language to use optionally, else the default is text.
• The editor starts with two windows: a default one titled "<dir>" and the journal titled "stream". The former is editable, the latter is read-only. 
• The default window is "in" the default language (if chosen), ie is syntax highlighted for that language. Typing text into it and hitting enter causes text to show up like any editor, but it also causes the same text to show up in the stream window with a date/time stamp. Internally, the text in the stream window is a reference to the line in the original window, not a copy of its contents. In the background, all the text changes are saved back to <dir>.
• Typing F2 allows you to rename the window's title, which is also its tag. The editor allows creation of hierarchical tags like "tag1/tag2/tag3". It also allows setting the language of the content by naming the tag with the format "tag.type" - similar to how files are named today.
• Typing an esc puts you into "enter a tag" mode. a small edit area (or vim/emacs-style command area) shows up which allows you to type in the name of a tag such as "todo" or "comment" or whatever. hit enter when done and two things happen: the edit area disappears and a new window titled with the tag just entered shows up and focus moves to that window. Type text and hit enter to enter content in this mode, while it also appears the stream window like before. if the tag is already present, focus shifts to the existing window for that tag and text input proceeds from there.
• Selecting text and then pressing esc to select a tag will leave the text in the original window, but add that text to the window corresponding to the chosen tag (via reference again), as well as denote that the tagging event occured in the stream window. Presssing ctrl-esc instead of esc will cause the text to be moved to the tag chosen.
• Ctrl-tab will cycle through all open windows and allow you to make any window the main one. The same esc and enter behavior holds for all windows.
• Everything is save-as-you-type. No ctrl-s required. The tag is the name of the content.

More features:

• Windows are tiled, not free. this allows pinning of windows to specific locations so as to create familiar screen layouts.
• A third kind of window is a modal window, which can be in input or output mode. this will allow creation of repls. issue a command when its in input mode and view results when its in output mode, which is read-only. The "stream" window can also be recast as a modal window, allowing journal entries in input mode.

Ok, that was the easy part. What about a s-o-c language?

Such a language would:

• Allow for programs to be expressed one chunk at a time. The language runtime could be invoked at any time during the expression, at which event the runtime would evaluate the current set of chunks and determine if it can do anything with it. If the runtime is a compiler, it would decide if object code can be generated or not; and if its an interpreter, it would interpret the chunks available.
• Not have comments. Instead it would allow a more generic interpretation of commented content as "chunks from another language embedded within itself, which could be "understood" or "translated to itself" given an appropriate cross-interpreter/compiler.
• Allow chunks to be linked to each other and handle link lifecycle events. Eg: chunk A is related to chunk B could imply that if A changes, B should also change; thus flag such changes on A as requiring changes in B as well. Or, if chunk A is deleted, chunk B should be deleted as well because it doesnt make sense to keep it around anymore. Or, if chunk A is created, a corresponding chunk B MUST be created.

More thoughts on the language TBD.

Implementation Notes

Editor:

• Use node-webkit, codemirror and sharejs to allow evented entry of text
• Make a repl using a bot that's sitting at the other end of a sharejs connection.

WIP notes

Started work on a prototype using node-webkit. A basic version of the UI Design has been implemented, and the updated version of this post reflects some experiences from this implementation.