Monday, January 16, 2012

Rules for life

This is pretty similar to Slots, but has more direct application, methinks.

The app will have two main actions: Jot and Find.

Jot drops you into a Slots interface. You put down a sentence and define slots in it.
That's the first half. The other half is another Slots sentence that asks the question for which the first sentence is the answer.

Eg: Sentence1: I kept [the tools] in [the bottom drawer].
      Sentence2: Where are [the tools]?

Find drops you into a Slots interface populated with Sentence2-type sentences.

Simple. But I think it can be real powerful once we add true Slots functionality and automate actions based on query results.

Yes, I know there's www.ifttt.com, but this seems much more generic and less program-mey.

Friday, January 13, 2012

Sudocode

Create a language that abstracts out the differences between multiple OSs by wrapping them around English-y language.

Example: "Run the executable ex1" will internally do the right thing regardless of OS; on Windows it will set the path etc and execute it with backslashes and so forth; on unixes it will locate it, do the equivalent with forward slashes and execute it.

Call it sudocode. It'll be the Cobol of the 21st century.

Monday, December 26, 2011

Focus considered harmful

Most of us are used to the concept of focus in modern desktop UIs - there's usually one app that has it. But what is it? Essentially the app that has the magical focus is the target of all indirect input.

The way I see it, there are two classes of input devices in a common desktop system: direct and indirect. Direct input devices can readily and immediately target any app; while indirect ones cannot. Mice are direct input, keyboards are not.

The key problem with input is that you need to decide who gets it. This was not a problem with the OSs of yore where there was just one running app at all times, nor is it a problem with the new tablets because they have only one "full screen" app running at all times. Desktops with their multitasking, non-full screen apps however, have the problem in spades. Now that I use the Mac, I'm ticked off that cmd-tab doesn't necessarily mean you'll get the app front-n-center. You see, if you'd minimized the app before, you have to add opt to the cmd-tab just before the app you want so that it will pull up the minimized window as well. Otherwise its just the menu on the top for you. Why? I'm sure there's a really super important Mac/Apple reason for this behavior but it pisses the heck out of me.

Enter the newbie, exemplified by, in this case, my parents. My parents cannot figure focus out. They will painstakingly fire up an app, and start typing in the hope that Windows will magically figure out that where they're looking at (or were looking at before looking down at the keyboard) is where they want the keystrokes to go to. Of course, this is without having brought the cursor to the text field in the first place so all that input does is bupkis (Where do keystrokes go when typed without focus, I wonder). And if they'd actually remembered to "make sure the blinking line is at the user name field", McAfee decides to popup a message that silently steals that focus away.

Focus is an implementation artifact and a leaky abstraction at that. The currently available approaches are:

  1. Its a non-issue: This is what tablets do - you dont need to worry about who gets the input if its completely clear that there's always only one such target.
  2. No more indirect input: Again, this is what touch devices do; you don't need to pick a target when the act of providing input actually selects the target.
  3. Provide a switcher: This is the Alt/Cmd-tab solution. Provide a way to switch the target of the indirect input. However, as we've seen above, this has problems.
Would it help to be even more explicit about who gets the keystrokes than just highlighting the app's title bar? Would it help to avoid modal (and non-modal, but focus-stealing) dialogs?

Is there a better way that tells the user that any indirect input will be sent to a particular app? Or, alternatively, is there a seamless way of "figuring out" the intended target? I can think of two promising avenues:
  1. Computer vision is slowly getting better to the point where head tracking using web cams is actually possible. Can this reach the resolution required to figure out the app the user is actually looking at? Lots of false positives in the future down this path, but definitely its a head-on approach at the problem.
  2.  No more leaks in the abstraction: This is a cheaper solution to computer vision and essentially involves ensuring that:
    • There're separate "user controlled space" and "system use" space and never the twain shall overlap.
    • User controlled space will always have one app that has focus. It will lose focus only when the user wills it to. The action to switch from one app to another is atomic and results in the other app getting focus. This will be true even on tiling window managers.
      • Dialogs and other such UI elements that halt the app before user interaction will work as before.
    • In addition to serially switch the target of input using Cmd/Alt-tab there will be a key sequence to switch between apps - maybe like the Ctrl-Shift-F1-7 sequence that Linuxes have for the default ttys.
    • System use space will be read only, except for some suitable way to scroll through messages.
Todo: figure out how to deal with system messages that require immediate attention.

Functional vs Implementation Models

Our apartment building has a down ramp from the elevator to the basement level. My kids usually run down this ramp, stopping just short of the driveway for cars exiting the basement.

Of course, they stop short because they've been told that "Cars are coming"; or as my under-2-yr-old says "because car comin".

The other day he ran down the ramp as usual, stopped short of the driveway and chanted his usual "because car comin"; then paused to look, said "Car no comin" and proceeded to walk across the driveway.

I'd almost started to blurt out the "Look to your left, look to your right, then look to your left again.." routine in an effort to instill traffic safety at a early age (I know, parents) when he said "Car no comin". Since he did that, I kept quiet.

It then struck me that his way was much better than the "Look to left routine" because he's formed a mental model of when cars come and when they don't. Once he has that model, he's free to decide what to do: cross the driveway, stay put, or whatever.

The problem with the routine is that its prescriptive; and therefore - by definition - too restrictive. How would a child's mind associate such a routine to the situation? It seems to me there are quite a bit of wrong associations that can be made to the one right one:

Right:

  • When crossing a driveway, do the routine for your own safety.

Wrong:

  • When you run down a ramp, at the end do the routine
  • When you stop, do the routine
  • When daddy shouts as I'm running down a ramp, stop and do the routine
More importantly, the value of looking left again will be completely lost on a child IMO. Wouldn't it be much better if he arrived at that step by himself?

Thus too, it seems, with software. As long as we build in the right functional models, it should be easy to instill the right implementation model; and by extension - modify or maintain it.

Wednesday, December 14, 2011

OS.next

  • Hardware Compatibility:
    • Run on x86, AMD
  • Software Compatibility:
    • Support Win PE, Unix executables and Mac executables natively.
  • Boot loader:
    • Support booting from usb, sd card
  • OS:
    • Ability to have the whole OS on portable media (eg SD card) if so required
    • File system:
      • Tagged storage instead of directories
      • Sensible file system hierarchy that combines win/unix/mac concepts
      • FS shadowing for legacy executables so that they think they're still "at home"
      • Ability to move /home to an sd card if so required. More generically, ability to configure OS to have any "standard, should be on boot volume" directory elsewhere.
    • Memory:
      • Nothing special as of now.
    • Disk:
      • versioning of files built in. ie some kind of journaling file system.
      • Security built in - encrypt a file, directory, mount point or whole system.
    • UI:
      • Light, capability-based UI framework.
      • Support for UI paradigms such as radial menus - controls that use Fitts Law better, basically
      • "Back to the place I was before" mode for people not ready to make the plunge
      • Keyboard support for everything. EVERYTHING!
    • Software in general:
      • Solve the package manager/installer problem for good
      • Solve the registry vs 1000s of config files problem for good
      • Rolling versions of the OS, easy revert back to previous version
      • Tool to see files changed due to any install/ version upgrade built into the OS
    • Shell:
      • normalization of options and formats across at least the busybox commands
      • autocomplete for command options got from info/help files
      • oo-ize commands (ie dir.ls instead of cd dir;ls)
      • Structured editing of the command line a la zsh
Updates to this post as I think em up :)

Update #1:

Show a schematic box diagram of the computer system as it boots up and have icons (or text) depicting the status of each box in the diagram.

This is in lieu of the "screens of flying text" a la linux or the "startup progress bar that never quite ends" of windows/osx

Also missed out adding my idea about intelligent workspaces, but it should be in this list.

Tuesday, December 13, 2011

Codermetrics: A review

I picked a book called Codermetrics(book, website) yesterday - one of those impulse purchases because I liked what it said on the cover. I've not finished reading it completely, but I went through most of the meat of the book and its an interesting concept.

The core concept of the book is simple: Measure a developer - and by extension the development team and organization - in the same way the world measures sports persons, teams and organizations. The book's name is a play on Sabermetrics - which apparently is all the rage in Baseball these days as far as metrics go.

After a somewhat drawn-out beginning, the bulk of the book is focused on defining metrics for developers, teams/organizations and their combination. The developer metrics measure how good a developer (called coder throughout the book) is at advancing the team towards the org's goals (Offensive Impact); how good s/he is at preventing or avoiding the things that detract the team from the org's goals (Defensive Impact) and so forth. Specific attention is paid to work done beyond the call of duty (Pluses) and help given to others (Assists).

Next, the organization is defined in terms its successes (Wins), failures (Losses) and position relative to competitors. In each case, the end of the chapter gave some examples or archetypes that arise from specific combinations of the relative values that the metrics may take, for eg, what separates a junior coder from an architect, or what separates an enterprise company from a startup.

The final set of metrics - called Value metrics - were the most interesting. While the previous sets of metrics required some input data from the real world, these ones were derived from other metrics and were intent on exploring the contributions of individuals factored into the larger scheme of things in the organization. Terms like influence that a particular developer has on the team and company, the teamwork  demonstrated etc become measurable things!

The final section of the book has some advise on how to go about implementing such a process in an organization.

Overall, the concept of the book and the simple language used to explain the concept is a win for the book; as is the author's repeated disclaimers that we should heed to his overall construct, not the specific metrics that he provides. This is sane advice and a welcome change from the typical prescriptive nature of texts on metrics.

I've started trying this out on one of my teams which is particularly feeling the pain of under-delivery. Highly recommended.

Sunday, December 11, 2011

Information Density and textual vs visual

I was reading the Wikipedia page on Information Theory as part of my read the wiki project, when it struck me that there's possibly an objective way of measuring the effectiveness of text vs visual programming languages using Information Theory.

The central concepts (whose math I'm admittedly unable to fathom) are those of information, information rate, entropy and SNR. One of the age-old cases for text-based programming ( and therefore against non-textual programming languages) has been that it has very low SNR and the "information density" is high for given screen real estate.

Is that really true, though? How much "noise" does syntax add? On the other side of the spectrum, I've seen infographics that assuredly deliver more "understanding" in the given screen space than the equivalent textual description. Is it possible to design an "infographic-style" programming language that packs more power per square inch than ascii?

It would be interesting to do some analysis on this area. 

Descriptive Drawing Language

I have been thinking of the way by which UI mockups are created. In most Visio-style applications - which are the mainstay for this kind of thing - there's usually a pallette of known "widgets" that can be dragged onto a canvas and put into place using the Painter's Algorithm. The good part about such an arrangement is that they allow easy creation of UI mockups, and thereby expression of the look of the app-to-be.

This works when the UI framework is an established one and the set of widgets is known. But what if you're trying to create a new UI framework and you want it to be completely (or sufficiently) different from the current pack?

The other problem I see with the current way of doing things is that UI mockups/wireframes are still woefully disconnected from the rest of the development process: the designers build their mockups, pass it onto the graphic artists who finish up the design with dimensions, colors etc and finally deliver a finished design (typically in html for webapps, most probably psd for others) to the developer - who now has to tear down the carefully crafted design to get at its constituent pieces.

I was thinking of an alternative that's inspired in part by Graphviz.

Imagine being able to draw a picture by describing what it contained. So a description of a wheel would be something like:

wheel:
    circleA: circle
    circleB: circle
    line1, line2, line3, line4: line

    circleA contains circleB
    line1: pointAt(circleA, top) - pointAt(circleB,top)
    line2: pointAt(circleA, right) - pointAt(circleB,right)
    line3: pointAt(circleA, bottom) - pointAt(circleB,bottom)
    line4: pointAt(circleA, left) - pointAt(circleB,left)

The syntax is obviously very handwavy at this point, but hopefully the intent is clear:

  • You describe the scene, not paint it
  • You define relationships between the objects not place them in specific positions. 
  • You provide dimensions only if absolutely required, else the system figures it out for you.This is obviously not intended for a "to scale" diagram; although I can already imagine a Graphviz style mode that outputs the same format as the input - only annotated with dimension information that can further be processed to make it to scale.
The advantages I see with such an approach are:

  • Diagramming becomes descriptive. It could easily have a Canvas backend as a GL one, but the scene would be the same
  • UI wireframes can be built "from scratch" easier than before and thus support early exploration of new ways of expressing design intent
  •  UI wireframes become instantly accessible to version control
  • Developers can literally write diagrams!
Design notes:

  • There will obviously be primitives - line, circle, arc etc come to mind.
  • The system will allow creation of compound objects from simpler ones and primitives
  • Spatial organization in a 2D plane will be represented by a logical grid, or using predicates such as "contains", "to the left of", "to the right of" etc
  • Overlapping objects will be represented by a Z-ordering (as in most such tools) using predicates such as "in front of" "behind". Alternatively, the system could allow depiction of each "layer" in the Z-axis as a slice within which no front/behind predicates are allowed
  • Objects can have "anchor points" defined which allow other objects to use those specific points in other areas of the script.

Sunday, December 04, 2011

My Bowerick Wowbanger homage... aka the wikipedia breadth-first read

I've been meaning to start this for quite a while now.

If you've read the Hitchhiker's guide to the Galaxy, you'll probably remember Wowbanger the Infintely prolonged who takes upon himself to insult every living being in alphabetical order just to keep himself busy?

Well, aside from the immortality, the need to keep myself busy and the insulting bits, this is my Wowbanger-esque project.

I've started reading the Wikipedia at the Computer Science page, and I aim to keep reading it in breadth-first fashion from that page onwards. I hope to read one page every day. Along the way, I'll track my course with a graphviz graph that will eventually be a map of the wikipedia subtree (subgraph?) that is rooted at "Computer Science". Simple rule for picking the next page to read: I'll pick the one that I know the least about.

I've also created a wikipedia account, should it come of use somehow. I'll look to see if I can fix simple typos and such - as they arise; and if there indeed are pages that are stubs that I can fill out, I'll try.

Why
Because one my (admittedly naive) goals in life when I was still studying was to "know everything there is to know about computers". A decade and a half later I still remember it, so it must mean something. I figure this is the least I can do :)

PS: Yes, I did think about what I'd do if a page I'd read already changes. I dunno. We'll see how it goes. Even Wowbanger did Arthur twice :)

Thursday, December 01, 2011

I visited JavaFXville and all I got was this post

I tried to write an FLV player in JavaFX today.

I tried very hard.

I didnt get very far. Or when I did, it was not obvious how far I actually had.

It all started with me looking for a trustworthy way of downloading Youtube videos - you known, the Randy Pausch kind. To share with my team easily.

Googling for Java and FLV showed that JavaFX spoke FLV natively. Awesome. I'd heard enough of JavaFX in its previous incarnation as F3 to think it was worth a shot for a weekend project. Not to mention this is the closest that one can get these days to simple declarative GUI.

Of course, I'm sadly backed up on the news front. Oracle, in its infinite sagacity, decided that the declarative DSL was to go; replaced by a Java API. So what looked like a simple sample in the 1.3 documentation morphed into this beast of boilerplate in 2.0.

But wait, there's FXML - the Oracle blessed replacement for FXScript, right? It IS XML, but its still declarative, right?

Wrong. Conjuring up the right incantations to get MediaView instantiate a Media object is obviously beyond me. I've never liked XML as a UI description language and this abomination only stokes that feeling.

So I give that up and get back to the POJO version; which works for the sample FLV they provide.

So I proceed to download a Youtube video using a Sourceforge-based downloader (also Java) and the bloody thing doesnt load. Maybe the downloader is broken?

I dunno. The code looks like its doing multiple passes on the embed element's params; and justifiably so because that value's a snakepit of multiple URL encode sessions. Security by obscurity, perhaps?

Ugh.

Its 4 in the AM, and all I have to show for my efforts is this blogpost.

Final nail in the coffin: I now read that JavaFX has spotty support for Linux. The whole intent of this was to create something in Java so that I can share it with my linux-using team.

FML.

Saturday, November 19, 2011

Droidcon Bangalore 2011

Had great fun attending the first Droidcon in India at namma bengaluru. Here's my somewhat longish take (in schedule order):


The Good:

  • Diogo's keynote on Cyanogenmod's growth and potential. Interesting to know that they support 80 devices. My cheapo LG Optimus P 500 isnt one of them, however :(.
  • James Hugman's talk on making X-platform apps suck less. Bonus: the Kirin framework. Interesting idea of writing business logic in javascript, but using the builtin webview-based javascript engine as the runtime. Definitely worth checking out. 
  • Hard Earned Android experiences by Khashif. Lots of sane advice here. Some interesting face-offs with his own (seemingly very young) CEO made me feel like I was at a tennis match but in all a nice talk. Most controversial point: optimizing database writes. 
  • New features in ICS: Good round up of the new features. Had to ask the question on the new Google expectation to move physical buttons to soft ones even on phones. More on this below.
  • App demos: Selected output from the Hacknight event that HasGeek held as a leader for this event. droid2chrome was interesting, as was the bookcompare (i forget the actual name). The former was interesting because most of the team members didn't want to do android and yet managed to do something; while the latter was interesting cos they actually screen scraped book prices from indian vendors!
  • Anand Virani's end-of-day talk on the mobile growth wave: I caught the end of this talk but clearly this is a man who can talk and present well. Nice projections on how the mobile world is shaping up. Some sharp questions on open sourcing Qualcomm tech which he quite adroitly side stepped without pissing people off :) 
  • Tibaut's keynote on i'm not sure exactly what (it had strands of droidcon history,android growth and political incorrectness), but it was by far the most entertaining and engaging talk i've seen in a while :). He made an interesting connection between Minitel - the French govt sponsored draconian internet offering that was in blue, green and while colors - and the recent fragmentation of the mobile market between blue(nokia), green(android) and white(apple). I'm not really sure what his final slide asking us to challenge the status quo of mobile really meant, but he DID get a few hundred indian geeks to actually talk to each other! That's an achievement for sure.
  • IPR aka Who let the lawyer in the room: Again, I caught the last 5 mins of this talk, and immediately felt I should have taken it in whole. Malavika looks like she could be the india PJ and she likes open source. I asked her about groklaw's impact on relations between devs and lawyers, to which her quip was: "I think its great because usually developers never read the terms and lawyers never see the apps"! Nice.
  • Indoor tagging engine: Was intrigued before the session and liked what I saw. Tagin is a project that allows identify indoor locations based on the surrounding wifi beacons. If this thing is accurate 50% of the time, it has great personal potential. The only downside that I see is that you have to have wifi on all the time. Barring that, however, I can think of some simple applications that extend the GPS's accuracy - especially within buildings that have lots of wifi routers - like apartments. Example: "Ding! you're near the mailbox, do you want to drop your bills off?"
  • Creating apps that work at all sizes: Lots of good information here for UI devs. I couldn't help but feel that the raster vs vector wars were being fought all over again on mobile, however.
  • Android and Arduino: Very enthusiastic presenter who knew his stuff and presented it in an honest, no-frills way. Learnt how easy it was to interface with arduino more than anything else; and that all this stuff (and knock-off cheaper boards) were available locally. Now all I have to do is magically make my family disappear so I could just hole up playing with these toys :)
Ok, could be better:
  • Tech challenges of apps in limited connectivity: This is a topic that could have been a lot of things - especially given that mobile is becoming cheap and offline apps and syncing them up is something (IMO) a lot of developers should be worrying about. While the initial "here's the problem" part of the talk was good, the rest felt like reading a Craig Larman architecture book - dry, high on theory and boring as hell. If you read between the bullet points, there were gobbles of information just waiting to be let out. They stated the problem, stated the things you have to consider and then stopped. A better approach would have been: "here's the problem, here's the things you have to worry about, and here's what we did. Now lets discuss". My other takeway: offline and sync should be a framework problem, not an app developer one. More on this below.
Meh:
  • Android Service Patterns: Was over my head because I'm an android tourist dev. However, I'm surprised someone built a new IDL to define service contracts. Doesn't the world have enough of those already without android creating one more? God forbid some (very possible) future where EAI vendors have to support AIDL! Oh look, the horrors of integration combined with those of android fragmentation!
  • Honeycomb codelab: There were glitches in the setup - primarily because most people did not have the ICS Dev setup on their machines. It might have helped to have a note on the schedule asking people to prep for this one. Also, there was a whole lot of "how" and no "why" at all in the content - but maybe that's because I don't know the nuances between honeycomb and older versions. And: it WAS called a codelab, so maybe I shouldn't expect theory or reasons. Regardless, it would have been nice to have a brief this is why what we're going to do is important/fun/great. It lacked that oomph, IMO. Personal peeve: Seemed like the presenter was a bit miffed that "in this day and age" we dont have connectivity. The whole tutorial was online, with no way of even getting offline as it was on appspot. We ended up curl-ing the pages so that our unconnected buddies could follow along.
    • Note to organizers: Tell people to setup machines prior or tell them about the thumb drives at registration
    • Note to speaker: Do not expect a perfect environment; have fallbacks. Most of us were trying very hard to manage to follow you, but a little understanding would go a long way.
    • Note to self: Read up on the content to know the why before attending a codelab instead of bitching about feeling listless at it :)
  • Android app: A deconstruction: I was distracted; hence the meh. Seemed like a pretty ok breakdown of building an app for AAA. However, I've attended a similar talk earlier, and the guy then made me want to have the app; so I was interested in how it was made. This was a lot of the hows; and a lot of it was pretty standard stuff when looked at through the lens of web development. My bias, I know.

Did not attend, would have liked to:

  • Android Multimedia Internals
  • Pricing models for apps
  • Continuous delivery for android apps
  • Cloud to device messaging.
  • Android Memory optimization
  • Enterprise App Development, admin api.
  • Close to metal programming with the NDK.
  • OpenGL on Android: My colleague attended it and liked it.
  • Humanizing Android.
  • Extending android with new devices
  • From stock to CM: Ericsson.
  • Smartphone Platform Frameworks.
  • Android porting for dummies.

Just plain Did not attend:

  • Android UI secrets: Heard from another guy that this was not as secretive gyaan as expected, but that's second hand info.
  • Sensors on UI: My colleagues attended; they were not the right audience, however. Must have been interesting to the the right people.
  • Demystifying mobile advertising
  • Infusing Android with social.
  • ICS Camera and connectivity.
  • The Phonegap session.
  • Android products with TI tech.

Sitting through some of the sessions, I couldn't help but feel that Google should:

  • Think of phones as phones, not as small tablets. Some of us actually use these things when we're not looking at them, and some of us actually (gasp) LIKE  "Call" and  "hang up" buttons. Not all things are meant to be used with two hands.
  • Create offline and sync apis: I'm not sure if there is an api for these in the android platform; but if there isnt, there should be one. I spoke to at least 2 app developers building their own solutions for these problems, and clearly, these are not application-level problems. "In this day and age". 

Surprises:

  • Robosoft: A company based in Udupi, doing Android! Go figure. I might just retire there - if they'll have my sorrily backed-up-in-tech-skills behind :)
  • Copperspiral; Fronted by a very informed and polite Vineeth, this RFID comapny was the first that showed up in India when I googled for local vendors; and I had quite a few informative chats with him. I'm not an RFID expert, but he's the guy to talk to if you are a hobbyist, methinks.
  • Github: They sponsored the party (which I couldnt attend owing to stupid excuses such as having a family) was conspicuous by they absence. Sure, the cut outs were there. Maybe there was more of them at the live music show? Thanks anyway, github. Love your software, love your support.
  • The 2 guys volunteering to set cyanogenmod for random strangers: Completely unplanned, apparently, and completely cool. My LG still didnt get love, sadly.
  • Dextera: I learnt that this company from Cochin built a Siri-clone called Iris while at their CEO's session. Have to definitely check it out.
  • The venue: Did not expect this good a venue, actually. Except for the small room, a top-notch place; and still a little away from the madding crowd. I particularly liked the steps that also doubled as an access ramp on the way to the courtyard. I'm by no means an architecture expert, but I have not seen that before.
The experience:
Hats off to HasGeek for the organization. I'm one of the entitled bunch that gave them grief about the lack of net connectivity, but Kiran's saga of their attempts to get wifi for a month won me over. Kiran, if you're reading this, I'm the guy who was willing to pay Rs.500 more. You have my apologies.That said, here are my suggestions:
  • A panel discussion would be nice. I know you guys had plans for it, but it didnt work out. Now you know the people will come, so have one for Droidcon 2012.
  • An app for the event. Methinks this could be community-sourced. I met some guys from Cochin who were in mobile tech training. Seems like they can whip this thing out :)
  • Prior notice for things like the codelab. Update: I just read your blog. You did tell them in advance.
  • Content: Android scripting, esp JRuby on Android aka Ruboto.
In all, a fun two days.

Friday, October 28, 2011

Jack and Turing completeness

From the previous posts on Jack's primitive capabilities it should be pretty obvious that it is turing complete. It supports sequences and selection naturally; and allows iteration of any node's children.

The intent, however, is to NOT use Jack as a primary programming language. I do not see a hello world in Jack's future - at least not in the conventional sense.

Jack's Turing completeness is intended to provide a full complement of computing power so that program logic may be effectively TRANSFORMED. The more appropriate set of hello world would programs may be:

  • A program that edited another program and represented the editing as a sequence of Jack steps
  • A program that reasoned about another program using Jack
  • A program that expressed the history of a program and exposed its "why" using Jack
  • A program that expressed the future roadmap of a program using Jack

Jack: the whys and wherefores

As I pen down these thoughts that I have about Jack, I realized that there's a little bit of the "losing the forest for the trees" going on - while the initial posts were  what-oriented, the latter ones have been decidedly how-oriented. The "why" is kinda sprinkled around. This post attempts to correct that gap.

Jack is the culmination of a series of vaguely connected concepts that have been swimming below the surface in my consciousness. Some of the individual strands are:

  • Code today suffers from want of better abstraction for itself. The simplest manifestation of this is the conflation of its persistent, modeling and editing formats into one single form - text. I posit that treating code in its true form - an AST - directly will reap higher benefits.
    • There are sub-strands to this where concepts from Subtext (and the whole language workbench movement, for example) finds resonance with my thinking and therefore has been subsumed into it.
  • True code structure is latent and the structure made apparent by its organization into files and folders is misleading. I posit that all code will look like graphs (a la social networks) once we represent (and visualize) the actual relations between code components. At that point, graph CRUD techniques can be used to manage the code and graph analysis techniques can be applied on it to reduce the comprehension overhead.
  • Today's applications require connecting together at least 4-5 different languages. There is no conceptual glue to hold these together, splintering comprehension and adding to the accidental complexity. I posit that treating apps as a graph containing code components will allow for easier comprehension and maintainability.
  • Today's development practice is woefully snapshot based. There is no way to represent the code's past (so that you may learn from it or make corrections easily) nor its future (so that you may plan for it or code in a certain way)
  • The average developer follows Reality Driven Development, no matter what methodology they say they follow. 
  • All of this has direct bearing on EXISTING code. Most approaches to bettering software development talks about tools and techniques that you can use on your NEXT project. I'd like to come up with tools and techniques that you can use on the CURRENT or PREVIOUS project, or even that ANCIENT one from the '80s - the one that's already in production and making money for the business and will never be replace (at least not completely) that you have to either maintain or talk to. This is also the hard and unglamorous side of the development cycle - the maintenance side. However this is also the place where code spends most of its time and yet is mistreated much. 

Jack is the code name I use to represent my solution to these issues. It started out addressing the format and representation issue and has burgeoned into including the rest of the concerns. It therefore has gained the larger qualities that any solution to the problems above should have:

  • It should not favor any particular language or framework - at least not design and definitely not in the ultimate implementation.
  • It should not introduce another language either; instead use existing language runtimes as host environments and introduce ways to work with the code by adding metadata and the ability to programmatically refactor it.
  • It should introduce a runtime environment that is live in the same way a LISP repl or Smalltalk IDE is. Compiled environments should get as close as possible to this ideal. This is fluent.
  • It should allow for models of the code to be built at varying levels of abstraction from the code itself. These models would be used to understand code and build it. It should therefore subsume disparate technologies used to build working apps or make it easy to add new ones into the modeling platform.
  • It should allow incremental adoption of itself. No big bang, lose-your-religion type steps because we're dealing with maintenance code and developers following RDD.
  • By definition, therefore, all tools and techniques within should have models for incompleteness: incomplete code, design, architecture, relations, etc. This should be used to help comprehension of the code in "comfortable bite sizes" and to allow description of future code whose specifics are not yet known.
  • It should have a native access to the history of the code so that comprehension can follow. This is the reason the previous post has the scm FFI.
So there you have it: the WHY and WHAT of Jack in one post.

Thursday, October 27, 2011

Jack: some new thoughts, a few iterations

Some strands of thoughts have come together of late as I re-read my posts about Jack, watched Subtext and talked about CodeViz to people. Things congealed into place as I revisited OMeta and decided to give it a shot as a prototyping platform.

Iteration 0
As a precursor to trying OMeta out I listed out the features that Jack was supposed to have based on my posts. Here's what I came up with:
Transforming this to OMeta syntax, here's what I came up with:
As you can see, some compression has happened from my BNF-ish syntax to the OMeta one.

Iteration 1
Pumped by this quick success, I immediately started counting my unhatched chicken. How would people easily share Jack code, I wondered? I'd already thought of YAML as a storage format so, I quickly wrote up the YAML version of the function from above:


I also came up with the whole slew of pipelines that would bring Jack (and some part of Fluent) to life:


text syntax   --JackTxtParser-------> JackAST
JackAST       --JackUIRenderer------> JackViewTree
JackViewOp    --JackUIModifier------> modifiedJackViewTree
JackViewTree  --JackInterpreter-----> JS Evaluation
JackViewTree  --JackCompiler--------> JS Source
JacViewTree   --JackTxtGenerator----> text syntax output


Iteration 2
Then reality set in. What are the primitives I'd need to build for this to take off? A reverse read of all jack posts revealed the following set of "things to be built":


  • A primitive conditional
  • The ability to make a module out of anything and vice versa
    • make module (code list)
    • devolve module into codelist(module)
  • A way to denote a function as optimized via a Foreign Function Interface (FFI). This could be in the fact language, but this means the fact language and the compiler should talk.This is probably required anyway; the interpreter should be able to query the analyzer for facts' veracity.Note: This facility is like annotations, but doesn't entail arbitrary side effects as the annotation processor allows through arbitrary code running.Each FFI, however, should expose a way to call the underlying optimization, with ways to map values to and forth.
  • The ability to refer to any single piece of code: built into the structure already. url tbd
  • The ability to refer to any set of code pieces: define a continuous code range, define an arbitrary list of code pieces (is this required)
  • The ability to comment on such a set of code pieces: ie attach a comment to a code set. this is somewhat similar to the modularize reqt above
  • An FFI to scm tools with the basic functions supported
    • check in/out
    • commit
    • branch
    • merge
    • snapshot
  • A functional version expression
  • The ability to represent WIP code
  • Most importantly, an interpreter that supports all this

Iteration 3
Then I took a step back and looked at the complexity growing. Could there be some abstraction done? Here's the outcome of some furious (re)thinking:
  • There are nodes
  • Nodes have attributes
  • Standard attributes are:
    • [id] is used to uniquely identify a node. it can be system-generated or user-provided
    • [comment] is used to provide a comment about the node.
    • [fact] is used to state a fact about the node using FOPL; and is typically used to derive some "higher order knowledge" about the node
    • [name] is used to provide a referencible alias for the node.
    • [kind] is used to identify the type of node. this may be dynamically assigned to implement duck-typing.
    • [context] is used to "run" or execute the node.
  • Other attributes can be added at will and used in execution
  • A collection of nodes is also a node, and therefore can have the same attributes and be executed similarly.
  • Standard nodes are:
    • base:
      • block : a continuous sequence of nodes to be executed one after the other
      • if : a node that conditionally executes one of its child nodes
      • define : a node that adds a name to a node
      • function : a named block
      • module : a collection of functions
      • app : a collection of modules
    • meta:
      • group : a node that groups other nodes (and optionally names)
      • split : a node that splits an existing group
      • insert : a node that inserts a node at a given point in a collection
      • delete : a node that deletes a node from a collection
    • versioning:
      • checkout : a node that checksout a version of its input node ref from scm
      • commit : a node that commits a version of its input node ref to scm
      • branch : a node that branches a version of its input node into a new branch
      • merge : a node that merges two node refs
      • standin : a node that can stand in for any other node. to be used for nodes that dont exist yet 
Thoughts
I reached this far and realized something larger abstractions are possible:
  • Jack code could be stored in any data format that can handle trees - backward compatible data format!
  • Jack could use any language as host - backward compatible code! Jack is "just another scripting language on top of $my_fav_lang
So, it sounds like:
  • Jack's true role will be to consolidate sequences to specific hosts, create and break interfaces and manage the change that happens - a sort of super shell.
  • Ultimately Jack should enable code comprehension and legacy support.
Maybe Jack should be renamed Glue.

Tuesday, October 18, 2011

Intent to sms contact

Write an android intent to allow sms-ing a contact, cos the default phone app doesnt have one.

Update: This has been Implemented. Woohoo!

Saturday, October 08, 2011

Physics of Software: a review

I stumbled upon an interesting series of blog posts that attempts to bring in a "physics" for software. This attempt in and of itself seems to me a worthy cause as IMO Computer Science could do with some foundations like these. There's too much on the engineering side and less on the science one.

The other reason I liked the series (which I'm still reading) is my own latent ideas on using FEM-ish analysis on the structure of software.

Halfway through the series, however, it seems to me that the analysis is overly reliant on real world physical concepts. Software having a center seems fine, as does mass and inertia; but extrapolating that to speed and acceleration seems contrived.

My own take is that software DOES have a physics of its own. However, concepts from Natural Physics may be applicable as analogies, but not directly to express this phsyics. My suspicion is that the physics of software would be multi-dimentional (which the author of this series concedes as well), graph-like (which he also alludes to with his attraction-repulsion diagrams), fractal (also stated, but not evolved fully) and much in need for a set of operators that define the effect of change (which I've not found yet in these papers).

Still, a very worthy exercise indeed; and one that introduced me to Coplien's forays into the area from the late 90s. Bonus: Reinteroduced me to Christopher Alexander's Design pattern concepts.

PS: While the posts dont talk about change, I was led to them via link from Micheal Feather's attempt to quantify design change by measuring correlation between methods/classes changed in a single commit.

Wednesday, September 14, 2011

On self containment

I like systems that are self contained. No unnecessary dependencies.

I woke up today and decided that lispdown needs an implementation. After all, the rules are simple; should be a shell script, right?

True; but I couldn't help thinking: shouldn't I write lispdown in Lisp? Whoever needs it (lispdownville: pop 1) obviously will have lisp - and that's all they'll need. Sure they most probably will have bash, but why use it at all?

This is the same kind of thought process that led me to build cuke4jas. I found joy in the fact that the application being built and the tools being used to build it are both from the same base platform/language.

I know everyone has to be a grasshopper first, but...

... could I do it matrix "I know Kung-fu" style , please?

I bull-in-a-china-shopped my way through SICP so that I can get to the metacircular evaluator bit as soon as possible, but now I see even more clearly that each chapter firmly builds on the previous one in this book.

As I read the implementations of eval and apply I realize I have to go back and read the foundational material before I can expect to just grok them. My kingdom for the knowledge packs from the Matrix :)! 

Sunday, September 11, 2011

On a personal note: Remembering 9/11

It's been 10 years.

Like many in our generation, I remember where I was and what I was doing that day. I know Latur probably had greater deaths, as probably did Bhopal; but this is the one that I was closest to.

I lived in Parsippany, NJ then and remember heading out to my car to go to work only to be told that the towers were bombed. I remember watching in disbelief as people jumped from one certain death to another. And then the videos of the plans actually crashing. And then the short messages people sent their dear ones from the planes. I remember a nation coming together immediately.

I remember walking into our regular diner the weekend after 9/11 and being asked: "You're indian, right?" by the guy who'd seated us every time before. If I were him, I'd ask too. I remember the Indians touting American flags on their cars for the next few months.

I remember feeling sad that this tragedy of epic proportions was becoming a media circus, yet being so part of it that I couldn't look away. I remember going to Ground Zero a month or so after 9/11 and feeling viscerally affected - the scene left enough clues as to how it happened: from the huge gaping hole with twisted steel and crumbled slabs of concrete to the charred buildings nearby I could imagine the myriad personal horrors that the people who were there on that day went through. I remember being annoyed by the street hawkers who had already started selling touristy swag nearby, but was numbed enough by the event and the place themselves to ignore such pettiness.

I remember going to Ground Zero last year and still feeling the same as before. I remember being a tad ticked off that the scene didn't look much different from before - maybe it had, but where was the monument to all the people who died that day? Where was the imposing answer that showed that we as humans will not bow down to terrorism?

I remember the firemen being nice to my kids and posing for pictures (they asked to sit on the fire engine), but couldn't ignore the thought that maybe some of them had friends who were no more. And I still couldn't understand the picknicky people swarming the site nor the hawkers with their "I've been to Ground Zero" T-shirts and mugs.

Fiercely apolitical, I have actively shunned news media in all its forms since my college days, but I couldn't get enough of the coverage of the Bin Laden capture. Real or not, it gave me closure to those events from a decade ago - and I didn't even lose any dear ones.

I cannot remember  a date to save my life, but I remember going to the World Trade Center on July 4th 2001.  I was badly hung over from the previous night and the meal from the restaurant at the top of the tower set me right so I could enjoy the view. I remember there being barbed wire surrounding the deck to prevent jumpers - seems kinda pointless now.

I kept the ticket stub to the towers and joke to my friends that I'm keeping it for its antique value, but whenever I happen on it while sorting my keepsakes, I'm taken back to this day, ten years ago.

Thursday, September 08, 2011

A personal exploration on Java's Enum declaration

What:

A personal exploration on why Enum is declared as Enum<E extends Enum<E>>

Why

I stumbled upon this interesting question a while ago when a co-worker showed it to me. Googling it provied enough links to satisfy my initial curiosity, but recently the question surfaced again; so I tried to implement my own version of the explanation provided online to "see for myself".

What follows is largely based on Angelika Langer's FAQ and can be thought of as my worked-out version of that page. My enum class is called Enum1 to avoid clash with the SDK's version; and has been tested with JDK 1.7.0_07.

The exploration

Say you're the Java team and you want to support syntax like:

    enum Color(RED, GREEN,BLUE);

... so that it could be used like so:

  • As constants : Color.RED ... which means that public static constants must be available
  • In variables : Color color=Color.BLUE ... which means that it must of type Color
  • In comparison : color==Color.BLUE ... which means it must implement compareTo()

Further, say you intend to add this feature with as little syntactic sugar as possible; so you want to convert the basic enum syntax alone via a special operation in the compiler and generate everything else using standard Java language constructs.

The generated code would look something like so:

    public class Color implements Comparable<E>{
        private static final Color RED;
        private static final Color BLUE;
        private static final Color GREEN;
        private static final Color $VALUES[];
        static{
          RED = new Color("RED",0);
          BLUE = new Color("BLUE",1);
          GREEN = new Color("GREEN",2);
          $VALUES = (new Color[]{ RED, BLUE, GREEN });
        }

        private final String name;
        public final int ordinal;

        //private since nobody outside needs to use this
        private Color(String name, int ordinal){
            this.name = name;
            this.ordinal = ordinal;
        }

        public final int compareTo(Color c){
            return this.ordinal-c.ordinal;
        }
    }

If you wrote more than one of these, you'd have quite a bit of logic that's common and could be extracted out:

  • Each item in the enumeration has a name and an ordinal.
  • Each item is built privately and the instances built statically are made available publicly as constants and via an array
  • The constants have comparable values (optionally internal)

So you could write a base class like so:

    public class Enum1<E> implements Comparable<E>{ //line1
        protected static Enum1 $VALUES[]={};

        private final String name;
        public final int ordinal;

        protected Enum1(String name, int ordinal){
            this.name = name; 
            this.ordinal = ordinal;
        }

        public final int compareTo(E e){
            return this.ordinal-e.ordinal;  //line 2
        }
    }

... where Enum1 needs to be defined with a generic type E because any implementation of Comparable needs to be generic. Doing that, however will cause the compiler to throw a warning that line 2 above may be using unsafe operations. This is because Enum1<E> is equivalent to Enum1<E extends Object>; so we could potentially send in any subclass of Object into compareTo().

To fix that, we'll need to constrain the type of E to Enum1s or its subclasses. This can be done by changing the type parameter in line 1 to : Enum1<E extends Enum1>. The modified code thus becomes:

    public class Enum1<E extends Enum1> implements Comparable<E>{   //line1
        protected static Enum1 $VALUES[]={};

        private final String name;
        public final int ordinal;

        protected Enum1(String name, int ordinal){ 
            this.name = name;
            this.ordinal = ordinal;
        }

        public final int compareTo(E e){
            return this.ordinal-e.ordinal;  //line 2
        }
    }

Now Color can be written as:

    public class Color extends Enum1<Color>{
        public static final Color RED;
        public static final Color BLUE;
        public static final Color GREEN;

        static{
          RED = new Color("RED",0);
          BLUE = new Color("BLUE",1);
          GREEN = new Color("GREEN",2);
          $VALUES = (new Color[]{ RED, BLUE, GREEN });
        }

        protected Color(String name, int ordinal) 
            { super(name,ordinal); }
    }

...and test code can be written like so:

    public class TestEnum1{

        public static void main(String[] args){
            Color c1=Color.RED;
            Color c2=Color.RED;
            Color c3=Color.BLUE;
            Color c4=Color.GREEN;

            System.out.println((c1==c2));       // 0
            System.out.println(c1.compareTo(c3));   //-1
            System.out.println(c4.compareTo(c3));   // 1
        }
    }

We can even write another enum like so:

    public class Vehicle extends Enum1<Vehicle>{
        public static final Vehicle BIKE;
        public static final Vehicle CAR;
        public static final Vehicle BUS;

        static{
          BIKE = new Vehicle("BIKE",0);
          CAR = new Vehicle("CAR",1);
          BUS = new Vehicle("BUS",2);
          $VALUES = (new Vehicle[]{ BIKE, CAR, BUS });
        }

        protected Vehicle(String name, int ordinal) 
            { super(name,ordinal); }
    }

... and enhance the test code to check out things like the ordinals of two Enum1-derived enumerations not matching even though the numerical values are same:

    public class TestEnum1{
        public static void main(String[] args){
            Color c1=Color.RED;
            // ...

            Vehicle v1=Vehicle.BIKE;
            // fails with error: incomparable types
            System.out.println((c1==v1));
        }
    }

Pending Question

So I've still not figured out what the final <E> in:

    public class Enum1<E extends Enum1<E>>

... is for, considering the code above works without it.