An agent-based parser for a structural IDE

I had been thinking about the difference between text editing and structured editing and it dawned upon me that the latter will never win simply because it is not "natural" to edit structurally and that such editing is not forgiving of "mistakes". Sadly, text editing is good by default; and structured editing is not so by design.

We have a grammar, so we use it. But do grammars have to be policemen all the time? After all, we humans created grammar; and I have to think that we created it by convention to make sense of the otherwise chaotic barrage of communicative sounds we invented. So if natural language grammar is intended to be "guidelines and lighthouses to the islands of information in the oceans of noise", why shouldn't computer language grammar be the same?

The problem, of course, is that such a grammar would be non-deterministic. How would you specify a terminal symbol with any kind of certainty when any possible other symbol could intervene and has to be considered noise in a very context-sensitive way?

I wonder if there's an other way; and the rest of this post will record my thoughts to date on this other way.

Imagine each symbol of the language's alphabet as a cell waiting for user input. Imagine each non-terminal in the language's grammar also as a similar cell, only it waits for symbols to activate.The moment a symbol appears in the input (this could be the read of the next symbol from a file or the next key pressed by the user) the symbol grabs it and announces that its now "active". This triggers "interested" non-terminals to claim that symbol as part of themselves, although the ownership is shared - until all symbols of a particular non-terminal are satisfied and it claims all symbols to itself; and announces that it is active. This triggers the next level of non-terminals to wake up, and so forth.

This sounds pretty close to how Agent-based systems work, hence the term in the title. If I were Microsoft, I'd probably call it ActiveParsingTM.

The key advantage with such parsing would be that ambiguous input could be handled by the system. Also, input that doesn't match any production rule in the grammar would still be allowed into the system; which means that erroneous user input is tolerated, not berated.

In addition, such a system would be incremental from the ground up: although it is backed by a grammar, input is not constrained to "start at the top and end at the bottom of the grammar". It is also inherently event-driven (see design notes below) and therefore should be able to react to editing actions better. This ties in well with "natural" structural editing.

Finally, such a system would allow for hybrid languages that embed other languages within themselves, eg HTML+CSS+JS.

I cannot imagine that this hasn't been thought of before, as always. My Goog-fu not being upto snuff, however, I was not able to ferret up anything concrete. Of course, I did find a lot of references in the NLP literature to agent based systems and automata based parsers and such like, but they usually devolved into stats or deep math to prove the value which I couldn't comprehend.

Design Notes

• The system works on heartbeats called ticks. Everything happens once every one tick.
• There are cells. Cells wait for external input. All cells are guaranteed to get a new datum input into the system within the same tick. Cells activate (or not) during that same tick for a particular datum. Cell activate recognizes a single symbol or character.
• There are cells that wait on other cells for input. These cells  activate when all the cells they need are activated. Their activation recognizes a token or non-terminal.
• The input could be control input (such as the delete key) as well. This would result in previously activated cells becoming deactivated, changing the state of the system. The impact would be felt in the system over the next few ticks, but also means that it would be local to only those parts of the grammar that are indeed affected by the change, not universal as is typical in standard parsing systems.
• At all levels, activation triggers an event that can be handled outside the system. In particular, recognition of a non-terminal could be handled as highlighting it as a known keyword or identifier, etc. This is how syntax highlighting would happen
• At a sufficiently high level, activation of a non-terminal means something larger than syntax highlighting, eg, validation, interpretation/compilation or execution can happen.
• Finally, all cells can be supplied with a prerequisite activation signal which would HAVE to be "on" for them to become on. This would allow controlling the "viewport" of visible cells easily enough to constrain the calculation to that part of the text that is visible.

Update

Found it! An undated paper about Object Oriented Parallel Parsing for Context-free Grammars. Although I couldnt find a date on it, references to Symbolic Lisp machines dates it to at least the 90s if not the 80s. Google search terms "parallel parsing computer language" 

Idea: Email + Document Server in a box

I dont yet have a good name for this concept, but here's the idea anyway:

You know a lot of good ideas are better put in an email instead of a separate document because people will not open that attachment?

What if you could actually then easily extract out such content and make it available as a document? Further, what if the original recipients of your email continue to see it as an email, but it has a separate life as a document?

This is what I mean by the Email server being a document server as well. Each email could contain multiple such documents; or each email could itself be a document. The inline vs attachment line is blurred, but at the same time, individual bits within emails become addressable entities outside of the email thread.

Right now, we save such emails, re-format them as documents and stick them in a wiki or sharepoint. What if the Email server did all this for us?

JS++

Sometimes you dont think of the obviousness of an idea until someone hits you on the head with it.

Typescript was released today and it finally dawned on me (despite the very visible existence of Coffeescript in that exact same mold for ages) that creating a pre-processor for Javascript that cleaned up all the messy bits for you was a viable option.

After all, that's how C++ came to be, isnt it?

But if I were to do it, I'd do it a bit differently:

• I'd not have the TS-style classes, interfaces and the like; instead I'd just have a simple object creation syntax that's better than the default JS one or the JSON-style one
• Modules would map to CommonJS or AMD modules, but they would behave (from a deployment perspective) more like Fantom pods than the module-within-module craziness that TS seems to expound. It sucks in Java and I'm convinced the Fantom model is better simply because "One conceptual container == one deployment container" is such a reduction in cognitive overload.
• Embrace hoisting for "top-down style" writing of code, but introduce true scopes to retain sanity.
• Some implementation-neutral way of embedding the ultimate javascript into its host environment through the preprocessor. This is something tools like yeoman and bower miss, IMO.

Those are some of my main themes, I'd say; there's more that another reading of the Crockford bible should produce.

Design notes: 

1. Since this should be a simple "if you see this jspp code, spit out that js code" type thing, I'm thinking something as simple as StringTemplate might suffice, although it might be too early to speculate that far. If it did though, I'd build the whole preprocessor as a series of calls to a function called becomes(srcstr,deststr)
2. The implementation will have to support output of multiple files from the processing of a single input file.
3. If this turns out to be small enough, we could finally have a JS language that runs transparently on Rhino. CS currently faces the issue that its main code is larger than the size that Rhino can handle.

Postscript: I'm tentatively calling this idea JS++ simply because that's how C++ came to be; but I can see how this might not make good marketing sense. If so, here are some alternative names:

• Better Javascript : .bs :)
• Easier JS: .ejs

Idea: Use a map framework to depict code

Today's XKCD comic and its interpretation as a zoomable view using Leaflet had me thinking of the possibilities this presents:

Software cartography already demonstrated how code could be converted into a map. It even has the interesting property that it attempts to map the mental model of the code instead of its specific implementation - which IMO is way better than something like Code City simply because the city (or country) looks the same even if a few buildings disappear - if you know what I mean.

The only missing piece is scale - how do you scale this up to larger and larger codebases? Well, using a map engine is one way, IMO.

The problems of scale have already been solved there, as is that of display form factor: most of the map frameworks are already mobile-ready. The UI metaphors are familiar with most people too.

The only possible thing that detracts from my grandiose view of an n-dimensional version of CodeBubbles  to depict the true complexity of code is that map engines are decidedly 2 dimensional. But even that is a weak argument - layers provide sufficient degrees of freedom to annotate the display appropriately.

Why new bottles with the same old wine works... everytime!

I attended the second BangaloreJS meet today. The sessions were good and so was the food; and I met some really interesting people so in all a fun time was had.

Before the sessions started, however, Jon - the meetup organizer - showed a quick demo of Yeoman: a new tool released by Paul Irish and some of his friends from Twitter that eases building client-side JS projects.

I had a senior moment when I finally realized what it was.

This was Rake in client-side JS clothing, which was ant in Ruby clothing, which was make in Java clothing, which was ... I dont know what came before make because that was waaay before I was born, but I know there was something else that all the cool kids left for make.

Yes, I realize this does way more than a simple build tool; much like rake does way more than just build code. And it did it in a way that seemed magical when it came out first, too. You could do all that with ant tasks/plugins but it was not the same thing. But that's besides the point. It was a build tool in its core.

Why do we as developers keep.reinventing.these.common.tools?

Answer: For the same reason tool vendors have different versions of the same tool by language.

See, I happened to be browsing JetBrains' website the other day and saw that they claim that their non-Java IDEs share the "same features as other Jetbrains IDEs" or something to that effect. It struck me that they surely have a generic base version of the IDE that is skinned and customized for each "vertical" i.e., language-specific IDE.

Why then do they have one tool per language?

Because there are very few polyglot programmers in the world. Majority of us stick to one language, maybe a platform with a few related ones. That's IT. So we want something that "speaks to us, knows our problems", not something that can boil the PL ocean. And why would we need all those generic, all-language features anyway, right?

So the vendors sell us the shiny new bottles. And we drink it up.

And that's the way it should be. Why should client-side JS devs (who are in an unprecedented hype cycle after a long winter) be grudged a "robust and opinionated client-side stack, comprised of tools and frameworks that can help developers quickly build beautiful web applications"?

Note to self: When you finally finish creating Fluent, remember to brand it by language. FluentC, FluentRuby, FluentJava all sound very usable :)

The "Save Download to Category" Browser Add-on

I download things all the time, and all of them to go to the same place always - my downloads folder. Its nice to have a single place to look at all the things I downloaded, but most of the time, I'm moving the download to someplace else once its downloaded. This two step process is annoying.

So the idea is this:
Have a browser add-on that allows you to pick where to put the downloaded file(s) with one click. Preferably with category tags that map to actual folders.

Todo: Check if something already exists. It most probably does.

Tools are the bane of the Beginner

"Huh? Isn't that counter-intuitive?", you ask?
I realizing something particular as a result of some experiences and wrote that line down, but to me even it sounds counter-intuitive; so let me hasten to explain.

A beginner, for the purposes of this discussion, is somebody who's beginning something. This could be a novice starting to learn a skill; but it could also be an expert who's beginning a new project within his area of expertise.

A a beginner (thus defined) is hindered by the presence of tools and frameworks for two reasons: 

• They obscure the what and the how of the problem at hand by hiding it within ( usually for novice beginners).
• They prevent easy exploration of the why of the problem and its solution space through ceremony and preventing access(usually for experienced beginners).

Allow me to present the experiences that led to this realization. It all began one mundane work day in the not-too-distant past....

Story#1

My team had just got a bunch of freshers. They were recent engineering graduates (presumably with some exposure to programming) who have passed through the company's training program (which again presumably imparts further such exposure). We, however, found that they couldn't do some simple tasks like write code outside Eclipse. They didn't know how to deploy a web application except through the Eclipse plugin; had never debugged an application via logs and in fact didn't know  about webapps as an idea independent from "Tomcat". Their OO concepts were shaky at best, but they had implemented small-but-complete web applications using Tomcat, Struts and Hibernate and passed an EJB exam. When asked to build their study app the same from scratch using a command line, however, they were lost. When asked to build a different application (than the one they'd done) *using Eclipse*, they were similarly lost.

While a large portion of the blame should rightly lie in the teaching methodology (or lack of one), the tools and frameworks too, IMO, should bear some of it. "Deploying" for them meant clicking on the Tomcat icon in Eclipse, so they had no use of knowing what "web.xml" did nor did they know that it was no longer required. The same wizards and menu options that make the life of a practitioner easy actually obscure the underlying process (and why its required) to a beginner.

Story#2

The same group of freshers were slowly getting on track with (re)learning the basics of programming, when I thought it might be a good idea to instill in them at this "early age" the values of Test Driven Development. I immediately checked myself, however, because they'd have to learn JUnit and how to use it. On second thoughts, however, I realized that they didn't HAVE to use Junit or any such framework to do TDD. All they had to do was write a test before writing the actual code, have it fail, write the code and have it pass the test. The test could be code in the same function or in main() or as a series of calls to the program stored as a shell script or as JUnits. All of them are equally valid as "tests". We generally, however, recognize only the last of these as tests. The concept of TDD has been usurped by the concepts of the tools that implement it - to the extent that TDD doesn't seem have a life outside of those tools.

Tools, therefore, seem to be actively scuttling the consumption and adoption of Concepts, even when they were created explicitly for the reason of automating the repeated application of known concepts.

I personally have been struggling with this - the guilt of not doing TDD vs the allure of just seeing working code - especially when I'm beginning something and still feeling the problem and solution spaces out. In some cases, the solution space doesnt have readily available tools (BDD on the browser, anyone?) and in others there are tools but I'm still not ready to commit to them because I dont know what my solution is yet (should I build the parser first to figure out the syntax or the AST interpreter to see how it would run?). My liberation came when I declared that a test will be whatever I call a test for the situation at hand, not what some framework determines to be one. Since then the test-red-code-green-repeat cycle is a much more doable one.

Full Circle

Back to the initial "Huh?" moment from the beginning. Why then do we generally consider tools to be useful - especially for beginners? Tools are generally time-savers. They do one thing and they do it well; and that is their value. They do, however, have an "operating range" in which they're most useful. Below that, they're overkill and above, they're obstructive - as depicted in this highly accurate graph on problem size vs tool effectiveness:

So when we usually talk about tools being useful, we're talking about the useful operating range. Specifically for beginners, tools are solution accelerators at that range. The stories presented here represent the two ends of the spectrum, however, where tools are sub-optimal.

Note: I've glossed over frameworks in this discussion, but the concept is the same; or applicable even more so for frameworks. Frameworks by definition are a standard solution to a common problem, with room for customization so that application specifics can still be implemented. The framework is one because it has a known world view and exposes an interface that allows operations on that world view. The concept of operating range is well-ingrained, therefore; as are those of the limits on either side.So please read "tools/frameworks" wherever you see "tools" below.

So...

Armed with this framework for evaluating tools, we can start asking some interesting questions.

1. What is a good tool?
2. When are tools not required?
3. When are tools required?
4. How do we determine the operating range of a tool, then? 
5. What can we do to use tools more effectively?
6. What can tool builders do to make effective tools

Attempts at answers to these questions in part 2 of this article.

What if there IS no source code?

...yeah, kinda linkbait-y heading, I know.

What I mean is: What if there is no single source of the code?

Let me explain.

Typically we have source code. Its written by someone, stuck in a source control system somewhere, changed by others and so on.

The projectional editing school of thought modifies that picture somewhat by suggesting that we could have different views of the same code - a functional/domain-specific one, a folded one, a UML(ish) one, a running trace one and so forth. The relationship between the source and the multiple views, however, remains decidedly one-to-many.

What if instead of this master-slave relation, the different views themselves were the source? That is, the "whole picture" is distributed across the views - like a peer network?

Assuming the views are consistent with each other, modifying the source in one view should retain the true intent. But is that possible even? Views are, by definition, projections of the code; meaning some parts are included in the view and some aren't. So how would we maintain consistence across multiple views?

Two paths lead from here:
One: We cannot. This is why we need the "one true source" that's the parent of all views.
Two: Maybe we don't need consistency all the time. Maybe we can do with the "eventual consistency" that big data/nosql guys are raving about?

#2 seems like an interesting rabbit hole to explore :)

Naked Objects for Mobile

Why not, I ask?

The Naked Objects Pattern has been an idea that I've always found interesting and useful for quick-n-dirty apps.

The Apache Isis way of NO seems a little bit overwrought (as most of Apache projects are), but I've been following JMatter for quite a while now and have been impressed despite it being a Swing-only UI.

For the simple one-user app, the NO pattern seems sufficient: the thing gets built quickly, there's lowered expectation of customized UI from the user and probably even platform portability.

Snipped from the web: an idea related to GUTS

HN > Techcrunch story > Google Search for paper mentioned > Ch1 of paper, which had this nugget:

"I noticed that, often, the mere presence of a feature or capability incurs some cost, even when not being used. For example, as the number of services and options multiply, extra code is required to select from among them, and to check for possible interference between them. This code does no useful work processing the application's data, yet it adds overhead to each and every call."

I like this idea of the presence of code causing things to change around it - seems like something that can be quantified usefully by GUTS. 

I am Jack's total lack of unlayered design

A nebulous idea grows within me partly from GUTSE's insight that all programming is mere translation from one  language to another: All Von-Neumann machines are merely SSI (Sequence, Selection, Iteration) executors that essentially facilitate these translations. Why not enshrine these facts directly, then? The design of Jack, therefore, stands refined to:

• A simple SSI executor at the base with no idea of what data is beyond the generic map that it reads in as input.
• A data layer (built using the SSI + its environment) that actually does know about the bits of data stored in the map. This layer essentially provides the box/unbox, serialize/deserialize, parse/print dual functions as required
• The data transformation layer (built using the layers below) that enable and reason out the transformations declaratively so that they can be composed, collapsed etc.

Coin, Lambda, Jigsaw and EJB Home Automation: My take on the JavaOne 2012 Hyderabad

So nearly 13 years after my first java program, I finally attended a JavaOne. I remember sitting in Chennai years ago wishing I were in California when I first heard of them (or read about talks from them months later); so when I heard about a JavaOne in Hyderabad, I figured it was time. Of course, its not the same thing without Sun hosting it, but I was also interested to see how Oracle rolls one out. Few days of finagling with the powers-that-be later, I was set to go.

I've been to Hyderabad before, but not by air. So when I landed and found an airport that could be mistaken for any mid-size one in Europe or the US, I had to concede that Bangalore with its sneeze-n-you're-either-out-or-on-the-runway airport has been severely outclassed. Add to that the faithful reproduction of a US expressway (with exit numbers, no less) that led from the airport to the city and a die-hard bangalorean like me had to just weep and die a little bit inside. Damn you, you f-in politicians and whoever else who's responsible for our rinky-dink airport and the tragedy that is NH-7. You know you've had the revenue in spades to do a better and quicker job.

And then there was the convention center itself. Pretty much world class, IMO. Easily configurable main hall (attendees know what I'm talking about), a control console in each room, each room equipped with at least 2 screens and 2 LCDs facing the speaker - one to show the slides and one with a timer, ample wait staff and F&B counters, attached hotel - the works. Of course, the event management itself had some typical desi-ness to it that I'll mention below, but in all a very nice place. It had the feel of somebody with systemic thinking being in charge.

Anyhoo onto the sessions themselves. There were 5-7 parallel talks spread across Oracle-specific and Java themes so I'll talk about the sessions I attended:

• Day 1:
•  Keynote: Pretty OK state-of-mobile talk by Mr. Rego. Nice demos of Nano Ganesh (controlling irrigation pumps using Nokia phones) and of controlling cars via phone. Nice message about "looking for problems to solve and the money will follow". Also liked Nokia's message that they're not about creating new things themselves, but enabling and partnering with people who are creating those new things. I liked this message because I've always like Nokia phones as sturdy workhorses of communication and IMO with all the smartphone craziness this is a good middle path to take. Hope it works out for them.
• Aside: The Nano Ganesh project uses the phone's voice connection to control pumps. I spoke to Santosh Ostwal - the creator - and he confirmed there's no API to this - its direct control of the hardware. Sad. It would have been great to write this kind of phone app - the kind that doesnt need a separate data connection.
• Head and tail of Project Coin: Good "Directors cut" review of what its about by Dalibor Topic. Much better than just the changes.
• Data Parallelism with Fork/Join: Blew my mind when they got to Spliterable. Very well explained by David Holmes. He didn't like my fanboy idea of reviving the old java idea of sending code via RMI to another JVM so that we could federate fork/join over multiple JVMs, however. Probably with good reason :). 
• Oracle SOA suite: Caught the last of this because I was talking to David Holmes, so can't say much about it.
• Tech Keynote 1: Nice overviews of Coin, Jigsaw and Fork/Join. Particularly liked Dalibor's animated slides that converted boilerplate java into its functional equivalent.
• Improving MSQL performance with Hadoop: Lost patience. Speaker gave very naive definitions of MySQL and Hadoop and showed an agenda that didn't actually have the title of the talk listed.
• Project Lambda: Excellent build up of the material and exposition of the JSR as it stands.
• Java ME and LWUIT: Attended the end of this lab session. Seemed pretty standard UI stuff.
• OTN Night: Not easy to get a few thousand indian geeks to "get jiggy", but to their credit Oracle tried. First there was a stand-up comic who apparently was a developer at some time, so was billed as a comic in 4 languages: English, Hindi, Java and C++. Sadly his tech jokes stopped at calling himself a Null Pointer. Aside from asking a whole crowd of south indians if "there were any south indians" and making the cardinal sin of assuming all southies are the same when there was a whole wealth of jokes with just mallus alone, he was OK. That was followed by Vasundara Das trying really hard to get 5000 engineers to confirm that the "party was on the dance floor" to her chartbuster "Where's the party tonite?". When I left about 100 hands were up in the air somewhat in chorus with the slightly-off background troupe. The food was good, though :)
• Day 2
• Oracle Develop Keynote: Some clueless event management guy goaded me into this with a "Developer keynote starting, please go, please go", so I went in. 5 seconds in I was thinking "How the hell is this a developer keynote, I'm walking out right now". I remembered, however, that I should probably listen to some Oracle speak for use "at the office" and settled down. It turned out to be pretty Ok after all, with Oracle's Cloud strategy taking center stage. David Pease presented a nice (possibly standard, but nice anyway) cloud adoption strategy and spoke about their public and private cloud offerings. Afterwards, him and another Oracle Manager did a pretty good job of answering my two questions:  How the cloud affects their main revenue stream - big iron database servers (answer: complimentary, not cannibalistic), and what their official response was to NoSQL(ans: berkleyDB. Not convincing, IMO) and Big Data (Ans: Will be treated as extension of existing solutions, not a separate one).
• Interfacing with the interface: Impressive demo of JavaFX, Wiimote, Kinect and Hotspot. It was one of those "It CAN be done with Java too" kinda things. Cool, but dont see the point. Wouldnt you just use the native technologies for Wiimote and Kinect? Still a very balanced and nice talk. Take away: Heard about the JMonkey Engine. Also heard that its pretty buggy :)
• Agility and Scalability @ eBay: Good, solid talk; but nothing new. Walked away into the "interfacing.." talk after a while, returned towards the end and found it taking a predictable course. No fault of the speaker, though. It was me, not him; entirely like seeing George Carlin for the 5000th time and not laughing. The rest of the crowd seemed to like it just fine.
• Jigsaw: Another solid talk by David Holmes. I started reading about the project after the talk and immediately wanted those slides that presented the problem, its implications and solution(s) so well. Reaction to the project itself: Cant wait for it, cant wait to use it. Although I dont get the need for the "default" keyword for default methods. If you're writing a method body within an interface definition, shouldnt that be sufficient indication that its a default method? Holmes' take was to join the mailing lists and either understand or speak up. 
• Nashorn - Javascript on JDK: Short, well presented talk on the new JS interpreter on the JVM. Sunderrajan was awesome at spinning out quick examples to questions right on spot. He was evasive about performance due to safe harbor rules, but was forthcoming enough to say its way better than Rhino. Killer announcement: Node is coming to the JVM, and they already have it working internally. Cant wait.
• Aside: I mentioned that I was excited about Node-on-JVM to Steve Chin and Kevin Nilson and their response was: "why would anyone want to do that?". My answer: admins love the JVM. Put anything inside it, and they'll be OK with it. The JVM is the ultimate production grade sneakware platform :)
• OTN Room: Monitoring Cloud Apps: Spent 10 mins here. People in the audience kept interrupting the speaker with more knowledgable information than he had. Good to get good information, not so good to have the talk's flow being interrupted thus. IMO Ranganath should have just set the rules of engagement better than being polite like he was, or the session should have been a BOF one.
• Java beyond the IDE: Nice if boring-in-the-middle talk by Jay Suri on things less talked about in the Java scene, namely, issues in production. After a long list of issues the Java Platform Group has heard from customers (and their solutions), Suri introduced two tools that were coming to the JDK via JRockit - Mission Control and Flight Recorder. This latter tool is what everyone wants in production - instrumentation for a running JVM - and you can now have it! Expressly to be run in production! From the source! Of course, you'll need an Oracle license for production use, but depending on the price, I can easily see this being put on at least "Test in Production" servers for those hard-to-debug-in-preprod problems. Awesome.
• Having fun with JavEE and Home Automation: Who would have thought of using an EJB server to do home automation? A crazy, funny, energetic brazilian and his wife, that's who.  Vinicius and Yara Seneger were genuinely happy to be in India and gave what was possibly the most lively presentation that I saw. They demoed JHome - a home automation software platform that uses Glassfish and Arduino to create home automation such as lamps that turn on via http, LED lamps that turn to any RGB colour, controlling appliances  via sound. The final demo was a heartbeat monitor that Vinicius had hooked up to some EJB code (Yes! EJB code!) so as to light up a lamp when the heart rate goes over 110 BPS. And of course, he had to jog to get it working. Awesome and endearing. Who cares if they used a whole herd of elephants to swat a fly? The energy of the guy was great. Couldn't have asked for a better last session :).
• Speaking of: why wasn't there a one last hurrah Oracle? I'd have liked one last closing session on the center stage. And: some of us like to see the people behind the show, you know? Get 'em up the stage and let's take a gander at them. 

Tech Reactions/ Opinions:

• UI Frameworks: Java has AWT(hopefully dead now), Swing, JFX and LWUIT (which I didn't know existed till this conference). Exactly how many of them does one language need? I spoke a bit about this to Steve, but he seemed unconvinced that it might be a good idea to consolidate. Maybe the way I put it was off, but is it too much to ask for one language to have one (preferably declarative) way to define a UI across all its deployment profiles?
• Mobile: 
• The elephant in the room was walked all the way around pretty carefully by Oracle. Nokia had one A-bomb, while Oracle billed the "mobile platform that must not be named" quaintly as "other linux tablets". The stoopid questioners mentioned below, however asked pointed questions on integrating with it, however. Was kinda funny to watch the Oraclers tippy toe around these ones.
• The heartening news, however, was that the release schedules for J2ME was merging with the J2SE one and that Java 8 will have one of the ME profiles built in. Should bring solace to J2ME users, i guess. Do you know any? I dont.

General Opinions:

• Speakers almost always took up all the time, leaving no "public Q&A" time. They were of course, super available in the hallway after the talk, but that meant fighting the hordes to get your one question answered.
• There were some really stoopid questioners, which the speakers were gracious enough to deign to answer. 
• Sample: To the nashorn team member: "Will it support eval?". Ans: Of course, its a language requirement. "Will it support closures, lambdas and anonymous functions?" Ans: Of course, its a language requirement.". "Ok, I understand it will support lambdas because of Project Lambda, but what about closures and anonymous functions?" Ans: "Look, we have to, cos its part of the language. And this has nothing to do with Lambdas in Java". This went on for a few more rounds.
• It was even worse in the Oracle Dev sessions where people generally were summarizing the talks in the Q&A sessions.
• Event management was a bit weird in places:
• No wifi. Had to be said. In fact, their "suggestion" was to not get any laptops either. I get that the majority attendees are local and probably have their own data connections, but what about us out-of-towners who might want to tweet you some kudos? Surely the world's largest database company should be able to afford that?
• Long queues to get into a session because Oracle wanted head count. Couldn't you have set the Reader at the door and made it self-service? Many people kept leaving sessions and walking into others anyway thereby skewing your count; and this made for an annoying delay at each session.
• Food service had some desi quirks: 
• I had to literally fight for a second plate at lunch (I discarded my first one after my first course) because they dont give out a plate unless you have a token. Explaining that I'd given mine already and all I wanted was a fresh plate didn't go too well. The second day I learnt my lesson.
• Tea was served at these stations with really nice looking tea cups-n-saucers stacked up. But they didn't serve us tea in them. Instead we got it in paper cups. Yeah, no clue why, or even why they had them on display if they didn't intend to give them out. Once you got your tea and took a sip as you walked away was when you realized there's no sugar. Walk back, and you're pointed to these tables with bowls containing sugar sachets, but no stirrers. Those precious bits of cutlery obviously cannot be left unguarded, so they're in the safe custody of the F&B guy - who could have told you that the first time, but of course he didn't. So walk back, get stirrer, walk back, get sugar. Then drink. I saw this happen over and over again in those two days. Its funny when you know what's gonna happen and you watch some poor sap go over this drill.
• The Swag embargo: At the end of the conference I saw a line snaking towards the registration booth and some people emerging with a t-shirt and mug. The mug I didn't care about, but I'm a t-shirt whore. So I stood in line, not really taking note of the people in line frantically filling out the feedback form. Apparently, you HAVE to fill the feedback form, else you dont get the swag. What if I took the things and came back with the form filled out, I asked? No sir, its the rules, they said. I'll wait right where you can see me, I countered. No sir, those are the rules, they said; helpless against the all-seeing all-dancing mother directive. So I filled it out as Doofenschmirtz, Evil Genius. They gave me the stuff without once looking at the paper. 

Thus went my first JavaOne. Oracle, if you're reading: Now you have my real feedback. Looking forward to more of the good stuff and less of the silly stuff.

Update: Not finding the slides from the talks in some official place, I scoured the internets for them so I could share with my team. Here're what I've found to date:

• Tech Keynote, alternate
• Another version, but with Fork/Join and Spliterable stuff included.
• Fork/Join Alone
• Project Lambda
• JDK 7 and Beyond
• Nashorn: The sort-of-official link from Dalibor's blog post doesnt work. I couldnt find any other place that had the "Future of Javascript on JDK" slides. Comment below if you find it.
• Project Coin
• Project Jigsaw
• The JavaOne India site seems to be geared for uploading of slides at some future date, going by what I found on the site from the one that happened last year in South America. Look there a while later to see if they've uploaded any. Right now (Mid May 2012), the list of talks itself is incomplete.

Of Phonewords and Carpet Jingles: Its funny how the brain works

I was talking to a colleague about how my android phone's dialer doesn't have some basic features - like the ability to paste a number directly into it (LG Optimus 500, 2.2 - ymmv).

...which led to..

Me remembering how Blackberries had the ability to dial phonewords directly by pressing Alt.

...which led me to two things:

1. Googling if android does the same. It doesn't looks like it does. Funnily, all searches led to blackberry forums where the Alt trick is described. Isn't progress wonderful?
2. Googling 800-588-2300 - the one number (and jingle) I cannot forget.

... which led me to..

The metafilter discussion about Empire and the empire man.

... which led me to...

1. Wonder how I remember seeing the commercial in NJ when the post doesn't list it as one of states in which Empire operates? Maybe I remember it from my time in Florida? Man, that would make it memorable for a decade! That's some recall power for a product that I've not ever bought.
2. Now run around the whole day in Bangalore singing both the Empire AND Luna jingles!

PS: Best quotes from the metafilter page:

The man has gone, but the goddamn jingle will never die. I hope Hauldren's heaven is plush and stain resistant, with no monthly payments til July

and:

"we're been getting some ads in the same form for Luna carpet, but I haven't the foggiest idea what their phone number is."
877-241-LUNA. I'm ashamed of myself for knowing that.
What? No. 773-202-(boop beep boop beep) LUNAAAAA. 

Fluent in the real world: Light Table

Today's HN story on Light Table only tells me that I won't have to build Fluent myself - someone else will, soon enough!

In the discussion that ensued, there was however, this interesting comment:

Yes, but our code was entirely in these utterly unusable changeset files that couldn't work nicely with the version control that everyone else in the entire world was using; his version still uses files under the hood. There's a team that's trying to back Monticello with Git, I believe by saving each method into its own file in part of a Git source tree; that looks promising as a compromise.

That's a good lesson to learn from, methinks. Just as "text as code and not just its serialization format" has stood the test of time, so has version control using textual diffs. Anything else will face the migration gap.

I can't be bothered to find the link, but a while ago there was a post by somebody big in the blogosphere (Spolsky or Cringely) that spoke about a bill pay start-up that solved the migration problem by allowing users to forward their paper bills directly to them to have the bills digitized and paid. The idea was that once the bills were digitized they could be paid electronically thence, but it was the first time setup that was the hump that users didn't want to go over. Take that pain away, and you got yourself converts by the droves.

Tripit does something similar with Trip Management by parsing booking confirmation emails from travel websites.

I'm not so sure such an approach will apply to developers and their tools for a couple of reasons:

• Text is actually not a format that you want to get away from (unless you're in the structured camp)
• Providing the gateway solution involves fixing all the tools to work with the non-text format you come up with. And you must fix ALL of them - the IDE, the debugger, the built tool, you name it.
• Alternatively, you could change the environment to something where text input is actually less efficient. Like the Tablet. Then you have a chance.

Its a long, tough road ahead :)

Early thoughts on Jack

I found this writeup from 2009 in one of my umpteen storage media and thought it was evocative enough to publish. My later ramblings on the topic of Jack have been attempts to concretize the ideas rather than gaze starry-eyed at the wonder that is that ephemeral topic of a versionable language. This one is more of the latter kind than the others. There are 2 parts to the writeup: a exposition on top and notes at the bottom. The notes are actually the outline of the exposition, which is essentially unfinished. Except for some cosmetic formatting changes, I've reproduced the piece as is:

Think of an aspect. An aspect usually takes a chunk of code, and in the most general case (called the around advice), wraps around that chunk an envelope. This envelope acts as a guardian to any entry into the chunk and is not only capable of altering expected behavior of the chunk by altering its inputs, but is also capable of deciding if the chunk should execute at all.

Think of a monad. It effectively does the same, except it does it to separate non-functional bits from functional ones.

Think of an ESB. It too, does the same; except that it does it at a much higher level of granuality - at that of a service. Taking the generally understood definition that services are suitably large components that house some specific business functionality, an ESB orchestrates the sequence of operations between these components to achieve the affect of an application.

Now think of how we modify code. We do the same thing - alter the expected behavior, decide if the code should execute at all, (re)orchestrate the sequence of operations to achieve the effect we're expecting.

We are the aspect, the monad, the ESB. 

However there are major differences between us and these constructs that act to the disadvantage of humans while maintaining code:

• each of these constructs have the feature that it describes the change it effects on the component being changed. The humans changes are available only as diffs on an external tool - the version control system

• the human's change is at a textual, character level; not a statement/method/package/module/unit of execution level.So the change is not percieved as a change of language elements to effect it, but numerous characters being shuffled around

While this might seem like a tirade against text-based programming and possibly making a case for structural editors, there's more than that. The key problem is that the unit of execution is not identifiable. Statements in modern languages are anonymous, except by the line number of the source file. If they were identifiable, we could express something like "i had to remove the 5th if statement to after the assignment of var foo" instead of "cut line 150-234, paste at line 450". The former is what we usually do when we talk about it, but there's no direct way of enshrining that in a machine readable way.

Whats the use of such a feature you ask? Well, imagine a language that allowed us to identify the statements, and then express addition, deletion and modification of statements within itself. Something like:

insertStmt module3.class4.method1.if#5, AFTER, module3.class4.method1.letvarfoo

 The fact that the statements of code are addressable allows us to refer to them in a logical manner and the fact that the operations carried out to cause the code to change are operators allows us to maintain the change itself as code.Similar to insertStmt there'd be addStmt, deleteStmt and modifyStmt operators; and obviously we can extend the concept of a function to these operators too, so that the complete conversion from one version to another is expressed as a single operation - a changeset in code, if you will.Producing the next version of your app is no longer a snapshot activity - it can become incremental. Further, multiple changesets can be "compiled" into a fix pack of changes to produce any version at will. And all changes are expressed as logical changes, not textual deltas.

More importanly, think of what the language (or its units of execution - the statements) would have to support for this to work. They would have to become self-contained modules. Self-contained micro services, if you will, which can then be "orchestrated" my moving them to the location in the code which will cause whichever version we desire to be effected via these transformation functions. Code therefore becomes easier to change.

Now lets take it to the next level, and define these operators at all levels of abstraction/modularization that the language supports. So we'd have addMethod,addMethodArg, addModule, etc.

Notes:

• Identity
• modularity
• esb-style orchestration
• not just run time, but also statically, we can express the change occuring. which makes is amenable to machine learning.
• automatic modularization/aggregation is the key to useful versioning.
• forget versioning. what i'm really trying to do is to discover the steps in the computation being carried out that can be abstracted out such that an esb can act on it.
• deriving the higher order sequence from base description
• there are only 3 basic constructs in programming - the assignment, the goto (including the implied goto by the instruction pointer), and the conditional ie if. if is usually followed by the goto, and all instructions of the JZ, JNZ variety are combinations of the if/goto. so the real inflection point for the sequence of operations is the if. we can consider any block of code before an if as a single block with appropriate inputs, outputs and context, and similarly any block of code after an if. each if clause represents a micro service. if therefore is the micro esb. 
• so partitioning of code cana be done based on ifs. now if we take all the ifs at the same peer level - within a method, class or package/module, (or even app) and find the same conditions being checked, those paths can be collapsed, or refactored - this is similar to the ideas in subtext. find the right partitioning of code so that it can be expressed easily. 

Named ranges - a potentially useful concept for Jack (or any normal language)?

Consider the following code that calculates the maximum product of 3 integers possible from a given array (Java used for exposition; could be any language):

The code in the loop could probably be extracted into 2 subroutines but that would involve major interaction between the variables in this one and those subroutines. What if we were able to define the code between the recalc...end recalc comments as a named range that did not introduce a scope? Something like so:

IDEs could fold easily on this and this would only help comprehension.

Finally: Why a language construct and not a comment? Because it explicitly aids comprehsnsion. 

Nice idea from a HN thread

http://news.ycombinator.com/item?id=3650360

...storing here aa bookmark.

Two ideas to improve impress.js

1. Add a "debug mode" that shows the center and bounds of the currently selected step. This is useful in aligning 3d elements
2. Add incremental positioning attributes. That is, step n's position is based on step n-1's. This is really useful considering the normal flow is to move from step n-1 to n. The new attributes would be "data-dx", "data-dy", "data-dz", "data-rotate-dx" and so forth.

Implementation note:

impress.js's main logic where the position of each element is determined will have to be enhanced to use the previous element's position when these incremental attributes are used. Once they're calculated, however, the rest of the logic should work as expected.

Programmers have pretty low expectations from their languages

Hacker News had a post titled "Modern Language Wishlist". I bookmarked it, perused the comments that tried to guess the language that the post was alluding to, and finally clicked over to the actual article.

While the list is a pretty good and pragmatic one, I can't help but feel that we programmers are a content lot. All the things on the wishlist were incremental improvements at best.

We build these big skyscrapers using tools that were used to build huts, methinks. Shouldn't the prime questions be:

What is the biggest "thing" that a programming language models?
And how close is this to an actual running application?

... I ask? 

Rules for life

This is pretty similar to Slots, but has more direct application, methinks.

The app will have two main actions: Jot and Find.

Jot drops you into a Slots interface. You put down a sentence and define slots in it.
That's the first half. The other half is another Slots sentence that asks the question for which the first sentence is the answer.

Eg: Sentence1: I kept [the tools] in [the bottom drawer].
      Sentence2: Where are [the tools]?

Find drops you into a Slots interface populated with Sentence2-type sentences.

Simple. But I think it can be real powerful once we add true Slots functionality and automate actions based on query results.

Yes, I know there's www.ifttt.com, but this seems much more generic and less program-mey.

Sudocode

Create a language that abstracts out the differences between multiple OSs by wrapping them around English-y language.

Example: "Run the executable ex1" will internally do the right thing regardless of OS; on Windows it will set the path etc and execute it with backslashes and so forth; on unixes it will locate it, do the equivalent with forward slashes and execute it.

Call it sudocode. It'll be the Cobol of the 21st century.