{"id":171,"date":"2016-07-03T18:11:38","date_gmt":"2016-07-03T18:11:38","guid":{"rendered":"http:\/\/procyonic.org\/blog\/?p=171"},"modified":"2016-10-03T12:29:44","modified_gmt":"2016-10-03T12:29:44","slug":"a-critique-of-the-programming-language-j","status":"publish","type":"post","link":"https:\/\/procyonic.org\/blog\/a-critique-of-the-programming-language-j\/","title":{"rendered":"A Critique of The Programming Language J"},"content":{"rendered":"

I’ve spent around a year now fiddling with and eventually doing real
\ndata analytic work in the The Programming Language J<\/a>. J<\/em> is one of
\nthose languages which produces a special enthusiasm from its users and
\nin this way it is similar to other unusual programming languages like
\nForth or Lisp. My peculiar interest in the language was due to no
\nlonger having access to a Matlab license, wanting an array oriented
\nlanguage to do analysis in, and an attraction to brevity and the point
\nfree programming style, two aspects of programming which J emphasizes.<\/p>\n

\"Sorry,<\/a>
Sorry, Ken.<\/figcaption><\/figure>\n

I’ve been moderately happy with it, but after about a year of light
\nwork in the language and then a month of work-in-earnest (writing
\ninterfaces to gnuplot and hive and doing Bayesian inference and
\nspectral clustering) I now feel I am in a good position to offer a
\nfriendly critique of the language.<\/p>\n

First, The Good<\/h2>\n

J is terse to nearly the point of obscurity. While terseness is not a
\nparticularly valuable property in a general purpose programming
\nlanguage (that is, one meant for Software Engineering), there is a
\ncase to be made for it in a data analytical language. Much of my work
\ninvolves interactive exploration of the structure of data and for that sort
\nof workflow, being able to quickly try a few different ways of
\nchopping, slicing or reducing some big pile of data is pretty
\nhandy. That you can also just copy and paste these snippets into some
\nanalysis pipeline in a file somewhere is also nice. In other words,
\nterseness allows an agile sort of development style.<\/p>\n

Much of this terseness is enabled by built in support for tacit
\nprogramming. What this means is that certain expressions in J are
\ninterpreted at function level<\/em>. That is, they denote, given a set of
\nverbs in a particular arrangement, a new verb, without ever explicitly
\nmentioning values.<\/p>\n

For example, we might want a function which adds up all the maximum
\nvalues selected from the rows of an array. In J:<\/p>\n

+\/@:(>.\/\"1)\r\n<\/code><\/pre>\n
J takes considerable experience to read, particularly in Tacit
\nstyle. The above denotes, from RIGHT to LEFT: for each row (\"1<\/code>)
\nreduce (\/<\/code>) that row using the maximum operation >.<\/code> and then (@:<\/code>)
\nreduce (\/<\/code>) the result using addition (+<\/code>). In english, this means:
\nfind the max of each row and sum the results.<\/p>\n
Note that the meaning<\/em> of this expression is itself a verb, that is
\nsomething which operates on data. We may capture that meaning:<\/p>\n
sumMax =: +\/@:(>.\/\"1)\r\n<\/code><\/pre>\n
Or use it directly:<\/p>\n
+\/@:(>.\/\"1) ? (10 10 $ 10)\r\n<\/code><\/pre>\n
Tacit programming is enabled by a few syntactic<\/em> rules (the so-called
\nhooks and forks) and by a bunch of function level operators called
\nadverbs and conjuctions. (For instance, @:<\/code> is a conjunction rougly
\ndenoting function composition while the expression +\/ % #<\/code> is a fork,
\ndenoting the average operation. The forkness<\/code> is that it is three
\nexpressions denoting verbs separated by spaces.<\/p>\n
The details obscure the value: its nice to program at function level
\nand it is nice to have a terse denotation of common operations.<\/p>\n
J has one other really nice trick up its sleeve called verb
\nrank<\/em>. Rank itself is not an unusual idea in data analytic languages:
\nit just refers to the length of the shape of the matrix; that is, its
\ndimensionality.<\/p>\n
We might want to say a bit about J’s basic evaluation strategy before
\nexplaining rank, since it makes the origin of the idea more clear. All
\nverbs in J take one or two arguments on the left and the right. Single
\nargument verbs are called monads, two argument verbs are called dyads.
\nVerbs can be either monadic or dyadic in which case we call the
\ninvocation itself monadic or dyadic. Most of J’s built-in operators
\nare both monadic and dyadic, and often the two meanings are unrelated.<\/p>\n
NB. monadic and dyadic invocations of <
\n4 < 3 NB. evaluates to 0
\n<3 NB. evalutes to 3, but in a box.<\/p>\n
Give that the arguments (usually called x<\/code> and y<\/code> respectively) are
\noften matrices it is natural to think of a verb as some sort of matrix
\noperator, in which case it has, like any matrix operation, an expected
\ndimensionality on its two sides. This is sort of what verb rank is
\nlike in J: the verb itself carries along some information about how
\nits logic operates on its operands. For instance, the built-in verb
\n-:<\/code> (called match<\/em>) compares two things structurally. Naturally, it
\napplies to its operands as a whole. But we might want to compare two
\nlists<\/em> of objects via match, resulting in a list of results. We can
\ndo that by modifying the rank of -:<\/code><\/p>\n
x -:”(1 1) y<\/p>\n
The expression -:”(1 1) denotes a version of match which applies to
\nthe elements of x and y, each treated as a list. Rank in J is roughly
\nanalogous the the use of repmat, permute and reshape in Matlab: we can
\nuse rank annotations to quickly describe how verbs operate on their
\noperands in hopes of pushing looping down into the C engine, where
\nit can be executed quickly.<\/p>\n
To recap: array orientation, terseness, tacit programming and rank are
\nthe really nice parts of the language.<\/p>\n
The Bad and the Ugly<\/h2>\n
As a programming environment J can be productive and efficient, but it
\nis not without flaws. Most of these have to do with irregularities in
\nthe syntax and semantics which make the language confusing without
\noffering additional power. These unusual design choices are
\nparticularly apparent when J is compared to more modern programming
\nlanguages.<\/p>\n
Fixed Verb Arities<\/h3>\n
As indicated above, J verbs, the nearest cousin to functions or
\nprocedures from other programming languages, have arity 1 or
\narity 2. A single symbol may denote expressions of both arity, in
\nwhich case context determines which function body is executed.<\/p>\n
There are two issues here, at least. The first is that we often want
\nfunctions of more than<\/em> two arguments. In J the approach is to pass
\nboxed arrays to the verb. There is some syntactic sugar to support
\nthis strategy:<\/p>\n
multiArgVerb =: monad define
\n‘arg1 arg2 arg3’ =. y
\nNB. do stuff
\n)<\/p>\n
If a string appears as the left operand of the =.<\/code> operator, then
\nsimple destructuring occurs. Boxed items are unboxed by this
\noperation, so we typically see invocations like:<\/p>\n
multiArgVerb('a string';10;'another string')\r\n<\/code><\/pre>\n
But note that the expression on the right (starting with the open
\nparentheses) just denotes a boxed array.<\/p>\n
This solution is fine, but it does short-circuit J’s notion of verb
\nrank: we may specify the the rank with which the function operates on
\nits left or right operand as a whole, but not on the individual
\n“arguments” of a boxed array. But nothing about the concept of rank
\ndemands that it be restricted to one or two argument functions: rank
\nentirely relates to how arguments are extracted from array valued
\nprimitive arguments and dealt to the verb body. This idea can be
\ngeneralized to functions of arbitrary argument count.<\/p>\n
Apart from this, there is the minor gripe that denoting such single
\nuse boxed arrays with ;<\/code> feels clumsy. Call that the Lisper’s bias:
\nthe best separator is the space character.1<\/sup><\/p>\n
A second, related problem is that you can’t have a
\nzero<\/em> argument function either. This isn’t the only language where
\nthis happens (Standard ML and OCaml also have this tradition, though I
\nthink it is weird there too). The problem in J is that it would feel
\nnatural<\/em> to have such functions and to be able to mention them.<\/p>\n
Consider the following definitions:<\/p>\n
o1 =: 1&-\r\no2 =: -&1\r\n\r\n(o1 (0 1 2 3 4)); (o2 (0 1 2 3 4))\r\n\u250c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\r\n\u25021 0 _1 _2 _3\u2502_1 0 1 2 3\u2502\r\n\u2514\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\r\n<\/code><\/pre>\n
So far so good. Apparently using the &<\/code> conjunction (called “bond”)
\nwe can partially apply a two-argument verb on either the left or the
\nright. It is natural to ask what would happen if we bond<\/em>ed twice.<\/p>\n
(o1&1)
\no1&1<\/p><\/blockquote>\n
Ok, so it produces a verb.<\/p>\n
 3 3 $ ''\r\n  ;'o1'\r\n  ;'o2'\r\n  ;'right'\r\n  ;((o1&1 (0 1 2 3 4))\r\n  ; (o2&1 (0 1 2 3 4))\r\n  ;'left'\r\n  ; (1&o1 (0 1 2 3 4))\r\n  ; (1&o2 (0 1 2 3 4)))\r\n\r\n\u250c\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u252c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2510\r\n\u2502     \u2502o1          \u2502o2          \u2502\r\n\u251c\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\r\n\u2502right\u25021 0 1 0 1   \u25021 0 _1 _2 _3\u2502\r\n\u251c\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u253c\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2524\r\n\u2502left \u25021 0 _1 _2 _3\u2502_1 0 1 2 3  \u2502\r\n\u2514\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2534\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2500\u2518\r\n<\/code><\/pre>\n
I would describe these results as goofy, if not entirely impossible to
\nunderstand (though I challenge the reader to do so). However, none of
\nthem really seem right, in my opinion.<\/p>\n
I would argue that one of two possibilities would make some sense.<\/p>\n
    \n
  1. (1&-)&1 -> 0 (eg, 1-1)<\/li>\n
  2. (1&-)&1 -> 0″_ (that is, the constant function returning 0)<\/li>\n<\/ol>\n
    That many of these combinations evaluate to o1<\/code> or o2<\/code> is doubly
    \nconfusing because it ignores a value AND because we can denote
    \nconstant functions (via the rank conjunction), as in the expression
    \n0\"_<\/code>.<\/p>\n
    Generalizations<\/h2>\n
    What this is all about is that J<\/em> doesn’t handle the idea of a
    \nfunction<\/em> very well. Instead of having a single, unified abstraction
    \nrepresenting operations on things, it has a variety of different ideas
    \nthat are function-like (verbs, conjuctions, adverbs, hooks, forks,
    \ngerunds) which in a way puts it ahead of a lot of old-timey languages
    \nlike Java 7 without first order functions, but ultimately this
    \nhandful of disparate techniques fails to acheive the conceptual unity
    \nof first order functions with lexical scope.<\/p>\n
    Furthermore, I suggest that nothing whatsoever would be lost (except
    \nJ<\/em>‘s interesting historical development) by collapsing these ideas
    \ninto the more typical idea of closure capturing functions.<\/p>\n
    Other Warts<\/h2>\n
    Weird Block Syntax<\/h3>\n
    Getting top-level2<\/sup> semantics right is hard in any
    \nlanguage. Scheme is famously<\/a> ambiguous on the subject, but at
    \nleast for most practical purposes it is comprehensible. Top-level has
    \nthe same syntax and semantics as any other body of code in scheme
    \n(with some restrictions about where define<\/code> can be evaluated) but in
    \nJ<\/em> neither is the same.<\/p>\n
    We may write block strings<\/em> in J like so:<\/p>\n
    blockString =: 0 : 0 \r\nEverything in here is a block string.       \r\n)\r\n<\/code><\/pre>\n
    When the evaluator reads 0:0<\/code> it switches to sucking up characters
    \ninto a string until it encounters a line with a )<\/code> as its first
    \ncharacter. The verb 0:3<\/code> does the same except the resulting string is
    \nturned into a verb.<\/p>\n
    plus =: 3 : 0\r\n    x+y\r\n)\r\n<\/code><\/pre>\n
    However, we can’t nest this syntax, so we can’t define non-tacit
    \nfunctions inside non-tacit functions. That is, this is illegal:<\/p>\n
    plus =: 3 : 0\r\n  plusHelper =. 3 : 0\r\n    x+y\r\n  )\r\n  x plusHelper y\r\n)\r\n<\/code><\/pre>\n
    This forces to the programmer to do a lot of lambda lifting<\/em>
    \nmanually, which also forces them to bump into the restrictions on
    \nfunction arity and their poor interaction with rank behavior, for if
    \nwe wish to capture parts of the private environment, we are forced to
    \npass those parts of the environment in as an argument, forcing us to
    \ngive up rank behavior or forcing us to jump up a level to verb
    \nmodifiers.<\/p>\n
    Scope<\/h3>\n
    Of course, you can define local functions if you do it tacitly:<\/p>\n
    plus =: 3 : 0\r\n    plusHelper =. +\r\n    x plusHelper y   \r\n)\r\n<\/code><\/pre>\n
    But, even if you are defining a conjunction or an adverb, whence you
    \nare able to “return” a verb, you can’t capture any local functions –
    \nthey disappear as soon as execution leaves the conjunction or adverb
    \nscope.<\/p>\n
    That is because J<\/em> is dynamically scoped, so any capture has to be
    \nhandled manually, using things like adverbs, conjunctions, or the good
    \nold fashioned fix f.<\/code>, which inserts values from the current scope
    \ndirectly into the representation of a function. Essentially all modern
    \nlanguages use lexical scope, which is basically a rule which says: the
    \nvalue of a variable is exactly what it looks like<\/em> from reading the
    \nprogram. Dynamic scope says: the valuable of the variable is whatever
    \nits most recent binding is.<\/p>\n
    Recapitulation!<\/h1>\n
    The straight dope, so to speak, is that J is great for a lot of
    \nreasons (terseness, rank) but also a lot of irregular language
    \nfeatures (adverbs, conjunctions, hooks, forks, etc) which could be
    \nfolded all down into regular old functions without harming the
    \nbenefits of the language, and simplifying it enormously.<\/p>\n
    If you don’t believe that regular old first order functions with
    \nlexical scope can get us where we need to go, check out my
    \ntacit-programming libraries in     R<\/em><\/a> and Javascript<\/em><\/a>. I
    \neven wrote a complete, if ridiculously slow implementation of J<\/em>‘s
    \nrank feature, literate-style,     here<\/a>.<\/p>\n
    \n
    Footnotes<\/h2>\n
    1<\/sup> It bears noting that ;<\/code> in an expression like (a;b;c)<\/code>
    \nis not a syntactic element, but a semantic one. That is, it is the
    \nverb called “link” which has the effect of linking its arguments into
    \na boxed list. It is evaluated like this:<\/p>\n
    (a;(b;c))\r\n<\/code><\/pre>\n
    (a;b;c)<\/code> is nice looking but a little strange: In an expression
    \n(x;y)<\/code> the effect depends on y is boxed already or not: x is always boxed regardless, but y<\/code> is boxed only if it wasn’t boxed before.<\/p>\n
    2<\/sup> Top level? Top-level is the context where everything
    \n“happens,” if anything happens at all. Tricky things about top-level
    \nare like: can functions refer to functions which are not yet defined,
    \nif you read a program from top to bottom? What about values? Can you
    \nredefine functions, and if so, how do the semantics work? Do functions
    \nwhich call the redefined function change their behavior, or do they
    \ncontinue to refer to the old version? What if the calling interface
    \nchanges? Can you check types if you imagine that functions might be
    \nredefined at any time? If your language has classes, what about
    \ninstances created before a change in the class definition. Believe or
    \nnot, Common Lisp tries to let you do this – and its confusing!<\/p>\n
    On the opposite end of the spectrum are really static languages like
    \nHaskell, wherein type enforcement and purity ensure that the top-level
    \nis only meaningful as a monolith, for the most part.<\/p>\n
    \n","protected":false},"excerpt":{"rendered":"
    I’ve spent around a year now fiddling with and eventually doing real data analytic work in the The Programming Language J. J is one of those languages which produces a special enthusiasm from its users and […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_mi_skip_tracking":false,"footnotes":""},"categories":[19,10,18,47,16,38],"tags":[20,39,40,42,41],"class_list":["post-171","post","type-post","status-publish","format-standard","hentry","category-data-analysis","category-development","category-j","category-programming","category-science","category-work","tag-j-2","tag-jsoftware","tag-matlab","tag-rank","tag-tacit-programming"],"_links":{"self":[{"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/posts\/171","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/comments?post=171"}],"version-history":[{"count":11,"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/posts\/171\/revisions"}],"predecessor-version":[{"id":190,"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/posts\/171\/revisions\/190"}],"wp:attachment":[{"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/media?parent=171"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/categories?post=171"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/procyonic.org\/blog\/wp-json\/wp\/v2\/tags?post=171"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}