Learning Romanian is rapidly turning into a huge exercise in Yak-Shaving.
You often see the question “What stack should I use to develop my application on?” posed but rarely do you get any insight in the actual process once someone decides to answer it specifically, so both as an analogy to rubber-ducking this and showing you what went on while I made my choice I’ve decided to document the whole process. Even though you might not like or agree with the conclusion (which is to a very large extent personal) you may still get some value from the method or the information presented here.
Once I decided to have a go at building my own srs/flashcard web app the question arose to figure out what to build it in. I’ve been putting of this choice for quite a while now, but it really is time I bit the bullet and updated my toolbox and got with the times. There is a lot of general wisdom floating around about this theme (stick to what you know, don’t do your own plumbing (that translates into don’t build your own framework), concentrate on the problem you are trying to solve (yeah, right, learn Romanian), old = good and so on).
So technically speaking this should be a non-choice for me (LAMP + Yii). After all ‘stick to what you know’ is by far the best choice you could make here. And it allows me to re-use my library of home-grown code, which is quite an investment in time and effort. So I had more or less settled on that. But after messing around for a day trying to get eclipse to support Yii (very frustrating, to put it mildly) and the fact that so many of the Yii bits and pieces (such as the Yii eclipse plug-in) are depending on single individuals it is getting harder and harder to defend Yii as a viable choice at this point in time. Yii 2.0 is now in ‘beta’ for however long that will take, and though they write on their website that you definitely should not stop using Yii 1.1 for your new projects at the same time they write that Yii may or may not be supported past 2015. I don’t like gambling that much. And I remember how the Osborne Computer Corporation performed Seppuku on itself.
The good news I guess is that the web is designed around protocols, not pieces of software so that’s all fixable but to require a re-write every few years is really not an option, I simply do not have the time for that. Building things 3 times (MVP, first real version, good enough for long term use) is hard enough, a forced rewrite on someone else’s timetable sounds like a very bad idea.
Let me explain that, I tend to build things 3 times, the first is sort of an exploration of the space, I fiddle around with usually just a bit of data and an extremely rudimentary UI to figure out what it is that I really want to build. Then when that’s sort of crystalized out I build the second version, one that lives for a while, days, weeks or months. This usually has actual use(rs) and maps out the space even further. Inevitably this turns up some really bad decisions made in the first or the second phase that were not apparent at the time. And so after a relatively short time the whole thing gets ripped apart again and then I build it in a way that it can last for a decade or longer while keeping the exterior unchanged as much as possible. It’s surprising how much of the stuff I built in my nearly 3 decades of programming is still in active use today. Even more surprising that the things that are still in use all felt like they were temporary at best, projected lifespan a few year maximum. In one extreme case of this a prototype I built for a company in 1986 is still in production use today. So deciding what stack to build something on is a very important decision.
Choosing not to go with the most familiar previously used combo is against just about all the good advice, but I like learning and this is as good an excuse as any I’ll get to jump out of the box.
My list of requirements is:
- L-M-, to clarify, Linux and MySQL are non-negotiable (because I exclusively use linux for everything I do and I have a ton of data already stored in MySQL tables) This also limits the size of the Yak to some extent and given the direction this is going that is probably a good thing.
- The software should be actively maintained
- The license should be an open source license that is well known or one that is so simple that I can understand it.
- Issues should preferably be solvable using google searches and stackoverflow, I don’t like making tickets so community size and having others walk the path before is important
- The framework/language combo should play nice with other tools (editors, revision control systems and so on)
- It can be ‘opinionated’ but not so opinionated that I have to migrate databases to new schemas to make the framework happy
- It doesn’t have to be the fastest but it should play nice with caching to speed it up should the need arise. The reason for this requirement is that several of my little projects are now in the 50K+ and 100K+ per day visitor counts and if you eat up a lot of cpu per pageview then you’re going to spend a ton on hosting. Not all my projects are for profit so this really matters.
- It should be possible to find collaborators at some point (another community size factor), in my experience success of my toy projects is extremely hard to predict and I’ve been caught several times now with something doing a take-off all by itself.
- Reasonably low bar to entry (quick to get up to speed)
- Right now this is a one-man-show so programmer productivity is important.
- The interface with the framework should feel natural and idiomatic to the language underlying it. In my experience, if that is not the case it will sooner or later cause trouble.
- Actively maintained does not translate into ‘was made yesterday’.
- The team of maintainers of the language and framework should have more than 1 member.
- The language underlying the framework should be solid, can’t be classed as experimental and should not be a moving target.
- Sessions, authentication, routing, internationalization, MVC, caching, mobile, templating, ORM and so on should preferably all be part of the deal, so batteries included please.
Some of these requirements are no doubt contradictory and there will have to be compromise, for instance excellent programmer productivity, low bar to entry and good execution speed are rarely found together.
In other words, time for some web browsing, apply the above criteria, pick a language/framework combo and then get to work. Sounds easy, right?
This opens up the field quite a bit. Stick with PHP? (still plenty of choice in terms of frameworks) Move to a new language? (though the wisdom holds that using a new language for a project that you need or care about will make you hate both). Doing technical due-diligence exposes me to all sorts of technology, and it also helps me in making this choice because I’ve had the privilege of getting a ring-side seat in the arena of capable web start-ups and established companies. You get a ton of examples of how things could be solved and what really works (and sometimes, what really does not).
The amount of choice available to someone that is going to make this decision is absolutely incredible. This wikipedia table lists a staggering number of potentials to choose from, and on a first reading it isn’t even complete (for instance, the Go language is missing, are a whole bunch of frameworks (not just Go ones, for instance, Erlang’s Chicago Boss isn’t on that page). If you’re maintainer of some framework or care about your language ecosystem please update that page!).
What a beautiful illustration of The Paradox of Choice!
Because the list is simply way too long to start comparing the frameworks and then letting the framework dictate the language I’m going to turn things upside down for a while, and use the languages as an indirect way to get rid of a ton of framework selection work, then when the list is small enough we’ll reverse this again and look at the frameworks left over.
Since this is now a process of elimination, let’s come up with some (hopefully) good criteria to winnow down the language set to something a bit more workable.
Let’s start by getting rid of the languages that are in my opinion unsuitable for modern day web development. Goodbye C, Perl, Pascal, SmallTalk and Tcl. Smalltalk also does not conform to the requirement of playing nice with others and being able to use version control tools outside of the image.
Next, let’s get rid of the languages that are too new:
That removes Rust, even though in the long term I have high hopes for the language.
Haskell gets removed because I’m not in with the smart crowd (call me a blub programmer and see if I care) and because I’m not writing some financial application (which is, apparently where Haskell really shines).
Scala gets dropped because I don’t really know it but think it is really terribly ugly. This could be another instance of me not being smart enough or not getting it but I just can’t seem to get past that.
I still haven’t forgiven Microsoft for their countless misdeeds so Asp/C#/F# are all right out. That may be shooting myself in the foot here (cut off my nose to spite my face), but contributing to this decision is that I work exclusively with linux (both on desktops, laptops and servers) so the MS ecosystem has become totally unfamiliar to me and I don’t like running windows on servers (though, of course there are some workarounds possible here and Mono is quite good). So, the L in my LAMP is firmly established. And besides, I don’t have a lawyer on the payroll to make sure that my licenses are all read, ok’d and acceptable.
I’ve once maintained a coldfusion application for a client and I still wake up in cold sweat every now and then, that may have been a very bad example but the taste has not left me so strike coldfusion.
So, now we’re down to 9 candidates, still way too many but at least there is some breathing room. The eight are: Erlang, Elixir, Clojure, Go, Java, Lua, PHP, Python, Ruby.
I’m going to eliminate Clojure on the grounds of it being a super nice language but I really hate the runtime error messages (see above) and I see it used far more in data-analysis roles than in web sites. In a way this is a pity because Clojure has all the good stuff of Java (tons of libraries, compiled, reasonably fast). Now we’re down to 8.
Lua… lua is a very interesting little language, it’s elegant, I really like the concepts behind it, I’m aware of some extremely impressive applications built with it. Think of lua as hitting the sweet spot between Python and Ruby, with a good link to C thrown in for good measure. The grounds for which I’m dropping it are that lua does not feel like it is a stand-alone environment, I always think of it as a plug-in to something else to give it web capability or to make it configurable/extensible. The lack of libraries is also a huge drawback.
Python has always been a mixed bag for me. What I wrote in it worked, when I wrote it. Then, a little while later it stops working, either because some module that it depends on gets deprecated or changed in an incompatible way, starts to exhibit weird behavior or even disappears. I’ve written some thousands of lines of python altogether so I would definitely not consider myself fluent in the language. But contrasted with C code that I wrote 30 years ago that I can compile today this is not a good personal record (though it may very well have to do with my picks of python modules). The significant whitespace has always felt to me like driving on the autobahn at 200 km/hour without guardrails and every time I try using it for web development I end up re-writing it in something more suited to the task. For scientific stuff it has replaced Perl in a large number of use cases and it will be here for a very long time. So Python is out for me, but this is really becoming personal and less objective. Every web project I ever built in python also felt slow to use.
With Python eliminated we are now down to 6 entries in the language category, but Erlang and Elixer should probably be combined on account of their close relationship. (A similar thing could be said for clojure and Java but I think that clojure’s integration into the JVM is less elegant than Elixir is when looking at Beam).
Ecosystem size is a huge factor when evaluating a language/framework combo, and if the ecosystem around a language is small that means that for the most part you’re going to be on your own figuring things out and that if you ever want to hire someone (a contractor, an employee) to help out with some aspect of your project that it will probably be difficult (this would have been another strike against Clojure, come to think of it). Erlang & Elixir definitely have this problem, but I can’t help but be totally fascinated by the ease with which applications can be clustered and made ultra reliable. For instance, ‘whatsapp’ is built in Erlang. So erlang sounds like a worthwhile skill to have under my belt and the eternal draw of technology for its own sake is really making it hard to keep my eye on the ball here (which is, to learn Romanian, in case you forgot). So, with deep regret, pain in my heart and apologies to Joe Armstrong, whose thesis (or should I say Magnum Opus) is a must read for any programmer I’m going to have to drop Erlang, and by strong association, Elixir as well.
Java, PHP and Ruby are all mature and stable languages, Go is new but has some very big names behind it, people that I know are capable of building for the long haul. Sites like langpop don’t even have it on their radar. Go does concurrency very well (always a handy feature), compiles wicked fast and allows me to re-use at least a little bit of my C knowledge. So I’ll keep it in, even though it is only 4 1/2 year old and still extremely small in footprint (joke intended) when compared to Java, PHP or Ruby.
The short-list of languages is now 4 entries long: Go, Java, PHP, Ruby. Two of those (Go, Java) are compiled, and two are interpreted (PHP, Ruby), though, you could compile PHP using facebooks HHVM (HipHop Virtual Machine). I think that’s short enough that we can stop eliminating languages for good reasons and start to expand the list to include frameworks to get an idea of what each language has to offer that speaks for it, rather than to get rid of it, reversing us back to where we began, selecting a framework and let that dictate the language.
Infant mortality amongst frameworks and languages is ridiculously high so going with a ‘young’ framework or language is a significant risk and can turn out extremely expensive in the long run (for instance, after the framework ends up being orphaned you have to make sure your stuff remains afloat so you need to support/extend the framework and you have a serious risk of security issues going unnoticed for a longer period of time). If I’m going to place a long term bet on some chunk of code I would really like to at least have the feeling that it will be around for a while. This will make life extremely hard for Go to stay in the race, after all it is young itself so all frameworks are likely going to be young as well.
Let’s make a listing of what we’re let with at this stage (ripped from the wikipedia page):
||Current stable version
||Apache Software License 2.0 (ASL 2.0)
||Apache Software License 2.0 (ASL 2.0)
|Apache Struts 2
|Google Web Toolkit
|JavaServer Faces (Mojarra)
||CDDL, GPL 2, Apache 2.0,
||Oracle Technology Network Developer License
||Current stable version
||New BSD, GPLv2, GPLv3
||02 Build 01
||Current stable version
|Ruby on Rails
||Current stable version
||Copyright (c) 2012 Rodrigo Moraes. All rights reserved.
||Copyright 2013 Richard Crowley. All rights reserved.
||New BSD License
||New BSD License
That make for 75(!!) frameworks in total for the four languages still left. That’s still a bit much to do say a quick and dirty test application in all of them to see how they feel.
We can use similar criteria used above to get rid of a number of frameworks, first, let’s drop everything that has a proprietary license. I really don’t feel like expending a bunch of effort and then having the rug pulled out from under me because of some obscure passage in a license that I did not evaluate properly. So either it is open source under a well-known license that suits my needs or I’m going to have to pass on that particular framework.
This gets rid of Eclipse RAP, Formengine, Oracle ADF, Spring and Webobjects in the Java arena. For PHP we lose Ayoola, and Solodev CMS, for Ruby and Go we lose none.
7 down, 68 to go.
Next up, have I even heard of the framework? That’s quite subjective, especially for Go, but it’s a good criterium. If a project is so obscure that someone looking into as many companies as I do and a HN junkie to boot has not heard of a framework that’s reasonable grounds for dismissal. For Java we lose Apache Click, Apache OFBiz, Apache Shale, Hamlets, JavaServer Faces, Jspx-bay, JVx, OpenXava, Stripes (vaguely rings a bell somewhere though), Vaadin, ztemplates and 4WS.platform. For PHP that makes short work of Hazaar, Kajona, Logiks, Qcodo, Typo 3 Flow,
In the Ruby camp this loses us Camping and Ramaze. (Technically, I should probably delete all the frameworks in the Ruby category except for Rails, but I have heard of Sinatra, Merb and PureMVC, I just don’t remember if that was in a good or a bad context, which is anecdotal proof that there is no such thing as bad publicity). For Go this means the end of TigerTonic, it isn’t a framework anyway (more of a router) and it could be combined with another framework to do high-speed routing (which it does using a trie).
Another reason to discard frameworks is because they are stale, there is a very small chance that a framework is so perfect that it does not need maintenance for years on end but by my reading a framework that is not worked on continuously is a dead framework. This is because the web is a moving target and frameworks should keep that target in their sight at all times.
For Java this axes OpenLaszlo, RIFE, ThinWire, WebWork (now merged into struts2). For PHP we register the demise of Agavi, Aiki Framework, AppFlower, Cgiapp, PhPixie (or maybe they just ignore those github issues), PRADO (superceded by Yii), Seagull and Xyster. In the Ruby domain Merb has apparently been merged into Rails, and PureMVC looks quite dead. For GoLang it was a sad week because Martini bit the dust after some pretty well aimed criticism, to be replaced by negroni.
That was quite the slaughter, down to 35 frameworks now if I’ve counted that all correctly.
Since the start of this journey was PHP/Yii I’m really tempted to throw out Yii too, I really strongly dislike it when important chunks of code like frameworks are given vague future prospects, a framework is either eol’d or it should be in active development. Saying you may or may not sunset a framework in a years time and in the meantime to encourage users to build new software on top is very irresponsible. I have similar misgivings about Drupal, where a bi-annual re-write is pretty much the only way to stay current. So out of sheer spite and to get me out of my comfort zone I’m going to drop both Yii and Drupal.
The whole Martini/Negroni thing has me in a quandary. On the one hand, I think the response to stop developing the code and to move on to a more ‘proper’ approach is a very mature one. At the same time I’m so terribly happy that I’m not one of the people that started to develop some major chunk of code on top of Martini about 3 months ago or so. I would feel very much let down. The fact that a whole project gets axed by one well aimed blog-post is - to me - proof that one should be very distrustful of anything that has not yet withstood the test of time. Just like ‘too old’ and ‘stale’ are valid criteria for rejection, the same holds true for ‘too new’. Who is to day what will happen to Negroni 6 months from now, when another blog post by a competitor aims to derail it? (The fact that that particular blogpost aimed to also promote the product of the author was less than classy). So I’m dropping Negroni from the ‘Go’ set of frameworks on account of it being too fresh and the maintainer giving me the impression of being quite unpredictable.
Me not having heard of your framework is one thing, but google not really knowing about it is another red flag. Let’s do a little notoriety check and see what turns up:
|Apache Sling||44,000 results|
|AppFuse|| 6470, |
|Grails ||33,000, |
|GWT ||282,000, |
|JBoss ||27,000, |
|Vraptor ||3980, |
|ZK ||57,200. |
|CakePHP ||98,000, |
|CodeIgniter ||285,000, |
|Kohana ||113,000 (not in wp, eol’d!)|
|Fat-Free ||2,580,000 (wrong?)|
|FuelPHP ||8110, |
|Joomla ||513,000, |
|Laravel ||136,000, |
|Go Rest ||6580|
Web is impossible to google for in a normal way.
You can very clearly get an idea of the relative sizes, maturity and adoption of the various frameworks and language eco systems here. (I did not expect such a huge spread and it’s a good indication that I should not go by my gut on things like this, for instance, I would have estimated Sinatra much larger and never knew Play! was that big.)
Based on the relative sizes of the frameworks within each language category I’m going to retire the ones that are substantially smaller than the mid-range. This removes Tapestry, ApFuse, JBoss, Vraptor, wavemaker, FuelPHP, Lithium, Silverstripe, Sinatra and Gorilla, and leaves us with 21 frameworks.
Web frameworks are not all created equal and not all of them are created by teams. Each of them has its own target audience, is more or less feature rich and typically outcompete each other on some aspect but have a large chunk of overlap when it comes to the basics. Those basics had better be there though. For me, that translates into: authentication, routing, session management, multi-language, ajax, MVC, persistence, templating, input validation and testing. If those are not present in a framework then that means I’m going to have to roll my own for that aspect of the framework and in the long term I’d rather maintain the app than a chunk of framework (that’s the whole reason for choosing a framework to begin with!). I dropped my own one-man-show framework years ago because it was getting longer and longer in the tooth, I see no reason to go back, even partially to that situation if I don’t have to. Caching is optional, I assume that the makers of a framework are dedicated to keeping things secure. I also like it when frameworks have a team behind them rather than a single person as the main driver.
On account of not having testing, internationalization and an ORM I’m dropping Sling (though likely you could use ActiveRecord), Wicket does not do MVC so it goes, Grails is not written in Java, CodeIgniter has no ORM and no internationalization, Joomla is more of a CMS than a framework, Laravel has a ‘bus-factor’ suspiciously close to 1. The ‘Go’ frameworks are not documented in Wikipedia so this is a bit more work to figure out. GoRest appears to be mostly a router rather than a complete framework, GoCraft ditto (it’s a router ‘web’ and a simple orm ‘dbr’). Hoisie/web is another router. The Go world seems to be taking an extremly modular approach to the framework problem, in a way I like that (modularity is good), but it also means that you have to do a lot of hunting and picking to get a set that you like and it is extremely likely that you are then one of the few using that exact combination. It also signficantly increases the chances of one of those elements being suddenly end-of-lifed (the same thing happens in the Drupal world all the time, with every major release there are tons of orphaned modules).
12 frameworks left, and still, the variety is enormous. I think the next thing to do is to write a very small sample application in all of those (setting them up will be a total pain) to see how well it all comes together and what the friction points are. To make sure I end up with a good choice I have to at a minimum check that all my requirements listed above are fulfilled, and enough of them have to do with actually working in the environment that I won’t be able to check without getting my hands dirty.
The frameworks that are left now are Ruby on Rails, for php CakePHP, Fat-Free, Nette, Symfony and Zend, for Java Struts2, GWT, Play! ZK, for GoLang beego and Revel.
To be continued. Interesting reading along these lines on other people’s pages:
Edit: in the HN thread about this article user nemasu points out that Kohana is no longer actively supported so I will remove it from the list of candidates, also I incorrectly axed C# as being a Microsoft only product, but I feel that even though there are re-implementations of C# the language is under Microsoft control (just like Go is under Google’s control, even if someone else wrote another Go compiler). If and when Microsoft is no longer the dominant factor in the direction of C# would that situation change sufficiently for me to make C# an option for development.