Who needs math skills?

Após quase um mês de aulas no doutorado do CIN, percebi o quanto estou enferrujado em alguns princípios básicos de cálculo. Durante a minha graduação cursei 4 disciplinas de cálculo, 3 de álgebra e 1 de estatística. Mesmo assim, senti uma dificuldade enorme em acompanhar as primeiras aulas – em grande parte, pelo tempo que passou entre a graduação e o meu novo contato com o mundo dos números. Mas o principal motivo foi o simples fato que nunca precisei utilizar esses conhecimentos na minha carreira como analista de sistemas. Esse assunto veio à tona quando li o post de Alan Skorkin falando exatamente sobre os problemas que estou enfrentando agora. Vale a pena dar uma olhada.


You Don’t Need Math Skills To Be A Good Developer But You Do Need Them To Be A Great One

A little while ago I started thinking about math. You see, I’ve been writing software for quite a few years now and to be totally honest, I haven’t yet found a need for math in my work. There has been plenty of new stuff I’ve had to learn/master, languages, frameworks, tools, processes, communication skills and library upon library of stuff to do just about anything you can think of; math hasn’t been useful for any of it. Of course this is not surprising, the vast majority of the work I’ve been doing has been CRUD in one form or another, that’s the vast majority of the work most developers do in these interweb times of ours. You do consulting – you mostly build websites, you work for a large corporates – mostly build websites, you freelance – you mostly build websites. I am well aware that I am generalising quite a bit, but do bear with me, I am going somewhere.

Eventually you get a little tired of it, as I did. Don’t get me wrong it can be fun and challenging work, providing opportunities to solve problems and interact with interesting people – I am happy to do it during work hours. But the thought of building yet more websites in my personal time has somewhat lost its luster – you begin to look for something more interesting/cool/fun, as – once again – I did. Some people gravitate to front-end technologies and graphical things – visual feedback is seductive – I was not one of them (I love a nice front-end as much as the next guy, but it doesn’t really excite me), which is why, when I was confronted with some search-related problems I decided to dig a little further. And this brings me back to the start of this story because as soon as I grabbed the first metaphorical shovel-full of search, I ran smack-bang into some math and realized exactly just how far my skills have deteriorated. Unlike riding a bike – you certainly do forget (although I haven’t ridden a bike in years so maybe you forget that too :) ).

Broadening Horizons

Learning a little bit about search exposed me to all sorts of interesting software-y and computer science-y related things/problems (machine learning, natural language processing, algorithm analysis etc.) and now everywhere I turn I see math and so feel my lack of skills all the more keenly. I’ve come to the realization that you need a decent level of math skill if you want to do cool and interesting things with computers. Here are some more in addition to the ones I already mentioned – cryptography, games AI, compression, genetic algorithms, 3d graphics etc. You need math to understand the theory behind these fields which you can then apply if you want to write those libraries and tools that I was talking about – rather than just use them (be a producer rather than just a consumer – to borrow an OS metaphor :) ). And even if you don’t want to write any libraries, it makes for a much more satisfying time building software, when you really understand what makes things tick, rather than just plugging them in and hoping they do whatever the hell they’re supposed to.

The majority of developers will tell you that they’ve never needed math for their work (like I did a couple of paragraphs above :) ), but after musing on it for a while, I had a though. What we might have here is a reverse Maslow’s hammer problem. You know the one – when you have a hammer, everything looks like a nail. It is a metaphor for using a favourite tool even when it may not be best for the job at hand. Math is our hammer in reverse. We know the hammer exists but don’t quite know how to use it, so even when we meet a problem where our hammer would be the perfect tool, we never give it serious consideration. The screwdriver was good enough for my granddaddy, it was good enough for my daddy and it is good enough for me, who needs a hammer anyway? The trick with math is – people are afraid of it – even most programmers, you’d think we wouldn’t be, but we are. So, we turn our words into a self-fulfilling prophecy. It’s not that I don’t need math for my work it’s just that I don’t really know it and even if I do, I don’t know how to apply it. So I get by without it and when you make-do without something for long enough, after a while you don’t even notice it’s missing and so need it even less – self-fulfilling prophecy.

Here is some food for thought about something close to all our hearts – learning new skills. As a developer in the corporate world, you strive to be a generalizing specialist (read this book if you don’t know what I am talking about). You try to be decent at most things and really good at some. But what do you specialize in? Normally people choose a framework or two and a programming language and go with that, which is fine and worthwhile. But consider the fact that frameworks and to a lesser extent languages have a limited shelf life. If you’re building a career on being a Hibernate, Rails or Struts expert (the struts guys should really be getting worried now :) ), you will have to rinse and repeat all over again in a few years when new frameworks come along to supersede the current flavour of the month. So is it really the best investment of your time – maybe, but then again maybe not. Math, on the other hand is not going away any time soon. Everything we do in our field is built upon solid mathematical principles at its root (algorithms and data structures being a case in point), so time spent keeping up your math skills is arguably never wasted. And it, once again, comes down to really understanding something rather than just using it by rote – math can help you understand everything you do more deeply, when it comes to computers. Infact, as Steve Yegge said, what we do as programmers is so much like math we don’t even realise it.

What/Who Makes A Difference


You don’t believe me, then consider this. Most of the people who are almost universally respected  in our field as great programmers are also great mathematicians. I am talking people like Donald Knuth, Edsger W. Dijkstra, Noam Chomsky, Peter Norvig. But then again these guys weren’t really developers, they were computer scientists, so it doesn’t really count right? I guess, but then again, maybe we shouldn’t really talk until our output in pure lines of code even begins to approach 10% of what these people have produced. Of course, you can be successful and famous without being a boffin, everyone has heard of Gavin King or DHH. That’s kinda true (although it’s an arguable point whether or not many people have heard of Gavin or DHH outside their respective niches), but “heard of” and universally respected are different things, about as different as creating a framework and significantly advancing the sum-total of human knowledge in your field (don’t get me wrong, I respect Gavin And David, they’ve done a hell of a lot more than I have, but that doesn’t make what I said any less of a fact). How is all of this relevant? I dunno, it probably isn’t, but I thought I’d throw it in there anyway since we’re being introspective and all.

The world is getting filled up with data, there is more and more of it every day and whereas before we had the luxury of working with relatively small sets of it, these days the software we write must operate efficiently with enormous data sets. This is increasingly true even in the corporate world. What this means is that you will be less and less likely to be able to just “kick things off” to see how they run, because with the amount of data you’ll be dealing with it will just grind to a halt unless you’re smart about it. My prediction is that algorithm analysis will become increasingly important for the lay-programmer, not that it wasn’t before, but it will become more so. And what do you need to be a decent algorist – you guessed it, some math skills.

So, what about me? Well, I’ve decided to build up/revive my math skills a little bit at a time, there are still plenty of books to read and code to write, but I will try to devote a little bit of my time to math at least once in a while, because like exercise, a little bit once in a while, is better than nothing (to quote Steve Yegge yet again). Of course I have a bit of an ace up my sleeve when it comes to math, which is good for me, but luckily with this blog, we might all benefit (I know you’re curious, I’ll tell you about it soon :) ).

Where Do You See Yourself In 5 Years


So, is all this math gonna be good for anything? It’s hard to say in advance, I am pretty happy with where I am at right now and so might you be, but it’s all about potential. End of the day, if you’re a developer in the corporate world you don’t really need any math. If you’re happy to go your entire career doing enterprise CRUD apps during work hours and paragliding or wakeboarding (or whatever trendy ‘sport’ the geeky in-crowd is into these days) during your off time then by all means, invest some more time into Spring or Hibernate or Visual Studio or whatever. It will not really limit your potential in that particular niche; you can become extremely valuable – even sought after. But if you strive for diversity in your career and want to have the ability to try your hand at almost any activity that involves code, from information retrieval to Linux kernel hacking. In short if you want to be a perfect mix of developer, programmer and computer scientist, you have to make sure you math skills are up to scratch (and hell, you can still go wakeboarding if you really want :) ). Long story short, if you grok math, there are no doors that are closed to you in the software development field, if you don’t – it’s going to be all CRUD (pun intended)!

NoSQL: O fim dos bancos de dados relacionais?

O Digg é mais um grande nome da Web 2.0 que acaba de migrar os seus (gigantescos) conjuntos de dados do mundo relacional para o modelo “pós-relacional”, esse último conhecido como NoSQL. Eles se juntaram à empresas como Google, Amazon, e-Bay, LinkedIn, Twitter e Facebook, com o objetivo de prover níveis de performance mais adequados para as consultas realizadas em bases de dados monstruosas, típicas de aplicações da Web 2.0.  Para ter idéia do problema, imagine uma consulta na base de 2 PB (dois Petabytes) do e-Bay sendo feita online por um conjunto de centenas ou milhares de usuários simultaneamente. Imagine agora fazer um join com essa quantidade toda de linhas em tabelas de um banco relacional.

A estratégia do Digg está descrita em dois posts  no blog do serviço. O primeiro fala  sobre as dificuldades de escalabilidade da infraestrutura de banco de dados, baseada em uma solução mestre-escravo particionada um servidor MySQL. O texto mostra um exemplo de um a consulta com join que levava 14 segundos para ser completada. Após a migração para um datastore não-relacional, baseado no modelo distribuído do Cassandra, a mesma consulta pôde ser realizada em menos de um segundo. O segundo texto fala sobre a migração completa dos principais serviços do Digg usando o Cassandra e ainda lista as principais contribuições da equipe de desenvolvimento para o projeto.

O Futuro de Java

Interessante post de Eric Bruno sobre o futuro de Java como plataforma e linguagem de programação. O post original pode ser visto aqui. Um dado curioso extraído do texto é sobre a quantidade de linguagens dinâmicas que atualmente utilizam a JVM para rodar. A lista inclui Rexx, Ruby, JavaScript, Python, PHP, Groovy, Clojure e Scala.A justificativa para esse movimento é a melhora sensível na execução das aplicações através de recursos como os compiladores Just-In-Time da plataforma.

Apesar de muita gente nos últimos meses ter começado a fuçar outras linguagens em substituição a Java (eu me incluo nesse meio), a linguagem ainda é a mais utilizada pelos desenvolvedores, com mais de nove milhões deles construindo aplicações para as mais variadas plataformas e dispositivos. Um fato que assustou a comunidade Java foi a recente compra da Sun pela Oracle e as incertezas no futuro da plataforma geradas pela aquisição. Os caras da Oracle se comprometeram a manter o Java Community Process como o fórum de discussão e evolução da plataforma e adiantaram ainda que a nova versão (chamada provisoriamente de Java 7) trará algumas mudanças significativas, incluindo a modularização da JVM, o suporte nativo à outras linguagens e melhorias no suporte ao processamento com vários núcleos. A integração da JVM com o JRockit da BEA levará a criação de um novo mecanismo de garbage collection, trazendo alguns benefícios adicionais de performance para as aplicações.

Eric também prevê que o sofrimento atual dos desenvolvedores de aplicações para web na criação de interfaces com o usuário, deverá  ser amenizado quando ocorrer a evolução natural das novas versões de JavaFX. Confesso que não me animei muito em estudar essa nova API, e atualmente estou estudando e desenvolvendo uma pequena aplicação com o Vaadin, um framework criado como uma extensão dos componentes de GWT. Assim que a aplicação entrar no ar, eu divulgarei a URL para vocês acessarem.