Developer Q&A: Creativity, Productivity and the Future

Hello again, this is Kevin O’Shaughnessy from zombiecodekill.com presenting another question and answer session with some of the world’s top developers.

Last year’s Q&A on career strategies was so much fun and so popular that we just had to do it again. This time we cover a range of new areas. We cover creativity and productivity, two very important abilities for developers that tend to come and go and are always a challenge to possess consistently.

We also take a look into the future of programming, native vs open web, and the rise of functional programming.

Our expert panel is comprised of the following 4 developers:


JonSkeet

Jon Skeet

Jon is the highest rated user on Stack Overflow with over 32,924 answers at the time of writing. He is especially known for his detailed knowledge of C#, has written the book C# in Depth, and is a Microsoft MVP. He is also highly skilled in the Java language.

phk

Poul-Henning Kamp

Poul-Henning has committed many significant contributions to the FreeBSD operating system for the past 15 years. He is the lead developer and architect for the open
source Varnish cache project, an HTTP accelerator which is boosting the performance of more than 2 million websites.

Dan

Dan Abramov

Dan has made many significant contributions to Open Source JavaScript projects, such as React. He created React Hot Loader, Redux and React DnD. His video series on Getting Started With Redux is highly popular.

Erik

Erik Dietrich

Erik is a software architect, team leader and technologist.

An active blogger with extensive experience teaching and demonstrating software development techniques, Erik has been recently focusing on the .NET framework, though over the years he has worked with C++, Java, and a number of other languages.

His projects range from low level driver and kernel module programming all the way up to user interface design. He has produced four courses with Pluralsight, including “Making the Business Case For Best Practices


The Questions

I asked each developer 9 questions:

  1. Software engineering is all about creating, but programmers are usually thought of as more logical and analytical, rather than creative. What does creativity mean to you, and how creative are programmers in general?
  2. What’s your process for software design?
  3. What is the last thing you’ve worked on that you’ve found very interesting or exciting?
  4. Is there any advice that you can give on how to be more productive at work?
  5. Some people say that the key to being more productive is to do less, but this sounds counter-intuitive. Do you agree or disagree with this?
  6. Is creativity at odds with productivity, or can one help the other?
  7. What’s your take on the future of native mobile vs the open web?
  8. Given the increasing popularity of functional paradigms, what do you believe the future holds for object-oriented programming?
  9. Which industry figure has been the biggest source of inspiration for you and your career?

What does creativity mean to you, and how creative are programmers in general?

Jon:

I don’t think I’d like to generalize too much – every software engineer is different. There are many different aspects to creativity, and in some cases creativity needs to be actively stifled in production code. For example, I love thinking up different ways of abusing C#, and sometimes ways of using it in novel and genuinely beautiful ways.

But there’s always a balance between beauty and familiarity. If a particular solution works well but is very alien to the people who will be using it or maintaining it, that can lead to hard decisions.

So many aspects of software engineering can be creative. There are the more obvious ones involved in building an app which aren’t necessarily done by software engineers – level design in games, or graphic design in general – but even something as commonplace as naming can be very creative.

We all know naming is hard, and we’d be ready to credit a book author with creativity for finding just the right words to express an idea – so I don’t see why we shouldn’t do the same thing in software. Then there’s creativity in architecture, lower-level algorithm design, and the ability to recognize a real-world need and see a new solution to it.

PHK:

You’d think it would be simple: Programmers create programs, Q.E.D: they are creative.

But while people will call anybody engaged in arts “creative”, even if what they do, playing the piano for instance, demands the most exacting precision and attention to detail, programming (and engineering in general) cannot possibly be “creative”, because the work demands the most exacting precision and attention to detail.

Also creativity in programmers isn’t a very useful predictor: VI and EDLIN were both written by creative programmers, one is brilliant, the other horrible.

I am not sure we can ever bestow a sensible meaning on the word “creative” in a programming context.

Programming is special in that the art of programming is incomprehensible to the lay person: There will never be a “Museum of Modern Programming” where the public can appreciate the creative genius of Duffs device or the sheer and powerful simplicity of the original UNIX kernel.

While painters will pause and look stare at any painting they see, sculpturers will run their hand over any surface in reach and authors read books the way the rest of us eat cookies, programmers are almost totally unwilling to look at any other source code than their own.

Ask people to “mention five great programmers” and they rattle of well known names. But if you instead ask them to “mention five great programmers whose programs you have actually read” most come up short.

Dan:

I think creativity means having an open mind, being constrained in your work by external factors, and embracing synthesis.

Most of the times, rather than try to “invent” something, you need to have a good look around. You will likely find that somebody has tried this before—maybe in an obscure language or a library. It is good to learn from those attempts first. Keep an eye on what other people are doing.

I found that people produce the best work when they are constrained by something imperfect and grounded in the real world. Constraints guide creativity. Sometimes constraints change, and what couldn’t work a year ago becomes a good solution today. It’s a good idea to reevaluate why you gave up on certain approaches once in a while. Don’t develop knee-jerk reactions to the ideas you once rejected.

I think great developers try to avoid creativity when they can. Creativity usually means a gotcha, somebody will have to spend time understanding the result. I think creativity is best confined to small areas of your app or library, and most of the code should be boring. Boring means easily disposable, which I believe most of the code should be.

As Ryan Florence says, draw the right lines in your app, fill them with garbage, and ship. I can very much relate to that.

Erik:

To me, this implies a definition of “creative” that draws contrast to logical and means something like “open ended and focused on aesthetics.” I think of it as described once in a book I read. There’s a concept called divergent thinking that would say something like, “think of as many possible uses for a pillow as you can in next 2 minutes. Go.”

If you say, “putting your head on while you sleep, and…. that’s all I’ve got,” you’re a convergent thinker.

If you say, “sleeping, hitting without intent to injure, wedging my car door open, screaming at in frustration, etc,” then you’re a
divergent thinker.

To me, this concept of divergent thinking correlates heavily with creativity. And, often, programmers are thus quite creative. I look at a lot of legacy code for a living, and, I can definitely say that programmers find the most improbable ways to accomplish what seem like very straightforward tasks.

What’s your process for software design?

Jon:

That very much depends on the project. For personal somewhat-throwaway projects, I’ll just dive straight into code (or tests if I’m feeling diligent) and let the design emerge as I explore the space. For personal open source projects there’s a mixture of that and a lot of talking with people – I have great colleagues who are always ready to discuss API design, for example.

In other cases, there’s been a lot of whiteboarding involved – I’m longing for good, affordable electronic whiteboards to come on the market – and in plenty of cases it’s just a matter of writing documents.

Writing something down is a great forcing function for coming to terms with the pros and cons of different approaches, and at least listing areas for more work. The document is then a forum for collaboration – if a document isn’t peppered with comments, it probably hasn’t been shared enough yet.

PHK:

For me it works pretty much like the Feynman Method of Solving Problems:

1. Formulate the question.
2. Think about it.
3. Write down the answer.

There is a very large aesthetic component to it, in the sense that something “feels” right or wrong, but why that is so is often incredibly hard to put words around.

The actual nitty gritty design work is sort of a background mental process for me. Usually the pieces of the puzzle fall into place while I’m on my bike, pulling weeds in the garden or in the shower.

Dan:

For API design, I find that writing example apps that use those APIs is much more fruitful than drawing diagrams. The last time I remember drawing diagrams is when I used to take “Design Patterns” and UML seriously. I was 15 then. Thankfully I grew out of this.

I doodle a lot when I need to solve an algorithmic problem. I’m very bad at algorithms, data structures, geometry calculations, and similar mind-CPU-intensive problems. I need to make visual cues for myself because I forget what the numbers mean, how the entities relate to each other, or what happens to them over time. If you ask me to sort an array or to calculate the position of something based on the viewport and the scroll offset, I will fail miserably without a paper and a pen.

Erik:

The last stint I did as a salaried employee, I had a nicely appointed office with a pair of large, wall-mounted whiteboards. I got extremely addicted to using them while there, so I have a preference for whiteboards, when available. However, I do a lot of remote work now from all sorts of places, so I find myself going to pen and paper a lot. This is where I sketch out high level designs, using diagrams.

Once I feel good about general direction, though, it’s to the keyboard to start writing code.

What is the last thing you’ve worked on that you’ve found very interesting or exciting?

Jon:

I’m finding the C# standardization work for ECMA to be fascinating. Reading the specification through sufficiently thoroughly to try to find mistakes has definitely given me a fresh appreciation for how well it’s written – and a few places where there’s still room for improvement.

As we’re not trying to actually change the language, this is more of an effort around precision and communication than language design. It’s fairly exhausting work, but very rewarding.

PHK:

A prototype of the “Wave Front Real Time Computer”, part of the ESO Extremely Large Telescope currently being built on top of a remote mountain in Chile.

The ELT AO solution will be larger than anything ever done before: 5004 2D-vector wave front inputs and 6350 actuator outputs, 500 Hz update rate.

Adaptive optics for astronomical telescopes is usually implemented in racks full of custom built hardware with lots of DSPs or FPGAs, which leads to all sorts of maintenance issues 10 to 30 years down the road when the designer, the company and the chips are all unavailable.

ESO was looking for new ideas and our prototype used six standard Dell servers, running unmodified FreeBSD 9-RELEASE. It overperformed by a factor of two in speed and jitter, improving the optical performance, so that was really satisfying.

ESO Extremely Large Telescope prototype
ESO Extremely Large Telescope prototype

Dan:

I just joined the React team at Facebook, and naturally I find working there very exciting.

I am currently revamping the performance measurement tooling inside React. The old measurement code was very tightly coupled with the core codebase and as a result both team members and outside contributors kept breaking it all the time while adding features or fixing bugs. I like gradually untangling knots like this and refactoring them so people can move faster without breakage.

Erik:

I’ve had a few exciting projects and prospects lately for which I’m under NDA, but one of my newest projects helping aspiring freelance software developers hang out their shingles as quickly and easily as possible. Given that it’s the path I followed, I certainly believe in the cause, and one of the things I enjoy most is sharing things I’ve learned and encouraging developers.

Is there any advice that you can give on how to be more productive at work?

Jon:

Do work you love. If you’re not actually keen on doing the work, it’s unlikely that you’ll be particularly productive at it. If it’s something you’re passionate about, it’s amazing how much you can get done.

Of course, that’s easy to say, and I’m incredibly fortunate to be in a company and role which I adore. But if you find yourself in a job you’re not happy with, work out what you’d need to do to change that. It sounds a bit of an extreme way to improve productivity compared with “get a good keyboard” or “have a whiteboard near your desk” but I think the result can be more extreme too.

PHK:

Learn to recognize when you are blocked.

If you are blocked on something, stop beating your head against it, that will only waste your time and frustrate you.
Should you by accident happen to solve it, you will invariably replace the “solution” with something much better in a matter of days.

Stand up, walk away from the computer, and do something else, anything else.

Clean the kitchen, fix the cabling, wash the stairs, read Tolstoy.

It doesn’t matter what you do, as long as you are no longer wasting your life getting nothing done.

More often than not, it only takes 10-15 minutes before you think of a really good solution.

Dan:

Identify and attack the biggest risks first. For example, if you are building a feature with a nice animation and third-party service integration, investigate that integration before spending a week polishing the UI. If the third-party service is unreliable or doesn’t work the way you expect, you just wasted a lot of time for nothing.

If you don’t know what the biggest risks are, consider investing time into finding them. Providing good estimates is also work and takes time, so factor it in if you want the project to be successful.

Get over the fear of the blank sheet as soon as you can. When confronted with a problem that requires a thought-out solution, I deliberately write a super hacky and broken version first so I don’t feel bad about not thinking it through. Writing a hacky solution helps me internalize different aspects of the problem and “cache” it in my mind so I can spend less resources working on it.

This is when I can start working on clean solution, diverging into other “dirty” solutions whenever I feel I need to get a better grasp of the problem.

Erik:

Turn off notifications. If you can get the world to leave you alone for a block of time (say, a pomodoro), then do so. Put your phone on silent and in your pocket.

Close outlook.
Close your browser.
Unplug the phone at your desk.
Hide from other humans.

Then work for a block of time, telling yourself you’ll plug back in and answer people’s questions when the block of time is over.
You need truly uninterrupted time.

Some people say that the key to being more productive is to do less, but this sounds counter-intuitive. Do you agree or disagree with this?

Jon:

Differentiate between the activity and the output. In software, if you can end up with a smaller amount of clear code that has the same functionality as a larger amount, that’s a huge win for maintainability, but I often find I’ll get there via the larger amount of code.

It can be hard to find the essence of the code before you’ve really explored the space. That exploring could take place in an IDE, in a word processor or at a whiteboard of course. It depends on the task.

I’d certainly put high value on work/life balance – I don’t think working 60 hours a week is a good way to be productive. Down-time is important, and family and friends are priceless. Having said that, if you end up working 40 hours at your job, and then spending another 20 hours on personal open source projects, what does that count as?

PHK:

Very few people fully appreciate the sheer complexity of what we do.

The aircraft carrier Ronald Reagan, the worlds largest, was built from one million individually tracked SKU numbers in the shipyard.

The FreeBSD OS, which is embedded in countless devices aboard the ship, from televisions and photocopiers to missile maintenance equipment and radars, is 18 million lines of source code – and that is not counting the application programs which do the stuff people care about, showing television, making copies, checking missiles or spotting enemy aircraft.

At a typical bug rate of one bug per thousand lines, we should be skeptical and justify the existence of every single line.

In that sense it may have been a mistake to stop using punched cards: They gave a very tangible feedback that a code line had a cost and was something to be avoided, rather than a treasure to be accumulated.

I won’t say that 1 line of code is always better than 2 lines of code, but Antoine de Saint-Exupery was certainly on to something crucial when he said that “Perfection is Achieved Not When There Is Nothing More to Add, But When There Is Nothing Left to Take Away”

And the code lines you do not write run a lot faster than the ones you do write.

The problem is made worse by the above mentioned unwillingness to study other programs and the even more stubborn unwillingness to work together constructively.

At one point I did a survey and found 1342 copies of the MD5 algorithm in the 20,000 packages in the FreeBSD ports collection, because there were no standard crypto library with this function. If a handful of OS-programmers had spent a day standardizing a library of crypto functions, at least 1341 people wouldn’t have wasted time working around it not being there.

Dan:

I like thinking that if you can’t keep your todo list in your head, you are fooling yourself. Of course it’s useful to keep track of all the things that need to be done (e.g. before the release), but using the backlog as your todo list is very frustrating because most likely it will be too large, and make you feel stressed and guilty.

Do a role-play and let your inner developer and inner manager find their peace. They should respect one another. If your gut feeling tells you some task will just never get done, accept it, and let everyone know. Don’t pretend to others or to yourself that you’re going to do something that you know deep down will never happen. We have a poster at Facebook office saying “prioritize ruthlessly”—that’s what it means.

Erik:
I agree with the premise, but I’d offer a caveat. Do less, but not indiscriminately. The key to being more productive is to do fewer things that don’t matter.

Is creativity at odds with productivity, or can one help the other?

Jon:

I think I’d argue that in many cases, creativity is productivity – if I can find an elegant solution to a design problem that’s been stumping me for days, that’s a productive day. I would be concerned about how much creativity I was really achieving if I spent a lot of the time typing fast though… I wouldn’t equate productivity with “lines of code written” for example.

PHK:

How can it be, that a couple of dudes in a basement or garage can create the nucleus of Google or Facebook, but hundreds of millions of dollars and thousands of employees invariably screw up governmental IT?

How is it even possible that the productivity between all supposedly highly competent programmers vary by three orders of magnitude?

It’s no wonder that people think of IT and computers as a kind of magic, but it isn’t, and only when we stop treating it as such does productivity improve.

A lot of the magical thinking happens in what I call “the lost generation“.

During the dot-com boom the IT-business massively grew its headcount, and none of these people got introduced to what we had already learned and experienced.

Many still think the world started with the PC and the Internet, and can’t imagine what they could learn from the ancient and primitive computing on mainframes before 1990.

A really good book about productivity in programming was published in 1975, The Mythical Man-Month by Fred Brooks, but most programmers have never read it.

Slowly but surely the lost generation reinvent version control, and “Agile development”.
Few people know that Agile is essentially a reinvention of the whole job method described back in 1983:

“The “whole-job” software design method puts all responsibility for a project, including requirements and development, on a small design team. The principal advantage of this approach is that the developers themselves have a close relation with the clients. This results in other derivative advantages: high motivation, flexibility, and low overhead.” – Bell System Technical Journal, September 1983

The key to high productivity is to not waste your time on things which are impossible, difficult or stupid to do.

But to determine that you must understand the problem you are trying to solve, what part of the problem computers and humans are good at, and what part they are not good at.

Creativity is important, but experience and insight are crucial.

Dan:

I feel that many real-world solutions follow the “double diamond” approach. As long as you’re clearly signaling to others and to yourself in which stage you are right now, creativity and productivity go hand in hand.

However, as I said before, beyond the research and synthesis phase, execution is much more important than being seen as “creative”.

Erik:

Creativity, in the divergent thinking sense, is a gamble. Think of the “sharpening the saw” adage. Laboring away cutting down trees is mindlessly productive. Thinking of ways to make the cutting go faster is a form of brainstorming, and is creative.

Creativity helps if you have hundreds of trees that need to be cut down. It doesn’t if the goal is to cut down one tree, right now. Over the long haul, I’d say that creativity not only helps productivity, but is utterly vital to it. Sooner or later, all of your competitors will figure out better techniques for tree-cutting.

What’s your take on the future of native mobile vs the open web?

PHK:

I hate native mobile, because it goes against everything we have (or should have) learned from the Software Tools principle.

When the data are locked down inside independent and non-cooperative apps, we get robbed of the ability to (re)use the data in new ways the writers of those apps did not imagine.

Dan:
I’m optimistic about both. I would love to see their strong sides converge.

I think mobile development should have instantaneous feedback cycle like the web development. Recompiling apps for longer than several seconds in development is terrible, and as React Native shows, completely avoidable. Being able to deploy updates to JavaScript logic over the air is also a big win although I am still not sure how mobile gatekeepers like Apple look at it.

Web development should, in my opinion, offer more lower level APIs, in particular related to measuring text, low-level accessibility APIs, and other things that are impractical to do in userland. I think DOM comes with a ton of legacy baggage, and we can improve on it with a declarative model similar to React. Right now React in the browser is built on top of DOM but this takes away some valuable optimization opportunities. When Flipboard came out with React Canvas, many criticized their approach for the lack of accessibility considerations. However I think the underlying problem isn’t that we shouldn’t replace DOM; it’s that browser vendors should give us better tools to do so.

Finally, I’m enthusiastic about sharing code across platforms. With React Native you can write most of your app logic as cross platform code. Even parts of the UI that are not platform specific are shared, and you can introduce platform specific components at any point in your component hierarchy by creating files with different extensions (Button.ios.js, Button.android.js). I can imagine Button.web.js working the same way in the future.

The benefits of having a shared codebase are not so much technical as they are organizational: now you have a single product team owning the app, rather than two separate engineering teams separated by the technology. This gets me excited about React Native even though it is still in an early stage.

Erik:

It always reminds me of being a programmer at the beginning of my career. At that time, software often shipped on CD-ROM and you’d need to build entire, separate versions to ship to Linux, Windows, and Mac. Then the web app came along and changed all that, consolidating code.

Then, years later, Apple came along and changed it all back, restoring the native app to its former glory. I imagine the pendulum will keep swinging back and forth, frankly, since society seems to love over-correction.

Given the increasing popularity of functional paradigms, what do you believe the future holds for object-oriented programming?

Jon:

Well, I’d be surprised to see any new languages which didn’t have at least some functional aspects, for one thing. For some years now, C# has been openly embracing many of the features of functional languages (particularly F#) and reapplying them in a way which feels appropriate for C# developers, and I’d hope other languages will take inspiration in a similar way.

I’m not sure whether more “pure” functional languages will take over from object-oriented languages, but I think some of the underlying benefits of functional programming around clarity may prove fruitful in future language design. The less context you need in order to understand any one piece of code, the simpler it will be to understand that code. That’s at the heart of component-oriented design or simple object-oriented encapsulation, of course, but there are lessons to be learned from our functional friends in terms of how that works out – and how it lends itself to composition.

I’m interested to see the balance between new languages on existing platforms, new languages on genuinely new platforms, and the evolution of existing languages, too. There are very different costs and opportunities in each case. I doubt that the world will become boring any time soon!

PHK:

To me there is no real difference between programming and object-oriented programming, the difference is only which direction you drive your programs logic, and language has very little to do with it either:
I had written object-oriented code in assembler and C before Bjarne Stroustrup came up with Cfront.

I don’t see “traditional” programming, object-oriented or otherwise, going anywhere anytime soon, for the very simple reason that it works and is simple to understand and implement for a very large class of problems.

To me functional programming seems more like an existential coping mechanism for complexity than anything else: By restricting ourselves to elements we can mathematically prove the soundness of, we hope that a conglomerate of such elements will behave in a predictable way, or for the truly ambitious: a provably correct way.

As with any other “revolution in programming”, functional paradigms will have utility in a certain set of situations, governed by limitations that we will realize over time.

I’m probably more of the Von Neumann school there, thinking that the way forward is to learn how to build complex systems that will keep working even when bits and pieces of them are unreliable or unavailable.

We don’t know how to do that, but we do know enough to use functional programming paradigms, so gives functional programming a clear edge for now.

Dan:

Some parts of object-oriented programming very neatly map to how we think about things and it is not going away anytime soon. Grouping data with functions that operate on this data is easy to get used to, and in many cases it’s an okay pattern, and its convenience often outweighs its downsides.

We have also learned about weaker sides of object oriented programming. For example, deep inheritance (note: mixins are no better) introduces very complex indirection, implicit dependencies, and rigid code that does not respond well to changes in requirements. Mutations and side effects scattered across the codebase mean it’s hard to organize when they happen, interpret them differently, or to perform transactions.

The functional paradigm has shown us the benefits of separating the data from operations on this data, and keeping data immutable. This often gives great results but can have unfortunate corner cases in performance, interoperability with existing third-party code, and intuitiveness. We should carefully study these edge cases rather than brush them off. We should work on making functional programming user-friendly.

These are just two tools in your toolbox, and there are many more. Ultimately, I think the “right” paradigm can’t save a bad API, and the “wrong” paradigm can’t bury a good one. The difference between functional and object-oriented seems less important to me than the difference between declarative and imperative, for example. Both of the paradigms can be twisted in both ways.

The best APIs are the ones that let you write disposable code. They let you express the same ideas with less coupling and less “rippling changes” across the codebase when you make changes. In many cases, declarative programming helps achieves this, and it can use both functional and object-oriented programming as its means.

Erik:

In keeping with the spirit of the last answer, I’d say that Object-Oriented Programming will probably continue to wane in popularity with the rise of functional programming, until someone rediscovers it and re-popularizes it years from now. After all, I believe Functional Programming predates Object-Oriented Programming as a paradigm, so the new hotness is actually the retro hotness.

Which industry figure has been the biggest source of inspiration for you and your career?

Jon:

I’m privileged to know both Eric Lippert and Mads Torgersen, giants of the C# world. Eric has been the technical reviewer for all three editions (so far) of C# in Depth, and Mads is the PM for language design on C#.

I’ve learned huge amounts from both of them, and hope to continue doing so. I’d say that Eric has shown me different ways of thinking about technical aspects, and Mads has shown me different ways of thinking about people and how they interact with languages.

Both are great communicators as well, which is very important to me – the ability to make your ideas understandable to others, both in prose and code, is massively undervalued in my view.

PHK:

I will have to pick a triumvirate: Dennis, Ken & Brian of UNIX fame.

Ken for the key UNIX ideas, Dennis for the ‘C’ programming language, Brian for clear thinking and writing about programming as a craft and all three for the Software Tools principle.

Dan:

Jordan Walke and Sebastian Markbåge of React are inspiring people to me. They are humble, open-minded, and compassionate.

Erik:

I have to say Bob Martin.

Early in my career, I spent a lot of time working on older techs, in “dark matter” situations, and preceding Q&A sites and ubiquitous user groups. I often thought, as I looked at huge reams of procedural code, “this can’t be the best way to do this.”

I then stumbled upon Bob’s work and was hooked. The ideas of clean code, readability, professionalism, etc.

I greedily read through everything I could find, and this really marked the beginning of my feeling that it was important to be plugged into the development community and never to be satisfied that the work you were doing was “good enough.”

2 thoughts on “Developer Q&A: Creativity, Productivity and the Future

Leave a Reply

Your email address will not be published. Required fields are marked *