Being a programmer in the corporate world is a somewhat arduous task. There are a lot of assumptions thrown around from within and without. Those outside the world of development tend to see development departments as something of a police-free zone, where anyone can do whatever they want without repercussions or having to adhere to deadlines. This couldn't be further from the truth.
Developers have a bit more leeway when it comes to technology, because they must. In order to do their job (you'll note this is unqualified- it's a requirement, not optional), developers must have such software as Firefox, Firebug, Eclipse, Fiddler, open internet connections, and the list goes on and on. Corporate IT can't restrict them (or shouldn't), because all it will do is slow down the development cycle and force deadlines to be pushed back even further
Deadlines come and go for development, it's true. This is the single biggest driver of all the churn/turn over in development groups. Miss a deadline, and one day you have 250 contractors in a massive project, and within a month, you could be down to a hundred. Or you could go from 30 down to 15 almost overnight, with two managers stepping down. Both of these have occurred in my time with corporations. It's a given, because software development is more of an art than a science. It doesn't adhere well to dates because understanding the "whole picture" in a project is something that very few people can do. Even when a person can, there are always unexpected hiccups and stop gap measures. Imagine a construction project suddenly having a supplier run out of their building material. This causes timelines to be pushed, and that doesn't make executives happy.
While the outside perception definitely hurts development on occasion, it's not the piece that strains developers the most. It's the perceptions from the inside.
During college or learning periods for developers, they get taught certain ways of seeing problems. Many of which never question these perceptions- They just start to feel as if there is only one right way to look at problems, or a very limited number of ways to solve a given problem. When that is the exact opposite of what companies are looking for. Innovation is about bringing creative and new ideas to the table. Trying new things that aren't simply extensions of the old stagnant ones.
In order to be clear about what I mean, here is an example: In a recent rewrite of an application, we went from a Java stack based on Struts 1 with some Hibernate thrown in, to another Java stack with Spring, iBatis and a DMS library. With the hopes that this would help speed up productivity and make our code more maintainable.
The core problem in our move from one version of the application to another is that we did not evaluate other languages/architectures that may be more suited to developing a web application. Groovy on Grails, Ruby on Rails, Python/Django, et al. Because developer preconceptions got in the way. This concept extends beyond just the language and patterns. The idea should extend in to some of the broken areas of application development where developers spend more time in XML files or property files than actual code. Spending more time writing unit tests or performing a build than writing code.
I've said it before, and I'll say it again: A good developer should be able to learn new languages and new patterns quickly and easily. If they can't, I would call in to question if they fully understand application development.
2011-11-01
2011-05-19
Quick and dirty intro to REST within HTTP
RESTful web design in terms of the HTTP protocol breaks down quite simply. It all boils down to how the URI is formed. Many systems handle this in different ways on the server, but it's simply about following the basic URI pattern:
(action) http://(host)/(resource(s))
Request Body: (description)
Action: Verb based CRUD, where GET is Retrieve, POST is create, PUT is update and DELETE is delete.
Resource: Noun based object. Usually found in value pairs (object/id) when not creating a new resource.
Description: POST and PUT have request bodies. This should describe details about what you want the object to have, or what you want to change.
Several examples of each type of action:
Here are a couple examples of nested objects, where a shelf has rows, and a row has books:
It becomes pretty darn obvious exactly what you're doing when you look at URIs in this manner. The retrieved representation should have some kind of default, but you also should be able to specify a particular representation (if it makes sense for your implementation). For example:
Common REST pitfalls:
While this is a very basic primer, if you follow these rules on both client and server, and you should fall successfully in to the pit of resource oriented architecture. Don't be afraid to just try it out. It's a new way of organizing all of your URIs, and you're bound to have to try things a few different ways before finding the most optimized pattern. Everything else in REST terms is just expanding on the concepts introduced here, either on server code or client code.
(action) http://(host)/(resource(s))
Request Body: (description)
Action: Verb based CRUD, where GET is Retrieve, POST is create, PUT is update and DELETE is delete.
Resource: Noun based object. Usually found in value pairs (object/id) when not creating a new resource.
Description: POST and PUT have request bodies. This should describe details about what you want the object to have, or what you want to change.
Several examples of each type of action:
- GET http://host.com/book/123abc789
- POST http://host.com/book
Request Body: {book: {name: "the super duper", contents: "once upon a midnight dreary..."}} - DELETE http://host.com/book/987bca123
- PUT http://host.com/book/123abc789
Request Body: {book: {name: "new super"}}
Here are a couple examples of nested objects, where a shelf has rows, and a row has books:
- GET http://host.com/shelf/23a/row/5/book/21
- DELETE http://host.com/shelf23a/row/4
It becomes pretty darn obvious exactly what you're doing when you look at URIs in this manner. The retrieved representation should have some kind of default, but you also should be able to specify a particular representation (if it makes sense for your implementation). For example:
- GET http://host.com/shelf/23a/row/5/book/21.pdf
- GET http://host.com/shelf/23a/row/5/book/21.html
Common REST pitfalls:
- REST does not support any kind of API listing, so it needs to be published clearly somewhere for people to use.
- You may not update more than one resource/object at a time, with one call to the server.
While this is a very basic primer, if you follow these rules on both client and server, and you should fall successfully in to the pit of resource oriented architecture. Don't be afraid to just try it out. It's a new way of organizing all of your URIs, and you're bound to have to try things a few different ways before finding the most optimized pattern. Everything else in REST terms is just expanding on the concepts introduced here, either on server code or client code.
2011-05-17
Scaling Web Applications (Aka: "Does Rails Scale?")
I have spent a lot of time talking about Ruby and Rails with people recently. When you're in that domain, you can't help but have people asking you, "So, is it true? Rails can't scale?"
It's a fair question, if somewhat naive. A lot of people have heard about the issues that Twitter has had as it has become the monolithic application that it is. It's been blamed squarely on Rails, even by some of Twitter's management. It's just not that simple. The kind of application that will service the amount of requests they are seeing is not the kind of application that happens on accident. However, I'm not going to attempt to answer whether or not Rails can scale, because I believe the question is fundamentally flawed. Instead, I want to talk about the concept of scalability, and why your question shouldn't be "Can X scale," but rather, "How does X scale?"
Building an application is a bit like constructing a building. You have to choose the right materials, tools and techniques. You have to plan in advance. You have to make trade offs about durability, flexibility, and all the other -ilities. Most web sites out there require very little scalability, because they'll never see more than a request every ten seconds. Some may get lucky and hit once a second. The very best, may see more! Consider that a millions hits per day is only approximately ten hits per second. That's really not all that impressive.
There are two types of scaling that are widely accepted. "Scaling up," and "Scaling out." Each have their pros and cons, but I feel it's important to define them before considering the greater picture.
"Scaling Up" refers to the applications ability to be placed to "Big Metal"- Think old time main frames. They are applications that are meant to have one instance of the application servicing every request. This is the easiest to conceptualize. You only need one program running, and as long as you buy powerful enough hardware, you can get away with any number of requests. There is a hard limit to this kind of system, though. When you aren't parallelizing tasks, you can end up with a lot of downfalls. Such as deadlocks, stalls, and more. What happens on a hardware failure? How do you plan for that, without having a second massively expensive install? You don't. Pure and simple. It's expensive. Very expensive. But it's simple to maintain.
"Scaling Out" refers to the applications ability to be massively parallel on any number of given systems. This could be commodity level systems, on out to high powered blades (or even mainframes). It's not about the hardware. It's about the software. Run it wherever you want, they'll all cluster together. This kind of scalability requires a lot of advanced planning, and forethought to be able to run twenty applications side by side, and have them buzz along happily. This tends to be why many applications need to be reworked when they get to the point where thousands of users are accessing them regularly. But if your application is set up correctly, you can grow with it, on demand. Just by bringing up a few new servers to service more requests. Scaling out tends to be the preferred method of modern scaling needs. You don't anticipate your need, you buy hardware as you need it. Backups are only as costly as having a few extra systems standing by.
Now, taking the earlier example: Instead of having to service a million requests per day, what happens when you have to service a hundred million? Or more? You're now looking at more than one thousand requests per second. The same system that can happily buzz along and handle one or two, or even ten, requests per second, will no longer be capable of handling the load. It will realistically be crushed under the weight of the load. Crushed. You didn't plan for it, it wont be capable of it. When you build a doghouse, you don't expect it to house hundreds of people, right?
That means that you need to think about how to handle that load. Build a foundation that can handle it- Pick tools and frameworks that you can vet.
Some key questions you really should be asking are- How many requests per second can your system service? Will they talk to each other? How? Are you persisting data? If you are, how many requests can your persistence tier handle? Can it scale out, too? How? Has someone else done what you're trying to do with the tools that you're using? At what scale? What pitfalls did they run in to? How can you avoid them?
The bottom line is... Don't fall in to the Sucks/Rocks dichotomy. Especially if you haven't fully evaluated what you're talking about.
Remember- Facebook is written in PHP, YouTube is written in Python, Twitter is written in Ruby, Amazon's systems are written in multiple languages, as are Google's. It's not about the language. It's about how you utilize it.
It's a fair question, if somewhat naive. A lot of people have heard about the issues that Twitter has had as it has become the monolithic application that it is. It's been blamed squarely on Rails, even by some of Twitter's management. It's just not that simple. The kind of application that will service the amount of requests they are seeing is not the kind of application that happens on accident. However, I'm not going to attempt to answer whether or not Rails can scale, because I believe the question is fundamentally flawed. Instead, I want to talk about the concept of scalability, and why your question shouldn't be "Can X scale," but rather, "How does X scale?"
Building an application is a bit like constructing a building. You have to choose the right materials, tools and techniques. You have to plan in advance. You have to make trade offs about durability, flexibility, and all the other -ilities. Most web sites out there require very little scalability, because they'll never see more than a request every ten seconds. Some may get lucky and hit once a second. The very best, may see more! Consider that a millions hits per day is only approximately ten hits per second. That's really not all that impressive.
There are two types of scaling that are widely accepted. "Scaling up," and "Scaling out." Each have their pros and cons, but I feel it's important to define them before considering the greater picture.
"Scaling Up" refers to the applications ability to be placed to "Big Metal"- Think old time main frames. They are applications that are meant to have one instance of the application servicing every request. This is the easiest to conceptualize. You only need one program running, and as long as you buy powerful enough hardware, you can get away with any number of requests. There is a hard limit to this kind of system, though. When you aren't parallelizing tasks, you can end up with a lot of downfalls. Such as deadlocks, stalls, and more. What happens on a hardware failure? How do you plan for that, without having a second massively expensive install? You don't. Pure and simple. It's expensive. Very expensive. But it's simple to maintain.
"Scaling Out" refers to the applications ability to be massively parallel on any number of given systems. This could be commodity level systems, on out to high powered blades (or even mainframes). It's not about the hardware. It's about the software. Run it wherever you want, they'll all cluster together. This kind of scalability requires a lot of advanced planning, and forethought to be able to run twenty applications side by side, and have them buzz along happily. This tends to be why many applications need to be reworked when they get to the point where thousands of users are accessing them regularly. But if your application is set up correctly, you can grow with it, on demand. Just by bringing up a few new servers to service more requests. Scaling out tends to be the preferred method of modern scaling needs. You don't anticipate your need, you buy hardware as you need it. Backups are only as costly as having a few extra systems standing by.
Now, taking the earlier example: Instead of having to service a million requests per day, what happens when you have to service a hundred million? Or more? You're now looking at more than one thousand requests per second. The same system that can happily buzz along and handle one or two, or even ten, requests per second, will no longer be capable of handling the load. It will realistically be crushed under the weight of the load. Crushed. You didn't plan for it, it wont be capable of it. When you build a doghouse, you don't expect it to house hundreds of people, right?
That means that you need to think about how to handle that load. Build a foundation that can handle it- Pick tools and frameworks that you can vet.
Some key questions you really should be asking are- How many requests per second can your system service? Will they talk to each other? How? Are you persisting data? If you are, how many requests can your persistence tier handle? Can it scale out, too? How? Has someone else done what you're trying to do with the tools that you're using? At what scale? What pitfalls did they run in to? How can you avoid them?
The bottom line is... Don't fall in to the Sucks/Rocks dichotomy. Especially if you haven't fully evaluated what you're talking about.
Remember- Facebook is written in PHP, YouTube is written in Python, Twitter is written in Ruby, Amazon's systems are written in multiple languages, as are Google's. It's not about the language. It's about how you utilize it.
2011-05-06
Parkinson's Law
It's pretty rare that I come across a work saying that I feel compelled to share with people, but Parkinson's Law is a very important law to keep in mind in software development. What is it?
This has been said in many different ways, in many different professions, or even in nature. Those include...
"Work expands so as to fill the time available for its completion."
This has been said in many different ways, in many different professions, or even in nature. Those include...
"Data expands to fill the space available for storage."
"Storage requirements will increase to meet storage capacity."
"Nature abhors a vacuum."
Why does this matter? One simple reason. Programmers, and all other IT people, tend to aim for the stars, and will happily spend from now until the end of eternity designing the absolutely perfect, beautiful system. Instead of shipping the product.
In the end, real developers ship.
In the end, real developers ship.
Thinking in Code: JavaScript and Java
JavaScript is not Java.
Let me repeat that again: JavaScript is to Java what Hamster is to Ham.
There is no direct correlation, other than the fact that they both are a form of programming language, and a way to instruct a computing environment to perform specific tasks. Their paradigms are completely different, and they are not formed of the same types of features. Let me highlight some of the key features of each language:
JavaScript is...
Why Does it Matter?
I have run across an awful lot of JavaScript code that was clearly written by a Java developer. Over architected and excessively complex code which takes hundreds of times the CPU cycles than ten lines of well written, clear, easy to use, self documenting code. This highlights the differences between the two languages, and the two different thought processes between the two. The correct way to implement a particular feature in one programming language may not only be wrong in another language, but may go against the very intention and fabric of the language. This is why companies hire programmers to participate in the development of a particular application based on language, rather than just hiring the best generalist that they can find. Not that generalists don't have their place, by all means they do. But in order to be well grounded in many languages, it not only takes passion for coding, but motivation and drive. Despite what you may believe, you're probably not one. They are few and far between, and when found, they are worth their weight in gold.
Let me repeat that again: JavaScript is to Java what Hamster is to Ham.
There is no direct correlation, other than the fact that they both are a form of programming language, and a way to instruct a computing environment to perform specific tasks. Their paradigms are completely different, and they are not formed of the same types of features. Let me highlight some of the key features of each language:
JavaScript is...
- a functional language with basic facilities for forcing object oriented notation, despite not being object oriented at all.
- a dynamic, loosely typed language that is event driven
- typically used in the web browser, though there are server side implementations that are catching on.
- a mostly object oriented language
- a static, strongly typed language
- typically used for implementing very powerful server based processes which are cross platform compatible
Why Does it Matter?
I have run across an awful lot of JavaScript code that was clearly written by a Java developer. Over architected and excessively complex code which takes hundreds of times the CPU cycles than ten lines of well written, clear, easy to use, self documenting code. This highlights the differences between the two languages, and the two different thought processes between the two. The correct way to implement a particular feature in one programming language may not only be wrong in another language, but may go against the very intention and fabric of the language. This is why companies hire programmers to participate in the development of a particular application based on language, rather than just hiring the best generalist that they can find. Not that generalists don't have their place, by all means they do. But in order to be well grounded in many languages, it not only takes passion for coding, but motivation and drive. Despite what you may believe, you're probably not one. They are few and far between, and when found, they are worth their weight in gold.
2010-10-17
Ruby on Rails and Sencha's ExtJS brings serious productivity to web application development.
As someone who has been living in the Java world when it comes to developing web applications, development times have always been ridiculously long. Cycles spent figuring out what should take hours, not weeks. I recently started playing with two major technologies that really enhance the web sphere. Both of which are squarely aimed at taking application development time, and dropping the curve as sharply as possible.
Ruby on Rails
Ruby is a modern programming language that is about economy of code, and productivity. It is fully featured, and a pleasure to code in due to it's pure object oriented nature. This also contributes to how easy it is to write DSLs in. Ruby could be considered a hybrid between Smalltalk, Lisp and Perl. Due to how many areas it pulls from, the language itself can take a little while to wrap your head around. It's very different than the "big" languages out there (C#, Java, PHP). I plan on writing a much longer post about Ruby specifically, but here are a couple highlights:
Probably the biggest distinguishing factor of Ruby is it's Gems. Perl started going down this route with CPAN, but Ruby took it all the way to its logical conclusion with Ruby Gems. Gems can be libraries, DSLs or executables. It could be something you use in your code, or a command you run to push code to a server. The biggest difference from the 'big' languages out there in this realm is that they can be installed with one simple command on the command link, unlike other library dependency managements systems that require you to either hunt down the dependencies, or download the internet every time you build. For example, pulling in the "rails" framework is as simple as: "gem install rails" and waiting a few minutes.
The other big highlight of Ruby that I'll make is the Rails framework. It's one of the most amazingly productive frameworks for designing your web application, and despite rumors, scales perfectly well. Assuming you know how to write scalable code.There are a ton of great tutorials out there on how to get started with Rails, so I don't ever really plan to cover Rails in any more detail (unless there are specifics I decide are worth covering). Suffice it to say, you could have a running application in about five commands on the command line. Which includes full database backing (with a database of your choice), basic unit tests for generated models and controllers, with proper separation of concerns between layers.
Sencha's ExtJS Framework
ExtJS is a JavaScript framework that makes building fully featured user interfaces in web browsers a fast and nearly painless process. When used creatively, it can make fully customized web applications in much less time than it would take to try and cobble together all the pieces with various javascript libraries, CSS coding and HTML, all in a standards compliant way for all the modern web browsers out there.
There are plenty of tutorials out there for both, so I wont belabor those subjects. But all in all? At this moment in time, I'm thinking that Ruby on Rails partnered with ExtJS is a match made in development heaven.
2010-05-11
On whitespace and other coding "monstrocities"...!
Why exactly is it that programmers all seem to be anal retentive?
Some of the best coders that I have been witness to have been extremely reflexitively irritated by certain things that do not fit their view of 'good coding practices'. Most recently, I was reading archives of coding horror when I came across this post, which can be pretty much summed up as 'OMG WTF dem white spaze not l33T nuff!' While I definitely respect Jeff's work (okay, most of it anyways!), and feel that he's spot on most of the time (really!), I wish that people would stop propogating the misconception that you have to be anal retentive to be a good programmer.
Having worked in mid-to-large sized teams for most of my development career, I can safely say that everyone will run up against different coding practices, and different ways that people believe things should be done. Take this for example:
public String doStuff() {
return "Because I said so!";
}
Can become..
public String doStuff()
{
return "Because I said so!";
}
Or..
private String EVERYONE_LOVES_CONSTANTS = "Because I said so!";
public String doStuff() {
return EVERYONE_LOVES_CONSTANTS;
}
...and there are plenty of other permutations!...
At most shops there is someone that sets standards, and usually gets pretty upset if they aren't followed. Really, it doesn't matter! Honest! They're all usable. All can be self-documenting. Conventions are helpful, but that's all they are. Conventions. If you can't read code you haven't programmed (particularly with use of a decent code auto-formatter), then you have bigger problems to solve! All I have to say is this: If you want my code to have extra CR/LFs, I darn well better have a huge monitor to deal with a heck of a lot of whitespace on my screen at a time.
Yes, most good programmer are pretty OCD about a lot of things. Going 'over the edge' about little things like that only leads down the path of madness, especially if you wander through many different shops in the corporate world.
Some of the best coders that I have been witness to have been extremely reflexitively irritated by certain things that do not fit their view of 'good coding practices'. Most recently, I was reading archives of coding horror when I came across this post, which can be pretty much summed up as 'OMG WTF dem white spaze not l33T nuff!' While I definitely respect Jeff's work (okay, most of it anyways!), and feel that he's spot on most of the time (really!), I wish that people would stop propogating the misconception that you have to be anal retentive to be a good programmer.
Having worked in mid-to-large sized teams for most of my development career, I can safely say that everyone will run up against different coding practices, and different ways that people believe things should be done. Take this for example:
public String doStuff() {
return "Because I said so!";
}
Can become..
public String doStuff()
{
return "Because I said so!";
}
Or..
private String EVERYONE_LOVES_CONSTANTS = "Because I said so!";
public String doStuff() {
return EVERYONE_LOVES_CONSTANTS;
}
...and there are plenty of other permutations!...
At most shops there is someone that sets standards, and usually gets pretty upset if they aren't followed. Really, it doesn't matter! Honest! They're all usable. All can be self-documenting. Conventions are helpful, but that's all they are. Conventions. If you can't read code you haven't programmed (particularly with use of a decent code auto-formatter), then you have bigger problems to solve! All I have to say is this: If you want my code to have extra CR/LFs, I darn well better have a huge monitor to deal with a heck of a lot of whitespace on my screen at a time.
Yes, most good programmer are pretty OCD about a lot of things. Going 'over the edge' about little things like that only leads down the path of madness, especially if you wander through many different shops in the corporate world.
Subscribe to:
Posts (Atom)