Web App Software Development

Agile development is all the rage nowadays. Anyone who has worked in web development for any length of time has, undoubtedly, worked for a company that "does Agile". Now, agile is one thing if its kept within the development team, but when "business people", who don't know the first thing about developing software, start talking about "Agile", a development team can be in for a rough ride. What "being Agile" usually means to a business person is something along the lines of:

"We use an iterative, flexible process to position our development team to quickly respond to the needs of our client base in a fast-changing marketplace."

What they are really saying, through their actions, is:

"We don't have anyone in our business who remotely understands our product's business domain, nor do we have anyone who has the communication skills to talk with current and prospective clients to determine what their needs may be in advance. Rather, we are going to just sit around with our heads in the sand and tell you to start working on something and then change direction halfway through. Our customers are always right and, no matter how ridiculous or impractical their requests, we will always drop everything to accommodate them."

Now notice, I'm talking about when Business People say they want to use agile, not people whose primary duties are either working as a software engineer or in managing the engineering team directly. The problem here is that the business folks, good intentioned as they are, have hijacked a term that sounds appealing and re-purposed the phrase.

Agile is an engineering term, plain and simple. It can encourage good practice by keeping projects modularized and concise. It encourages breaking down large concepts and ideas into smaller chunks that can be iteratively fit into release cycles. Agile does not, however, mean that ideas like "market research" and "product vision" suddenly become passe. At its core, being Agile really just means that the engineering team will use sound development practices by making sure that the code is loosely coupled so that some things that may have been planned originally can be left out, if needed, and some small things, namely "usability bug" fixes, can be put in, without too much hassle. 

What Agile should not mean is that every few days some "idea man" businessperson pops his head into the development meeting and proposes a "new direction" for the project that the team will start work on "as soon as we get started on the next iteration". If this happens, its likely because the "idea man" is actually prototyping a product, not overseeing something that is being engineered to go into production and start being used by real users. 

The inability of some business people to understand the difference between prototyping and regular development is, what I believe, often leads to problems. Not surprisingly this tends to occur the most in small companies that have limited resources. There's always the non-technical startup founder who wants to be bringing in revenue today by finishing off the remaining bugs on the application that's "just about ready" to start a case study with customers, but also needs a little help in developing prototypes, since this "helps him think" and develop ideas for adding on to the product. 

The Occupy Wall Street (OWS) / Occupy Everywhere protests are refreshing and invigorating challenge to the current status quo. Technology has already changed the way most people do business and work, its about time that it be used to help change the way the government operates and expose the massive old-school corruption that has followed us all the way into the second decade of the 21st century. As someone who works as a web application software developer, I often wonder in what ways the internet could be used to further social causes, and during a time like now, when so many people are uniting to make their voices heard, it can be hard not to imagine what could be done to help. 

There are plenty of online collaboration tools that already exist to help the OWS protests and, undoubtedly, many of the protesters are already taking advantage of these tools to coordinate and communicate. I still feel, however, that there are voids that could be filled that would help further organize the messages behind the protests and aid in more permanently establishing resources to help in enacting long-lasting change. To this end, I don't think that the protesters need another application to help in mapping protest spots, organizing meetups, or, for that matter, in soliciting donations from well-wishers. The internet is already saturated with applications that can meet these needs.

While I am certainly no expert on the protests, the voids that I can see, from my perspective, are primarily related to media censorship and lack of a coherent, fact-supported, message. The issue of police brutality and corruption, as well as political and corporate corruption also seems to be inextricably linked to OWS. Based on this, I see potential for new web applications that approach the issues of exposing illegal and morally reprehensible activities by authorities, including the police and politicians, in a manner that is a) impossible for them to delete and erase (which is not the case with YouTube, for example, as they can pressure them into doing so), and b) will do so in a fact-based manner so as to, as much as possible, serve as a vehicle for evidence rather than an opionated editorial.

I'd rather save specifics for future discussions and posts, but I do see a lot of potential in integrating a web application with the use of torrents and peer-to-peer networks for hosting actual media content. For example, a web app could be used to track, and add comments about, a video that shows police illegally beating a protester, but the application would not actually host the video, rather its location would be tracked on peer-to-peer networks. The use of p2p could prove vital in thwarting efforts by corrupt authorities who attempt to remove evidence from the public domain.

Anyways, these are just my thoughts, at the moment, I'd love to hear from others who have different perspectives. If you see this and are interested please get in touch with me, I'd be happy to start linking to other relevant blog posts from here as well.

The Java Persistence API is a core API that forms the basic building block of different implementations of this API which include Hibernate and OpenJPA. These technologies are not necessarily ground-breaking as they are simply the latest in a long line of database abstraction technologies for the platform. Database abstraction is, in my opinion, something that any software developer who works with a database should learn to understand and utilize. Regular SQL, in its various forms, is not particularly well suited to be used from within an object-oriented context, as database normalization, which is generally a desirable trait in a database's architecture, means that pulling in a set of data usually means having to pull from rows in numerous tables. Using regular SQL, this is usually done using join queries or by sprinkling database calls throughout the code, as needed, to pull in bits and pieces of a record-set, as the code needs them.

To a developer who has any architectural sense or security knowledge, it quickly becomes apparent why the method of using straight SQL calls becomes either inefficient, dangerous, or both. While the issue of security deserves its own attention, suffice it to say that the use of regular SQL, within an application that is taking user input, creates potential security risks of SQL injection attacks and that the headache needed to make sure that they are prevented, itself, should be reason enough not to use regular SQL calls within a web application. From an architectural perspective, having to write extra code that converts results from SQL into an object-oriented data structure also creates headaches and takes away time from actually building the parts of the application that are specific to the business goals.

It is likely that if a team of developers begins trying to use regular SQL within an application, while addressing the issues above, that they would most likely begin building a layer of an application that would be attempting to resemble features already available within the Java Persistence API. Therefore, rather than re-invent the wheel, the decision to use the Java API should come naturally. JPA provides a simple, intuitive way to interact with a SQL database without having to code to the specific database, so that, for example, an application that starts out using MySQL could easily be changed to use PostgreSQL with only minor changes to configuration files. 

JPA allows database tables to be mapped to Java classes, with annotations being offered that help streamline the process. Rather than forcing a developer to concoct long-winded join queries, JPA essentially does this for you. All the developer has to do is define relationships, using annotations, within the table Java classes and you have the power of full data-sets at your fingertips. For example, you simply retrieve an object representing an entry from one database table, based on parameters passed in, and then you are able to call getter methods, where you would normally have foreign-key fields, to pull in the an actual object, or collection of objects, that are associated with the initial entry. After getting used to this, it can be hard to imagine ever going back to using straight SQL.

Beyond what is mentioned above, JPA has a number of useful features that are too complex to get into here in detail, including caching of queries and lazy initialization. Both of these features are meant to enable configuration to save on resource usage. Caching enables saving on calls to the database which reduces strain placed on the DB and increases application performance. Lazy initialization also saves on unnecessary queries, while preserving application RAM usage, by only retrieving an associated data-set object when it is called from within the application code.