Survival of the Fastest: Software Development

Today instead of talking about code and my previous projects I would like to talk about something really important for all of us coders/programmers and any one even remotely associated with it. Today's topic for sermon is "Speedy Software Development".

I know that the best of you, readers (which I hope is most of you readers), know about Agile Software Development Methodology. (Don't sweat! If you don't know, then look up the "Further Reading" resources links at the bottom of this post). It's an innovative way of developing software in the fastest means possible. But, what if I say some (additional) simple techniques could yield a faster method? Some of you might be aware of the techniques and tricks I'll be discussing today and none of what I'm writing here is novel.

So, let's cut to the chase;

Open-Source

You are not alone (in your problems). Someone somewhere sometime has already tackled with your requirements and problems. All you have to do is embrace his legacy and yes, credit him for his contribution. Most of all the major problems we face has been presented in an optimized and professional open source package. Companies and programmers, alike, must learn to leverage it for rapid software development. You don't want to reinvent the wheel. Do you?

For the n00b:

A lot of new programmers and students overlook this useful resource and later, over the years, they gain the wisdom of using Open Source Technology. So, it's my best advice to anybody new to the coding community to immerse himself or herself into Open Source Projects. I'm not saying you need to contribute to it. I'm saying to learn and use it in whatever project you want to do. Reading and studying Open-Source code will also make you a better programmer. You'll often stumble into some new repository or API which will showcase you great new features which you had never dreamed/thought of before or previously considered impossible owing to your naivety in the subject.

For the PRO:

For the professional it is of utmost importance to keep up with the latest developments in his/her field of expertise. Open Source software and APIs can also use some polished distilled wisdom for upgrades and new features which seasoned professionals are replete with. This will also help team leaders, software architects and researchers to decide, plan as well as utilize the cutting edge of current times for the best delivered projects and solutions.

For the Company!!

Now, comes the important part. Companies worldwide can help in the Open Source Initiative by kind or cash. They can help by allowing employees to work on certain OS projects when they're free or assign employees (without any current tasks/projects) tasks to either contribute or use OS to build new products for the company or even the world at large.

Also, when companies devote teams and departments to Open Source Initiatives they can seek funding and donations from other companies for such projects and gain a small share of profits over it or at least balance the overhead expenses to make it a non-profit. Giving burned out or overworked/pressed employees the option to switch to a deadline free OS projects will recuperate their coding spirit and mind, while staying on the payroll and not costing much capital for the company (considering its funded enough with OS donations).

Automation, Code Generation and Company Project Repository

Ever found your task to be repetitive and boring? Does you work feel like it should be done by a robot? Well, then build one (or use one)! What I'm most surprised at is that even experienced programmers and experts get into this kind of menial chores. They either copy previous source code or unintentionally repeat large chunks of project components already developed earlier by someone else in the same company for some other client.

The Archive

 In the era of paper dominant office days and even now, a lot of offices kept archives of projects and documents for future reference. This is most useful to tackle repetitive tasks like building a web-service, a database driven desktop application etc. Over the days, months and years an organization will continue to accumulate a lot of achievements and accomplished projects under it's belt. This when well preserved and circulated among the coders as a library of information/samples will be a boon for rapid software development. 

Question: What is common between an application which maintains pharmacy inventory and a grocery inventory application? 

Ans: A lot! There is a lot of similarity in database schema, application logic and even business usage. Then why don't we just re-purpose the previous grocery store app to handle our new pharmaceutical demands? This will save you a lot of time to work on the core features of this new pharmacy inventory system. You may want to add a small feature that will notify alternative drugs or harmful drug combinations which may have been prescribed incorrectly by the doctor.

Thus, every IT or software development company must keep a unified distributed project repository which can be accessed by an employee in any of it's branches worldwide with a click of a button. Think of it? How simple will our tasks be? But then there are certain challenges to this approach:

1. Bad Code (style, approach, architecture, performance etc). Almost any product that can run is shipped out the door to customers or clients. It may happen to have some small nifty hack or jury rigging and duct tape under the hood to keep it working. Such a product is volatile and may break upon alteration or up-gradation.
   
   Fix: A set of informal personal log should be provided to programmers/employees working on the project to document their contributions, potential problems, hacks applied and jury-rigging done with the project while they are still working on it. This log should be kept together with source code and documentation for better reference in the future.
2. Obsolete Code primarily designed in archaic languages for ancient systems may clog up the archives with no further re-usability.
   
   Fix: Such projects should be carefully weeded out into another obsolete or dead code "cemetery repository". This may also require a special archivist with the dedicated duty of studying code, maintaining the archives and guiding other programmers with useful information.The archivist can be anybody from rookies to experts or even dedicated professional (will probably require a team due to the varying age of archived projects and diverse languages applied to it).
   
   Archived projects can also be made to reorganize themselves by timed automation.(For example a Project developed in a new popular language may be timed for a decade or two while perennial projects in C, Java etc can be set for five decades or so which may be extended upon inquiry from the archivist. I hope you too believe that the JVM will not be obsolete anytime soon nor will C.) Such obsolete projects may even be distributed as open source for education and experimentation purposes.
3. Dirty Code is source code which is well, dirty. You know what I mean. The same old, lack of comments when needed, incorrect design, ambiguous functions and suspicious noise code (Don't know what that small method does? Not there in the documentation? Not mentioned in the programmer logs? Could be a logic bomb? Or, am I being paranoid?). Dirty code is unsafe to work with and providing global access to source code for any of your programmer anywhere on the planet with sufficient internet bandwidth can cause such dirt to accumulate.
   
   Fix: A proper access logging mechanism is required to observe who accessed the archives, when was the archive accessed, what was accessed in it and what was done with it? Also, a rule of forking existing repositories must be enforced to preserve original source code (which can be easily handled by any version control system worth its salt). Also, new recruits can be assigned to study and observe for such anomalies while cleaning such legacy systems.
4. Cheat Code? Yes a repository or source code that is stolen from your archives under corporate espionage.
   Fix: Already discussed measures of access logging and version control systems which handle such problems.

 Wisdom of Wizards and Templates

The repetitive work can be easily identified with the help of a proper archiving system and such projects and problems should be streamlined with the help of incorporating Wizards and creating Templates or Stub generating programs. Also known as Code Generators. Such tools can accelerate software development exponentially. Want to build a social networking site every time? This might be your tool. But, careful thought should be made before applying such automation, as design flaws or hard coded errors could creep into new projects and will be difficult to track. Again, one must strive to abstract the mind work from the menial chore carefully. Custom requirements may require you to hand craft such solutions, sometimes, even from scratch. Don't be tempted to use this as a silver bullet.

Code Generators can also be scripts to code all the lines of the project as per observed design or pattern. Such, custom tools are generally one-off and built by the best programmers (who will also be of the laziest programmers - quoting Bill Gates).

Extended Archives

Companies can extend their archives to other companies as read-only or reusable resources. This can be done by levying some pay per view or annual rental/subscription charges. As the archive is categorized and with proper access rights and restrictions in place, will hopefully keep sensitive projects off limits. This can also help colleges and universities in exposing students to existing industrial endeavors.

Colleges can too build such repositories for student and teacher projects, examples, notes, resources etc. This can help in learning rapidly from existing source code and extending them further with new features.

Colleges may even extend such archives with collaborating with other colleges and industries. (Hey, you may even want to shift to GitHub.But, if certain features have to be specialized for students then good go build such repositories and archives.)

[Unfortunately in India almost every school and college dumps their students assignments and projects without a moments thought. While some teachers do retain some good projects but, is that enough?]

Points of Importance

Some points to keep in mind are;

• Software should be developed like component assembly, where each set of features is provided by a certain package or component of code which can be either purchased (in case of Licensed), copied (in case of Open Source) or custom built. This makes extension, maintenance and reuse very efficient.
• Programmer should learn and use clean coding style and follow proper coding conventions in order to keep their code legible and useful for others.
• Style and standards can be enforced by companies to keep it easy for projects to be legible to other programmers outside the original developer team.
• We should pause to think about our work. If its iterative without any requirement of creativity or customization then a proper approach should be taken to automate or simplify the task.
• We should read more and research more while in the planing and design phase in order to optimize our approach and speed up the actual software coding process.
• You may thank me for this blog post. I'll be super happy. ; ) If you have any other cool tips add them in the comments.

Resources for Further Reading:

Agile Software Development Methodology 

• Wikipedia (It's your next best friend after Google, of course.) 
  http://en.wikipedia.org/wiki/Agile_software_development
• The Agile Manifesto
  http://agilemanifesto.org/ 
• Agile Books
  http://www.softwaretestingclass.com/top-20-best-agile-development-books/ 

Must read books and inspiration (as well as resource) for this blog post

•  The Developer's Code: What Real Programmers Do. By Ka Wai Cheung.
• New Programmer’s Survival Manual. Navigate Your Workplace, Cube Farm, or Start up. By Josh Carter?

Project Souce Code sharing repositories and other version control systems

• Git Official Site
  http://git-scm.com/
• Git on Wikipedia
  http://en.wikipedia.org/wiki/Git_%28software%29
• GitHub Official Site.
  https://github.com/
• GitHub on Wikipedia.
  http://en.wikipedia.org/wiki/GitHub

Experiments with The Epic Unreal Engine

I had played around with UDK for my minor project in Computer Science Engineering undergraduate course and had come into a new world of it's own. UDK or Unreal Development Kit is a professional game development kit built upon the Unreal (Game) Engine by Epic. It's a proprietary product but UDK is free for noncommercial use.
It was my first time playing around with a 3D graphics software and it took me a while to master it's various view-ports and other 3D graphics manipulations. It is very powerful tool for game development truly built for the "serious" gamer. Though I had to scrap the project due to it's multiple skill requirements in 3D Graphics (Game Level Design, Asset Creation), Story Boarding, Cinematography and Programming (If you're creating something truly original) and my small team.

A lot of core features are locked down into the source-code but, trust me, to build blockbuster games you would not require to peek under the hood. Well, as I've said before, until you're truly doing something not directly intended by the UDK or you're a professional game developer.

The programming can be done with UnrealScript (an object oriented language tied up with the Unreal Engine) while you may want to use C++ for programming in a licensed version. Another revolutionary alternative to programming game behaviors is to use Kismet, a visual scripting engine. Kismet is the prime tool used by new comers and graphic artists and game level designers to implement simple functionality like opening of a door or build a chain of events to perform sophisticated actions.

Another tool which must be used in parallel to the Unreal Development Kit is UnCodeX (A Source Code Browser for Unreal Engine libraries). It will be very useful to learn and experiment with the source-code unreal scripts to hack new cool in game features.

A lot of simple predefined source code and tools are provided to rapidly build a first person shooter while other genres may require to get under the hood or perform hacks. Hence, I built a simple FPS which showcases more of my Game Level Designing rather than coding prowess or any other cinematography skills.

Here's a list of Resources you can use to start tinkering with UDK:

Official Site-

http://www.unrealengine.com/ 

http://www.unrealengine.com/

Required Books-

Unreal Development Kit Game Programming with UnrealScript: Beginner's Guide

http://www.packtpub.com/unreal-development-kit-game-programming-with-unrealscript-beginners-guide/book

Mastering Unreal Technology: The Art of Level Design

http://www.amazon.com/Mastering-Unreal-Technology-Level-Design/dp/0672326922

You can get major help, tutorials on its excellent network and web communities. There are plenty of books too for your aid. Good Luck!

Here's my last major work with UDK;

https://docs.google.com/file/d/0B4e1TZA7mwrNLUIyRXVOQ2hXOUE/edit?usp=sharing

Note: You'll need the UDK to open the attached Map. I'm a little lazy and inept in using UDK, hence, no cooked files or executable.

Here are a few screen shots (Looks pretty good, eh?):

Disclaimer: The contents/Assets were prepackaged into the UDK. I did not import custom assets to the map.

Now, I "Git" it.

Recently, I picked up two of “The Pragmatic Programmers” books, “New Programmer’s Survival Manual” by Josh Carter and “The Developer’s Code” by Ka Wai Cheung, and started skimming through them. A lot of the ideas, suggestions and lessons there was an eye opener for me. Barely 30-40 pages into both texts I was astonished to see how many vital skills and knowledge goes untaught in colleges (especially here in India).  No wonder, we are a cheap labor market for the lowest intellectual level of IT industry, which is in self-delusion of working on the next big thing, while we only do menial labor of code maintenance or simple web-design and the world is rapidly switching to mobile/cloud computing and augmented-reality.

One tool that I came to learn of is the Version Control System. Such a system is used to carefully keep all source-files and dependencies intact over the development cycle of the project. It logs all changes and can be used to “rollback” to previous development stages or versions, if required. It can also be used to “fork” a repository (the project directory containing everything relating to your project) which is another word for copying/cloning your project repository for sensitive changes or features that may otherwise break your previous tested and working code. This way you can test new code and features on existing project without ever messing up your original source-code.

I started learning Git and it was pretty simple and easy to learn, for me at least, as I’ve dabbled with some Bash Terminals on Ubuntu (Linux) and command line terminals don’t intimidate me as much as they used to. I had my humble beginning from Microsoft Visual Studio with Visual Basic 6.0.  Microsoft Visual Studio with its RAD (Rapid Application Development) IDE may have been easy to learn swiftly with simple drag and drop interface, auto-completion and other features. Actually, now, after a few minutes of recalling, I can say that I had the fortunate opportunity of playing with QBASIC and C, in my early school days.

But, here is a note of caution: RAD is not for a true programmer. Visual Studio may have it’s advantages but when something breaks (I am not talking about Bugs), you’ll have to return to the drawing board. Beware! You’ll become complacent, delusional and xenophobic. Trust me. I’ve been there. It is best to learn your craft with the smallest of tools no matter how crude or cumbersome they may seem to you. Once, you’ve built a full fledged “personal” project with them, congratulations, you’ve graduated! From “Personal Project” I mean, a problem or idea that you found independent of books/exercises and implemented it after your fair share of research and development. Then you can switch to whichever tool you want to use and use them.

Another disadvantage of RAD and IDEs are that it’ll never show you the internal system level implementations. How your files are stored or the system/hardware specific dependencies used by your project unbeknownst to you and when you migrate your project or deploy it somewhere else or on the web….Oh boy what a ruckus!

RAD is useful for simple and quick development of applications and systems to test your idea/product and prototype business models linked with that product. They can then be hacked to improve and converted to independent architecture/platforms like Java etc later on (only when you know what’s under the hood). I personally have never witnessed high performance or high scalable systems built with MS Studio. No, sir, I have not. Look around everywhere. They are either built on Open Source and/or custom built.

Returning to Git. Git was developed by Linus Torvalds and his Open Source buddies. I wont detail these unnecessary information that can be wikied. But, I’d like to add that Git has the additional advantage of being distributed and multiuser. It may have other features but, I won’t get  into its details.

Now, people, Let’s get our hands dirty!

Setup::

First, we’ll need to install Git on our systems as only a lucky few of you all would be fortunate enough to have it pre-installed on your system. If you’re running a flavor of Linux you may have it installed somewhere in there. Whatever, here’s the link to it’s setup and official site:

http://gitscm.com/

Tutorial::

Now that you’ve got the link to the Official Site. I presume you’d prefer the official guides rather than mine, so I’ll skip the hard part of hand crafting a tutorial and directly link you to a tutorial that I found very useful (especially for the Terminal literate):

http://net.tutsplus.com/tutorials/other/easy-version-control-with-git/

No need to be intimidated by this command line tool, it’s available in a GUI friendly version, too. Still, some of you might want to learn the Terminal first and then start using Git, if you’ve got no prior experience with Command Line Terminals. Take another pearl of wisdom, Try the Terminal available with Linux and/or Mac OS X, not Windows. The Windows Command Prompt was never meant to be used as a powerful tool as Windows was originally constructed for the technologically impaired ordinary gentry and many developer tools were scorned upon.

If you’ve returned to this page after getting “Git”. Thank you very much. You didn’t abandon me while I was ranting on and on. A comment or a token of appreciation would work wonders for my aching back that strained, against this uncomfortable sofa, in order to illuminate you with such good ideas.

You must now try GitHub. I’ll leave the requisite research on your dependable Google search skills.

Related articles

• Source Control – Git (leyalisme.wordpress.com)
• Git for Visual Studio and .NET developers (agile.dzone.com)
• Getting Started with Git (codecademy.com)

Javadoc: The Bible of Your Java Source Code

Today I'll be sermonizing on Java, my dear readers. I have observed since I began learning Java in my third semester of Computer Science Engineering undergraduate course that comments are a vital part of a program or rather the source code. But, never were we taught about good commenting standards and commenting conventions. Also, our exposure to the Javadocs was minimal. Our only interaction with the Javadoc was when the IDE (Netbeans or Eclipse or any other) would produce a small popup window with a summary about the keyword over which our mouse pointer was hovering.

It fills me with sadness that such vital skills of reading, navigating, understanding and producing Javadocs was never taught or even appreciated. Like some unspoken truth, we all kept mum and so did our teachers, that we’ll eventually learn it “on the job” or “later in life”. I seriously doubt about the many Indian, so called, software professionals knowledge on how to comment code correctly and generate usable Javadocs or read other programmer’s Javadocs. Any Java programmer worth his salt must have this essential skill of playing around with the Javadocs.

There are various commenting styles in the art of programming each style has a certain aim in mind. Some of the commenting styles can be roughly categorized as:

• Explanatory/Verbose Commenting – Used by experts, teachers and trainers to explain in detail what each line of the code does. It is generally regarded as “How?” themed commenting. It is also used when a complicated piece of algorithm is implemented which cannot be understood without certain explanatory aid. This style is generally picked up by students from their teachers, but, is not good for day to day professional use. Most of what any programmer will code in life will be simple and easy to understand without any comments. The best code is self-explanatory without necessitating a single line of code.

• Reasoning/Terse Commenting – Used by professionals in day to day coding, it involves adding comments only when a certain part of the code is not self-explanatory and departs from the ordinary trend of coding. For example the forking of a certain thread to handle background tasks etc. It is generally regarded as “Why?” themed commenting. In other words it is a minimalist approach to commenting. You comment only when it is most required.

• Documentation Commenting – Used to generate documents or guide other programmers using your source-code either as an API or a Legacy code. It may detail hardware and software dependencies, performance issues, code vulnerability (Yeah, you’ll find in some legacy codes useless warning comments such as, “//DON’T TOUCH THIS CODE! EVERYTHING WILL BREAK. TALK TO BOB BEFORE EDITING THIS CLASS”, where Bob is someone who has left the company 15 years ago. Good luck with that, mate.) This commenting style is best seen in Open Source APIs, where you can seek help from Open Source communities which may take a longer time to resolve your issues and even third party licenced APIs, though they provide a paid support service to resolve your issues in the least time possible.

Note: No adherence to a single style is requisite. Often, a mix of all styles will be required to deliver a useable and malleable source-code.

Javadocs, ahoy!!

 

Javadoc is a documentation generator from Oracle Corporation for generating API documentation in HTML format from Java source code. (Yes. This line is ripped off from Wikipedia. Now, please, don’t label me as a plagiarist.)

Javadoc handles only a specific format of commenting within the Java source codes. It can be either generated by using the pre-bundled javadoc tool with the JDK or invoking it via any Java compatible IDE (I’ll be sticking with Eclipse in this blog. Go to hell, Netbeans! *Just joking! I know how many of you still depend on it and I can empathize with you.*).

I’ve attached the source-code of a simple java program (which I did not care to test run even once. Don’t bother compiling and running it. Our focus should be on commenting for Javadoc.) with screen shots of generating and using the related Javadocs of this project. [The screen-shots are available here only, as my paid internet data usage quota has expired and I am currently blogging at the speed of 3~4 Kbps. I promise to update the bundled project with screen shots after I get my internet connection recharged which is looking doubtful considering I have already paid for the recharge and the confirmation SMS or transaction SMS has not yet been received since today morning 8 AM or so.]

Now, let’s see. Step 1:

Select the Element you want to comment on. Like in the above screen grab, I’ve selected method Circle from class FigureStore.

 

Step 2:

Select the Generate Element Comment option from the drop down menu of Source button from the main menu.

Step 3:

Eclipse auto-generates the comments depending on the selected element. As here the selected method Circle has two parameters, the Radius as X and the Diameter as Y.

Step 4:

Now, edit the element comment as required. Try adding details as briefly as possible.

Step 5:

After you’ve added all the element comments you need to generate the Javadoc. For generating the Javadoc for your project, select it in the package explorer and the choose Generate Javadoc from the Project option in the main menu.

Step 6:

Now, you’ll see a dialog as shown in the above image. Make sure your project is selected as well as all of its components. The Javadoc tool is selected via the Configure button. The Javadoc tool is present in the “bin” directory of the JDK. Also, set the path of your doclet to a new “doc” directory of your project.

Step 7:

You can add a specific document title to your doclet and you may add any referenced/dependent jars and packages which mainly comprise of the JDK libraries. Then select Finish. (Yes, I don’t go any further with that alluring “Next” button.)

Step 8:

Now, sit back and relax for a few moments while the Javadoc is being generated. The time take for Javadoc generation is dependent on the number of Elemental Comments, program size, structure etc. Your Console should show something like this:

 

Alternate Method:

Here’s an alternate method to generate Javadocs. Right click on your project in the Package Explorer. Select “Export” from the drop down menu and a dialog will appear as shown below.

Select Javadoc from the Java directory and Bam! It’s done (It’ll take you to the Javadoc Generaton Dialog). 

What to expect from Javadocs?

You’ll see a new directory under your Project in your Package Explorer called “doc”. Exanded it’ll look like this;

The “index” html file is the start or main page of your Javadoc. Double click on it to open it inside Eclipse. A sample Javadoc is shown below;

 

Also, now you’ll be able to invoke small popup help windows for your project/package which can be used in other projects as well if the Javadoc is imported in there Project directories.

 

IMPORTANT END NOTE:

A good habit is to keep on making Element Comments while programming in order to keep the burden of commenting low. Once the size of the source code gets out of hand commenting becomes difficult and you’ll not be able to leverage the Javadoc prompts for other parts of your source-code.

“Also, one must always try to keep deprecated (i.e discontinued) methods in future iterations and newer versions of the program/package in order to maintain backward compatibility.”

Sample Documents and Source Codes::

https://drive.google.com/folderview?id=0B4e1TZA7mwrNWnFRMmxmS05pc3M&usp=sharing