Continuous Integration Systems are one of the most important tools for agile software development. They automate the process of building and testing. A lot of people seem to have realized their importance and there are quite a few products in this arena. I already used Hudson and CruiseControl, but for OpenMRS we need to find one which is best suited to our needs. So I started out about 3 weeks back to create this roundup of continuous integration servers. This should be a useful roundup for any project with similar requirements.

Why does OpenMRS need a Continuous Integration System?

Any software development effort needs to take care that regression doesn’t happen with new code changes. Often a change in the API/module core results in breaking of modules dependent on an earlier method. A Continuous Integration System will rebuild OpenMRS after a change is committed and provide information on how that change is affecting related code.

OpenMRS would also benefit from an easy to understand UI that Continuous Integration Systems provide for number of failing tests. The number of passing or failing Unit Tests will indicate the quality of a build and help implementers/testers realize the stability of a build. We can also set some goals on how many test methods we need to write before an API method can be finalized or deprecated.

The work done in different branches and modules can be monitored and looked at easily by the community.

Building of OpenMRS Installer using NBI can be automated and new users can directly test with the latest build of OpenMRS using the cross-platform installer.

Thus to summarize, a continuous integration system will bring better release quality, more transparency, quicker bug finding and fixing, simplicity and TDD frame-of-mind.

Disadvantages for OpenMRS in using Continuous Integration System

• Additional Load on Servers
• Not every developer is motivated to write unit tests ;-)

Features that OpenMRS needs (Not exhaustive)

1. Easy to monitor tests and easy to understand dashboard
2. Support for SVN and Ant
3. Dependency integration
4. Email/RSS/IRC notifications when a build fails or bad code is committed
5. Warning flags when a committed code doesn’t follow coding rules (naming, newline format,etc.)
6. Allow code committers to modify build and test parameters from the GUI
7. Optimal Performance
8. Price & Open-source development

Comparison of Continuous Integration Systems:

Software Distribution is an essential part of Software Development and can sometimes be the first impression that can make or break the user's opinion about a software. We, as software programmers forget the importance of easy distribution and easy installation of software that we develop. We do not understand the problems that a new computer user or a non-programmer may face. And I experienced this first hand about 2 weeks back, when a physician friend of mine heard that I was working on OpenMRS.

I was lucky enough to work on OpenMRS this summer and learnt a lot more about Medical Informatics during this period than I expected. Hearing this, my friend wanted to install OpenMRS at his clinic which already used Tally (hehe... isn't that innovative??) for storing patient records, observations and prescriptions. He practices at Kolkata, visits different hospitals and sometimes the patients he attended at a hospital come to his clinic. When I told him that OpenMRS was a webapp, he got all excited and I narrated him all the features that OpenMRS could provide and help him manage his patients better through the web, only if he could host OpenMRS from his clinic. I'll skip the other interesting parts and his extra-terrestrial expressions ;-), since we are actually talking about software distribution.

So then came the day when I was about to leave office and he was in his clinic trying to install OpenMRS. It was Independence Day and the clinic was closed but he was excited to experience the new-age medical informatics :-)) When I first got his call he had downloaded the Windows Installer. I was pretty sure it was for an older version and hence told him to instead download the OpenMRS Appliance, which is a VM Image that can be run from one of the virtual machine softwares. Yaw Anokwa made this wonderful Virtual Image with Ubuntu + All Necessary Stuff (tomcat, mysql, demo data) and OpenMRS running. You just have to have VirtualBox or VMPlayer or VmWare Workstation and load the VMimage and wait for Ubuntu to start. It is simple, fast and safe to play with... But for novice users, I just realized it wasn't easy enough. My friend installed VirtualBox and loaded the image. It booted fine, but the network wasn't working and OpenMRS webapp could not be reached from the Windows host. After being on call for close to an hour, we just couldn't make the networking work!! I advised him to install VMPlayer instead and run the image. This time everything ran fine, but some changes had to be made in the Norton 360 Firewall. He kept complaining that Windows XP was punishably slow and then I realized that his 512Mb wasn't enough to virtualize :-( ... So we were back to where it all started!! The Windows Installer that OpenMRS distributes is based on Bitrock. He first tried the OpenMRS 1.1 Installer, but it is an older version that hasn't been upgraded for a year or so... Everything installed fine and he was happy to use it, but it didn't have the features I talked about that were added in newer releases of OpenMRS. I walked him through the manual installation and finally we managed to get OpenMRS up-and-running at 2am in the morning and he having spent about 8hrs on it. Last week when I asked him, he still wasn't using OpenMRS for his clinic and hospital. May be the first experience made him bitter!!

With that episode in my mind, I pledged him that within the next month or so I'll give him and easy to install setup and he'll be happy using OpenMRS. And that's when began my chase to find an easy to use, cross-platform installer framework. OpenMRS has lots of implementations on different platforms (Windows, Linux and Mac) and hence I wanted the installer to be cross-platform. At my office, we generally use Windows Installer or NSIS for making installers. But those are only for Windows. These 2 frameworks are so simple and extensible to use that I was thinking if there was something similar and cross-platform, I could make an OpenMRS Installer in an hour. But sadly, that wasn't the case... I tried a variety of installer frameworks, but couldn't find any of them as simple as NSIS or Windows Installer (msi). The following are the installer frameworks I tried working on:

So finally, I decided to work on using the Netbeans Installer. Netbeans Installer already has components like Tomcat, MySQL, Glassfish, OpenESB and their deployment scripts. And I thought it will simplify my effort... Dmitry Lipin of Sun Microsystems, the lead developer of the NBI team has been of great help over the past weeks and has helped a lot in explaining about nbi... While I was building the installer, 2 other colleagues of mine got interested in OpenMRS and have helped build some parts and want to contribute to OpenMRS code in a larger way!!

I have successfully been able to build an Installer/Uninstaller that can deploy Tomcat/Glassfish, MySQL and the OpenMRS web application on Windows, Linux, OSX, Solaris. The demo data set, JRE/JDK and starting the respective servers are yet to be completed.

Update: The OpenMRS Windows Installer based on Bitrock has been upgraded to install the latest version of OpenMRS. Is it useful for the OpenMRS community to have a cross-platform installer?? Or do the Windows guys only need an Installer ??

The Registration Module is supposed to generate Barcode images that will be printed on stickers and given to patients on their ID cards. This will help easier   identification of patients and quicker patient registration. For this purpose, we use the “Patient Identifier” to create barcodes. The “Patient Identifier” is hopefully unique and will help create bug-free barcodes.

Barcodes have a lot of different standards like Code39, Code128, UPC-A, UPC-E etc. Each of these standards were designed for a specific industry and application. And the Registration Module should support different standards so that it can be easy for the implementers of OpenMRS to choose any standard that they want. Thus, began my journey to find a way to generate customizable and easy barcodes.

Having just finished with the commit for the barcode generation in my registration module, I am extremely happy to say that Barcode Generation through the Registration Module is very easy. Still have to implement a print dialog box, but then the way to generate barcode is done through a pre-built servlet... I used an open-source barcode library called Barcode4j, which can generate barcodes in a variety of standards as well as image formats.

Barcode4j is an excellent library and after comparing about 15 different barcode libraries, I found it to be the most easy-to-use and extensible. Barcode4j already provides a Java Servlet which needs to be passed different parameters and it generates the barcode image “just-like-that”. The only thing I had to do was create a mapping for the servlet in the modules “config.xml”.

And that was it... Those nice black lines were shown on screen beside the patient search results. Now I needed to see if the barcodes are accurate and working. I took a printout of the image and scanned it through the barcode scanner. Hurray!! It worked!! And thus I realized that the servlet was accurately creating barcodes. Now moving onto creating a nice AJAX print dialog and probably some useful UI for creating identity cards.

From the very first day that I had been working on OpenMRS, I felt that OpenMRS ran a little slower than I expected. Probably the old OpenMRS demo server adds to the slowness. Later, when we were discussing about how Hibernate sessions should be implemented in OpenMRS and Java Web Apps in general, I was again brought to think about OpenMRS performance.

Since OpenMRS community generally implements on Tomcat, my main aim was to improve performance of the servlet container. One simple way to improve performance, which I had heard of earlier was the use of Apache has the “Apache Portable Runtime” (APR) project with Native Libraries. The APR uses native libraries with JNI to improve the server performance on a specific platform. In short, Tomcat is given some local OS steroids and currently works on Windows and POSIX-based systems.

The APR library is somewhat an irony for 2 main reasons:

• I’ve heard this argument that Tomcat runs faster than Apache in some benchmarks. These guys argue that Java is faster than C/C++ and hence Tomcat wins.
• On the other hand, APR and Native Tomcat uses JNI code written in C/C++ to improve performance.

Either ways, I think generalizing the above statements isn’t correct and hence I went forward to see if APR does improve performance of our web application. I used Windows Vista and Tomcat 6.0.16 for the test and Windows is probably what most OpenMRS implementations use. You can download the native binaries for Windows from here & APR from here.  Add the extracted files to Path and place the tcnative-1.dll in APR’s bin folder.

And the first thing I observed tomcat started little faster and even OpenMRS initialized slightly faster.

But startup improvement is not all. We want to check how good the application is performing and Apache Benchmark (ab) is a good way to test static content, but isn’t very good at dynamic content... I wanted to use Faban after I remembered Scott Oak’s writeup from last year, but couldn’t find enough time for the testing with Faban...

Instead, I used JMeter which is a nice generalized test that replicates how a user interacts with the web application. You can send POST requests with parameters and also simulate your test plan, just like a normal web user would use your application. Here are some of the results on different OpenMRS pages with 10 concurrent requests and average of 3 runs on my dual core server:

My first observation was that the first run on the test completely sucks. The later runs improve performance drastically. This is because of tomcat 6 has good caching mechanism and was shown with or without APR. Another thing I observed was that beyond 500 concurrent users the application was crying and tomcat was hanging up. APR or no APR didn’t matter much... I’ve yet to analyze why it wouldn’t scale any further, but must be something related to Hibernate sessions. May be some experienced developer can look into these figures, perform some more specific benchmarks and improve scalability.

Last week was really a hectic time for me and hence haven’t found much time to code or blog. A close friend Debojit is in a new Idol-like reality show called “Jo Jeeta Wohi Superstar” and we are back in the publicity campaign like we were when last time he won Saregamapa Challenge 2005. The good thing is that I’m still writing code (to game the online voting), but not exactly for OpenMRS… But I did some work on OpenMRS and nearly had a deliverable basic patient search.

The patient search on my Registration Module has taught me a few important lessons. OpenMRS’s web application uses the Model-View-Controller (MVC) through Spring Framework. I have a controller which calls some methods from the OpenMRS API and retrieves patient information. Normally, the practice is to return values from the Controller to the View (JSP here) is through the use of a bean’s getter methods. This means that the Controller sets the Bean object with the values from the database and the View (i.e. JSP) page uses to the getter methods to get values.

But instead of a bean, I tried to return a double-dimensional array and got stuck with the following... I’m still wondering why I can’t access length variable of the array. Look at the code snippets below and may be I’ll get some hints from you!!

RegistrationController:

protected String[][] formBackingObject(HttpServletRequest request) throws Exception {
        String[][] searchedPatients = new String[0][0];
        if (request.getParameter("phrase") != null) {
            List<Patient> patients = Context.getPatientService().getPatients(request.getParameter("phrase"));
            searchedPatients = new String[patients.size()][8];
            for (int i = 0; i < patients.size(); i++) {
                searchedPatients [i][0] = (patients.get(i)).getPatientIdentifier().getIdentifier();
                searchedPatients [i][1] = (patients.get(i)).getGivenName();
                searchedPatients [i][2] = (patients.get(i)).getMiddleName();
                searchedPatients [i][3] = (patients.get(i)).getFamilyName();
                searchedPatients [i][4] = String.valueOf((patients.get(i)).getAge());
                searchedPatients [i][5] = (patients.get(i)).getGender();
                searchedPatients [i][6] = (patients.get(i)).getTribe().getName();
                searchedPatients [i][7] = (patients.get(i)).getBirthdate().toString(); 
        } 
        log.info("# of patients found: "+searchedPatients.length); 
        return searchedPatients;           
        }
        return searchedPatients; 
}

This String[][] called searchedPatients can be accessed as registrationForm according to my moduleApplicationContext mapping. But in my JSP page when I try to access the .length variable of the registration form there seems to be a problem.

registrationForm.jsp

<c:forEach var="row" begin="0" end="${registrationForm.length}">
    <tr>
        <td>${registrationForm[row][0]}</td>
        <td>${registrationForm[row][1]}</td>
        <td>${registrationForm[row][2]}</td>
        <td>${registrationForm[row][3]}</td>
        <td>${registrationForm[row][4]}</td>
        <td>${registrationForm[row][5]}</td>
        <td>${registrationForm[row][6]}</td>
        <td>${registrationForm[row][7]}</td>
    </tr>
    </c:forEach>

I get an error where the JSP page throws a NumberFormatException for “length” input. Now I was baffled why it was trying to take “length” as input string, when it should have taken the length of the array as its input.

Anyways, with the Bean the current patient search is working, but I have to get back to coding quickly and build a good UI, for which I’m using jQuery. With jQuery I plan to implement AJAX and also some simple but useful UI improvements. Next in-line is searching using the barcode reader and a lot more left to do… Hopefully, I’ll do more!!

The work on the OpenMRS module continues and I have just got a little more excited because of the barcode scanner that I just purchased. The Barcode Scanner is required to test the working of the module, which will be able to identify patients based on barcodes present on patient identity cards. The Barcodes will help solve problems where patients don't give correct information leading to duplicate records for the same patient.

The Barcode Scanner that I purchased is Argox AS-8000, near-range CCD scanner. Detailed specification on the product can be found here. I tested the scanner and it was able to detect a variety of barcodes. Its a low-end CCD scanner, but it detected nearly every thing I put in front of it. Even round jars and bottle!! Brian advised me not to spend much and hence I brought this open-box one. So was in a hurry to test, coz they have just 3-day testing warranty on these stuff.

Screenshots of the Barcode Scanner:

The Barcode Scanner works just like a keyboard and when scanning, the OS feels someone typed in from the keyboard. The scanner works on the PS/2 port of the keyboard and not on the PS/2 port of the mouse. My keyboard is connected to the PS/2 port and hence probably I'll buy a USB converter which will enable me to connect the scanner to the USB port. The Barcode Scanner won't be required by me until I reach the end of the search routines of the registration module... I'm still stuck on getting multiple patients from a search query!!

The Google Summer of Code 2008 timeline shows that we start coding from 26th May and I've already started coding. Before the coding there was this "Community Bonding Period", and I'm not sure if I've done bonding with my community members, but sure I've been talking with my mentor Brian... and he's really a cool guy! We talked a lot about technical as well as personal stuff. He invited me to this house virtually through Google Street View and it was really nice talking to him. I bonded with him well !!

In the meantime, I have been pondering and working on the ideas for the OpenMRS Registration Module.  My Introductory post on OpenMRS showed a lot of interest from friends and got a lot of questions from colleagues. Thanks to r0bby, who told me on the IRC about SoC and talked about contributing to OpenMRS.

The work on the Registration Module has begun and I created the Registration Module documentation page. It is still incomplete on the UML diagrams, but is good enough to show the workflow through the Use-case. Brian and I have also worked on my project plan and have made a timeline that I need to follow. The timeline is a good motivator and will be putting that up on the project page as well. OpenMRS and open-source developers in general, advocate the idea of quick and early commits. OpenMRS shares the Story of FLOSS to make this idea stand out. I'm still trying to get into this mould of development, coz I mainly advocate the planning/design approach. You take twice as much time to design compared to code, but I need to get agile and work on the Bazaar way of open-source. More eyes... More interest and faster working deliverable!!

With this in mind, I tried to commit the basic module changed to registration module, but it gives me a 403 error:

Error: CHECKOUT of '/!svn/ver/4193/openmrs-modules/registration': 403 Forbidden (http://svn.openmrs.org)

I can login through the trac web page. So probably some issue with either my SVN client (TortoiseSVN) settings... or something on the SVN server side!!