Well, it’s been over 5 weeks since my last post (eep). The good news is that development in this time has been going just as strong as ever. Development actually sped up towards the end of the coding period of the Summer of Code, and the code is pretty much ready for a 1.0 release. The only thing that it is waiting on is the Logic Service project to be integrated into the alpha branch from the logic branch. So, before I stop updating this blog again, I will go through a short description about how the project has come along.

First, the Logic Web Service module. This part of the project went through a good number of changes from its start to finish. First, we planned to use the REST module, a module already implemented with simple functionality to access some data about patients. However, after some discussion and work on that, we discovered that the REST Module would need some considerable workarounds to allow the kind of functionality we needed. It lacked the ability to work with parameters in the URL, and required two separate handlers for the old functionality and the new functionality. In light of this, we set up a new module, the Logic Web Service module, which you’ll find the code for at the OpenMRS svn repository. The new module has a servlet named “api” that responds to api functions: getFilters, getTokenTags, getTokens?token=X, and getData?filter=X&filtertype={static | dynamic}&token=Y&token=Z&token=A&token=…. The Logic Web Service very roughly gives access to some basic Logic Service functionality through an XML interface using REST.

The ODA driver has come along nicely too. The UI finished milestone 3 fairly quickly, then went on to add in milestone 4, which dropped in another box, a “selected tokens” box. The user moved tokens between the available and selected tokens boxes. From that, which was supposed to be the final GUI implementation for the version 1.0 release, there were numerous changes that came with the addition of a way to group tokens–token tags. Token tags work in the same way as the much-hyped tags of the web 2.0 movement. To find tokens easier from the list of 1000s, you may just select a particular token tag from a list, and only the few tokens that share that tag would be selectable. The UI and Runtime drivers were updated to allow for this, and soon after the UI was updated slightly to more easily follow the many steps required to create a data set.

The goals for the version 1.0 release were all met, as far as I can tell. I’m really happy with the results, and I can’t wait until this gets into the hands of some actual users of OpenMRS. I can’t wait to get started on the next version of the BIRT ODA Plugin and the Logic Web Service, and while school makes it so I no longer have that much time, I’ll still continue working on it. There’s still many places to take the ODA driver (for instance, choosing datatypes, dynamic logic criteria creation and multiple-column tokens), and I can’t wait to take it there.

I’m not sure on whether or not this blog will continue, but you can track further progress on the Reporting Framework Integration project at http://openmrs.org/wiki/Reporting_Framework_Integration_Project. Thanks for reading!

So I find myself just past the midterm of the program, at the end of week 7, looking back at the progress that I’ve made so far. I’m quite happy with what I’ve achieved so far. Not because I’ve got a monstrous load of code written, or because I’ve finished ahead of schedule. Rather, I’m doing fine on the code front, but the thing I’m most happy with is what I’ve learned about doing real work in the software development world. But I won’t go into that now… I need some material for an end-of-summer blog post, don’t I?

I’ve all but finished the runtime driver (a task that feels like it should be given more than half a sentence), and work on the user interface has started. There probably should have been more time for work on the interface, but I’m going to have much fewer false starts on this, now that I know where I can find information. Documentation has been created and updated, both at the Reporting Framework Integration Project page and at the BIRT ODA Plugin User Guide page. The user guide is incomplete, as the plugin has yet to be finished, and there is nothing to return data as of yet. The Project page now outlines the project better, and includes an outline for the project in what I’ve done so far and what is planned.

The project page also includes some of the latest news in the XML schema, which includes dropping of the <table> tag. We couldn’t really imagine people needing multiple tables, so it was dropped. I’m starting to think that some functions may change before long, so expect that page to be updated soon.

So what’s left in the project, you ask? Well, there are a few that need to be addressed big time.

• User interface needs to be implemented
• Each of the api functions of the Logic Web Service needs to be implemented in the REST Module

In addition to that, there are few things that would be nice to change before the project is done.  For one, it would be nice to update the plugin to play the best it can with the new version of BIRT, BIRT2.2.  It’s currently designed for 2.1, and I’ve made a few changes to make it compatible, so it does work, but I’d like to make sure everything is working just right.  Also, the user interface that gets implemented at first will likely be a stub that is not so polished, and could use work. So, the interface will need work. In addition to that, we may want to include searchable boxes of tokens, instead of forcing the user to search through the tokens themselves. We may also want the same kind of fancy setup that exists in the Flat File driver, allowing users to select tokens from a box on the left and move them over to a box on the left, choosing those tokens easily without having to click multiple tokens all while holding a key to select them all.

Milestones for the User Interface:

1. User can select a filter/cohort. Patient is the only allowed entity. All tokens are returned, and can be selectively placed in the report.
2. User can select an entity, which will affect the choices for filters. All tokens are returned based on which entity is chosen.
3. User selects entity, which affects the choices for filters and tokens. A filter is chosen, and tokens are selected from a list in a box.

This is the end of the user interface stub, which works fine. At this point, the REST Module will have to be implemented. There’s been a lot of planning, but I think lots of work is going to have to go into this, including many many questions to my mentor Justin.

All in all, though, the project is going well and I look forward to these final weeks.

It’s been a fairly long time since my last entry, so I figured it’s time to write another one. A good amount of work has been getting done.

Firstly, actual work on the plug-in has started. I most recently committed code letting the user ping the server with their connection settings. I’ve also been working on the rest of the base code for the ODA driver, and am nearing completion on the easy parts. The hard part is figuring out how to use JAXP (the Java API for XML Processing) to process the XML. I imagine it can’t be too hard, though it will take a few days I’m sure. However, the rest of the runtime driver is going well.

Design for the UI portion of the driver is coming along, and has deviated a good bit from the original design. It’s still not done, but it’s coming along. Originally, I planned to allow the user to choose from the api functions “patient,” “findPatient,” and “cohort,” each of which was specified differently, and filled out the query to the api (patient identifier, patient search string, and cohort respectively). They then selected the tokens they wished to include for the columns. However, this ignored the eventual inclusions of other entities (e.g., encounter, observation) and only cared about the patient entity. So now we allow the user to select from a list of entities (we’re actually only caring about patient now, but the option is there). The user then selects tokens based on the entity (how this will be updated is still undecided). Also, the user selects a constraint/filter, which in the case of the Patient entity is the cohort. As of now, if I’m correct there are no filters/constraints for the other entities yet, but they are possible and will probably arise in a short time.

Work has also been happening on the REST Module that fulfills the need for a Logic Web Service. I made a small bugfix the other day, updating it for the current OpenMRS API, we’ve been planning the new functions for the module, and we’ve been planning the XML formats that will be used, taking into account how other there will be applications of the REST module other than the BIRT plug-in. I have a feeling that work will continue on the REST module well after the summer ends, as more and more applications that can make use of it appear.

As for the XML formats that are going to be used, we’re actively developing them. Currently, we’re working with something that defines a dataset with tables (only one table will be used for our purposes), which in turn defines columns by their metadata, and rows by their data. An example looks something like this:

<?xml version="1.0" encoding="utf-8"?>
<dataset>
 <table entity=”PATIENT”>
  <columns>
   <column token=”PATIENT.PATIENT_ID” />
   <column token=”LAST WEIGHT” />
   <column token=”LAST CD4 COUNT” />
  </columns>
  <rows>
   <row>
    <value>1</value>
    <value>180</value>
    <value>209</value>
   </row>
   <row>
    <value>2</value>
    <value>200</value>
    <value>198</value>
   </row>
  </rows>
 </table>
</dataset>

This isn’t really finalized, but it’s making progress. This is just a small snapshot into what’s been going on. I want to emphasize again that this is definitely the most planning I’ve ever done for a coding project, but it’s actually nice. I feel like this is what software development is really all about.

Of course, adding to the things on my list of things to check out is the release of BIRT 2.2. Congrats to the developers.

Over the past few days, I’ve been experimenting with the plug-ins and the ODA API. I finally got a nice Javadoc set up of the ODA API, generated from getting the source to the ODA interfaces and getting it into Eclipse and generating the Javadocs from within Eclipse. I’ve put them on the Internet for anyone to see, where they can hopefully stay for a while, accessibly on my Cornell site. I’m quite glad to have made this, as now my complaints from last time no longer really apply. Though I question why this is not made readily available already. But now, on to the Eclipse work…

First, I must say that I first tried following a small tutorial on creating “your first Eclipse plug-in.” As there are not any tutorials that I can find on ODA drivers specifically, I settled for a generic plug-in. I followed a tutorial I found somewhere on the Eclipse website, that was hosted over at EclipseZone. It wasn’t terribly exciting, but it did give me a little button in my Eclipse toolbar that, when clicked said “Hello, World!” Yes, everyone’s favorite phrase makes a comeback. I must admit that I love writing “Hello, World!” programs. One of these days I’ll find out who started this trend. Anyway, on to the Flat File ODA plug-in.

Apparently, the version of the Flat File plug-in that was in CVS on the dev.eclipse.org website is a newer version than the release version. That’s perfectly fine, normally. Unfortunately, this means that it’s completely incompatible with old versions of the ODA framework, such as the one that is currently available to the general public from the Eclipse datatools project website and through the Eclipse download manager.

Luckily for me, I figure out that in plug-in development mode, I can access the plug-ins that are currently usable in Eclipse, and import the source as a project to my workspace. It’s just one tab over, next to the Package Explorer. How convenient!

I’ve been playing with those a good bit since I found that. I made some changes, just to figure out how it all works. First in property and XML files, so that I could change the name from “Flat File Data Source” to “Flat File Duck Source” (first change I could think of):

I’ve made a few other changes as well, such as editing the group on the next page of the New Data Source selector to have 5 instead of 4 rows, and filled the 5th row with another radio button. The previous four were CSV (comma-separated values), SSV (semicolon-separated values), TSV (tab-separated values), and PSV (pipe-separated values). The new one was QSV (quack-separated values). I guess ducks are very important to my code. I didn’t implement the feature, mostly because it would involve a few different source files, but it wouldn’t have been too hard.

On that front, I am currently trying to set up new projects, openmrsdb and openmrsdb.ui, which at first will just be a copy of flatfile and flatfile.ui. Eventually these will be going into the OpenMRS source code in openmrs-contrib, as opposed to a module or the main code, as they plug into Eclipse and BIRT, rather than OpenMRS.

On the front of the Web Service module for OpenMRS, I’ve gone through the instructions on the OpenMRS wiki about Creating your first OpenMRS module. Of course, I’ll be working instead on Burke’s older REST module, so I checked that out as well, and added it as a project that I can directly work on. For that part, I am completely set up, as far as my developing environment goes. I will get some general practice with web services soon, so that work on the REST module can get started.

The next order of business is getting at least a shell of the ODA modules set up. But then again, that may wait a little bit, as today I got Integrating and Extending BIRT by Weathersby et al. in the mail. I would like to note here that the USPS is an odd service. I saw the mail truck drive by today, and went to check the status of my package, which wasn’t supposed to arrive until June 15th (10 days early!). As I noted that the tracking said it was still in Jersey City, NJ, USA, there was a knock on the front door, which was the mailman with my package. I can understand the fastness, but I would really like some better tracking. I live in New York, which is at least a day away from Jersey City by mail, if it was going fast. In any case, I’m glad to have the book, so that I can have a reference other than the API and the Flat File implementation.

Over the past few days, I’ve been getting set up.  I’ve mostly got up an Eclipse development setup, and I’ve been reading up on various API documentation.  Not the most glamourous part of this summer’s project, but I’ve got to gain background.

As for the Web service side of things, I’ve been reading on the Javadocs for the packages javax.servlet and javax.servlet.http.  Quite useful information, paired with a reference.  As I read the documentation, I was also viewing some servlet examples that came paired with Tomcat, as well as part of Burke Mamlin’s implementation of a REST module for OpenMRS.

Good news on the Web service front: we may be able to use Burke’s module as the base for the web service module.  Burke’s may have implemented what we need, and I would simply add on handlers for the kinds of URLs that need to be handled.  We’re not sure yet if we’ll have to change some things (e.g., to handle multiple tokens), but for now this seems like a great help.

As for the BIRT side of things, I’ve been trying to read through APIs, but it’s harder to find them.  To some degree, it is described on the DTP Open Data Access Overview, but it says for “detail ODA API documentation, see its Javadoc API Reference Documentation included in the DTP download.”  The DTP download, as far as I can tell, is available from the Data Tools Platform Project download page, but the Javadocs are not well organized, at least if you ask me.

I have the source to the Flat File ODA Plugin, which as far as I can tell is just a CSV File Plugin.  I believe this will be the most indispensable reference I will have in building my ODA adapter.  The CSV driver is “an exemplary implementation of the DTP ODA run-time API,” so I believe that following in its example will be my best option.  It won’t be the easiest to program without a highly organized and accessible API documentation, but hey, that’s half the fun.

While dev.openmrs.org is down for a Python update, I’m going to take time to go over my view of the project as of now.

The basic premise of my project is that OpenMRS lacks sufficient reporting capabilities, and needs to be able to better integrate with reporting tools, such as BIRT. BIRT, Business Intelligence and Reporting Tools, is an open source Eclipse-based reporting system. My project goal is to build a direct link between OpenMRS and BIRT, via two components.

The first component is a module to OpenMRS. This functions as a web service that runs and responds to certain queries. Our service will be a REST service, so that it takes standard HTTP queries and returns some kind of XML document. This is pretty cool in and of itself. The REST module responds to queries like

http://demo.openmrs.org/openmrs/getData?cohort=All_Patients&entity=Patient&token=CD4COUNT&token=AGE

Using an analogy to SQL, this looks up the table (entity) of patients, filters out some with a qualifier (WHERE in SQL, here it is the cohort, and All_Patients actually doesn’t filter out any), and returns the data from the columns (tokens) named CD4COUNT and AGE. I don’t have much experience with SQL, but I’d say that this is a good analogy, and the way I understand the web service. This module is a way to interface with the OpenMRS database without having to actually directly access the MySQL database.

The second component is an adapter for BIRT. In this part of the project, I build an Open Data Access (ODA) adapter that allows BIRT to interface with a new source, the OpenMRS web service. Whereas now BIRT is forced to communicate directly with the OpenMRS database through JDBC, the ODA adapter I build will allow BIRT to communicate transparently with OpenMRS. It is as of yet unsure how complicated the adapter will be, possibly allowing custom building of cohorts, or perhaps just cohorts defined in the web interface.

At the end of the project, generating compelling reports in BIRT should be only as hard as designing any report would be; the interface to OpenMRS will be as transparent as possible. The user will not have to worry about the connection to the database, but will rather only have to worry about making the report pretty.