Well, it’s been over 5 weeks since my last post (eep). The good news is that development in this time has been going just as strong as ever. Development actually sped up towards the end of the coding period of the Summer of Code, and the code is pretty much ready for a 1.0 release. The only thing that it is waiting on is the Logic Service project to be integrated into the alpha branch from the logic branch. So, before I stop updating this blog again, I will go through a short description about how the project has come along.

First, the Logic Web Service module. This part of the project went through a good number of changes from its start to finish. First, we planned to use the REST module, a module already implemented with simple functionality to access some data about patients. However, after some discussion and work on that, we discovered that the REST Module would need some considerable workarounds to allow the kind of functionality we needed. It lacked the ability to work with parameters in the URL, and required two separate handlers for the old functionality and the new functionality. In light of this, we set up a new module, the Logic Web Service module, which you’ll find the code for at the OpenMRS svn repository. The new module has a servlet named “api” that responds to api functions: getFilters, getTokenTags, getTokens?token=X, and getData?filter=X&filtertype={static | dynamic}&token=Y&token=Z&token=A&token=…. The Logic Web Service very roughly gives access to some basic Logic Service functionality through an XML interface using REST.

The ODA driver has come along nicely too. The UI finished milestone 3 fairly quickly, then went on to add in milestone 4, which dropped in another box, a “selected tokens” box. The user moved tokens between the available and selected tokens boxes. From that, which was supposed to be the final GUI implementation for the version 1.0 release, there were numerous changes that came with the addition of a way to group tokens–token tags. Token tags work in the same way as the much-hyped tags of the web 2.0 movement. To find tokens easier from the list of 1000s, you may just select a particular token tag from a list, and only the few tokens that share that tag would be selectable. The UI and Runtime drivers were updated to allow for this, and soon after the UI was updated slightly to more easily follow the many steps required to create a data set.

The goals for the version 1.0 release were all met, as far as I can tell. I’m really happy with the results, and I can’t wait until this gets into the hands of some actual users of OpenMRS. I can’t wait to get started on the next version of the BIRT ODA Plugin and the Logic Web Service, and while school makes it so I no longer have that much time, I’ll still continue working on it. There’s still many places to take the ODA driver (for instance, choosing datatypes, dynamic logic criteria creation and multiple-column tokens), and I can’t wait to take it there.

I’m not sure on whether or not this blog will continue, but you can track further progress on the Reporting Framework Integration project at http://openmrs.org/wiki/Reporting_Framework_Integration_Project. Thanks for reading!

So I find myself just past the midterm of the program, at the end of week 7, looking back at the progress that I’ve made so far. I’m quite happy with what I’ve achieved so far. Not because I’ve got a monstrous load of code written, or because I’ve finished ahead of schedule. Rather, I’m doing fine on the code front, but the thing I’m most happy with is what I’ve learned about doing real work in the software development world. But I won’t go into that now… I need some material for an end-of-summer blog post, don’t I?

I’ve all but finished the runtime driver (a task that feels like it should be given more than half a sentence), and work on the user interface has started. There probably should have been more time for work on the interface, but I’m going to have much fewer false starts on this, now that I know where I can find information. Documentation has been created and updated, both at the Reporting Framework Integration Project page and at the BIRT ODA Plugin User Guide page. The user guide is incomplete, as the plugin has yet to be finished, and there is nothing to return data as of yet. The Project page now outlines the project better, and includes an outline for the project in what I’ve done so far and what is planned.

The project page also includes some of the latest news in the XML schema, which includes dropping of the <table> tag. We couldn’t really imagine people needing multiple tables, so it was dropped. I’m starting to think that some functions may change before long, so expect that page to be updated soon.

So what’s left in the project, you ask? Well, there are a few that need to be addressed big time.

• User interface needs to be implemented
• Each of the api functions of the Logic Web Service needs to be implemented in the REST Module

In addition to that, there are few things that would be nice to change before the project is done.  For one, it would be nice to update the plugin to play the best it can with the new version of BIRT, BIRT2.2.  It’s currently designed for 2.1, and I’ve made a few changes to make it compatible, so it does work, but I’d like to make sure everything is working just right.  Also, the user interface that gets implemented at first will likely be a stub that is not so polished, and could use work. So, the interface will need work. In addition to that, we may want to include searchable boxes of tokens, instead of forcing the user to search through the tokens themselves. We may also want the same kind of fancy setup that exists in the Flat File driver, allowing users to select tokens from a box on the left and move them over to a box on the left, choosing those tokens easily without having to click multiple tokens all while holding a key to select them all.

Milestones for the User Interface:

1. User can select a filter/cohort. Patient is the only allowed entity. All tokens are returned, and can be selectively placed in the report.
2. User can select an entity, which will affect the choices for filters. All tokens are returned based on which entity is chosen.
3. User selects entity, which affects the choices for filters and tokens. A filter is chosen, and tokens are selected from a list in a box.

This is the end of the user interface stub, which works fine. At this point, the REST Module will have to be implemented. There’s been a lot of planning, but I think lots of work is going to have to go into this, including many many questions to my mentor Justin.

All in all, though, the project is going well and I look forward to these final weeks.

It’s been a fairly long time since my last entry, so I figured it’s time to write another one. A good amount of work has been getting done.

Firstly, actual work on the plug-in has started. I most recently committed code letting the user ping the server with their connection settings. I’ve also been working on the rest of the base code for the ODA driver, and am nearing completion on the easy parts. The hard part is figuring out how to use JAXP (the Java API for XML Processing) to process the XML. I imagine it can’t be too hard, though it will take a few days I’m sure. However, the rest of the runtime driver is going well.

Design for the UI portion of the driver is coming along, and has deviated a good bit from the original design. It’s still not done, but it’s coming along. Originally, I planned to allow the user to choose from the api functions “patient,” “findPatient,” and “cohort,” each of which was specified differently, and filled out the query to the api (patient identifier, patient search string, and cohort respectively). They then selected the tokens they wished to include for the columns. However, this ignored the eventual inclusions of other entities (e.g., encounter, observation) and only cared about the patient entity. So now we allow the user to select from a list of entities (we’re actually only caring about patient now, but the option is there). The user then selects tokens based on the entity (how this will be updated is still undecided). Also, the user selects a constraint/filter, which in the case of the Patient entity is the cohort. As of now, if I’m correct there are no filters/constraints for the other entities yet, but they are possible and will probably arise in a short time.

Work has also been happening on the REST Module that fulfills the need for a Logic Web Service. I made a small bugfix the other day, updating it for the current OpenMRS API, we’ve been planning the new functions for the module, and we’ve been planning the XML formats that will be used, taking into account how other there will be applications of the REST module other than the BIRT plug-in. I have a feeling that work will continue on the REST module well after the summer ends, as more and more applications that can make use of it appear.

As for the XML formats that are going to be used, we’re actively developing them. Currently, we’re working with something that defines a dataset with tables (only one table will be used for our purposes), which in turn defines columns by their metadata, and rows by their data. An example looks something like this:

<?xml version="1.0" encoding="utf-8"?>
<dataset>
 <table entity=”PATIENT”>
  <columns>
   <column token=”PATIENT.PATIENT_ID” />
   <column token=”LAST WEIGHT” />
   <column token=”LAST CD4 COUNT” />
  </columns>
  <rows>
   <row>
    <value>1</value>
    <value>180</value>
    <value>209</value>
   </row>
   <row>
    <value>2</value>
    <value>200</value>
    <value>198</value>
   </row>
  </rows>
 </table>
</dataset>

This isn’t really finalized, but it’s making progress. This is just a small snapshot into what’s been going on. I want to emphasize again that this is definitely the most planning I’ve ever done for a coding project, but it’s actually nice. I feel like this is what software development is really all about.

Of course, adding to the things on my list of things to check out is the release of BIRT 2.2. Congrats to the developers.

The last few days have been spent learning about and understanding the workings of the ODA plug-in. It’s fairly complicated, and the actual documentation on how it interacts with BIRT is sparse. So, among other things recently, I’ve put together a small document that basically does a walkthrough of the user’s interaction with BIRT and the ODA plug-in. It includes some of the main functions called, to give an idea of what kind of data is being created and requested at each step, using the FlatFile ODA plug-in as a basis. I’ve uploaded a copy of that at this link, though the original is part of a larger, growing document in Google Docs.

I believe that eventually, the (larger, currently growing) document will become quite helpful to those who are trying to build an ODA plug-in of their own. The document actually includes a lot of OpenMRS specifics, but I’d like to create a sort of partially genericized version eventually.

Other news of late includes me finally understanding how the OpenMRS database is accessed through the code. I’ll talk about that more some other time, as currently there’s still more I have to learn, such as how I can work through the Logic Service, and I feel it’d be an incomplete entry as of now. I don’t usually do this much planning before jumping into a coding project, but this is such a complex system that any coding I had written a week ago I’d likely want to throw away today. The project is progressing nevertheless.

Well, I’ve finally got a basic, no-functionality plug-in. I tried copying various aspects of other ODA adapters, such as the Flat File and JDBC ODA Plug-ins. However, nothing worked better than just going with a lot of pre-set-up options. So, here is a walkthrough on how to get a shell of a plug-in. I should note that these instructions are for BIRT release 2.1, the official release as of this posting. Soon BIRT 2.2 will be out–it’s currently in RC0 (release candidate 0), so it should not be more than a few weeks until BIRT 2.2 is unleashed. I haven’t worked with BIRT 2.2 yet, but it should definitely be something to look forward to. I haven’t worked with any release other than BIRT 2.1, so if you’re using another version (such as BIRT 2.2 RC0), then this may not work. I’d be surprised if it did not work on an official release though, so go ahead and try anyway if you’re on a full release.

First, we’ll probably want to switch to the Plug-in Development Environment (PDE) in Eclipse. This is accessible by accessing the menus Window => Open Perspective => Other… and selecting Plug-in Development from the pop-up box.

Now onto the creating. Create a new project, select Plug-in Project and click Next.

Enter a Project name. For this, I used “openmrsdb”. The rest should probably stay the same, unless you don’t want this project in your normal workspace. Click Next.

Our Plug-in ID will stay the same, as will the version number and name. I did enter a Plug-in provider, “OpenMRS,” but you don’t need to enter anything. Classpath I left blank for now, even though some tutorials included some things. I didn’t have anything I felt needed to be included, so it stayed blank. If this actually changes in the future, then I’ll update this post then.

Leave the default Activator settings (checked, default name). As for the next checkbox, uncheck it. We are currently building the Runtime Plug-in, which is only a backend. We will make a separate plug-in for the UI soon. And leave the setting for rich client application (No). Click Next.

We do want to use a template, so leave that checkbox marked and click on “ODA Data Source Runtime Driver.” Click Next.

Most of this can stay as it is right now. However, we do want to change “Number of Data Source properties” to 3; you’ll see why soon. Also, I changed “Data Source Display Name” and “Data Set Display Name.” This is purely cosmetic, but who wants a bad user experience? By default, these will be “Openmrsdb Data Source” and “Openmrsdb Data Set,” which are right, but a little odd to say or think about. I change the Data Source name to “OpenMRS Database,” and the Data Set name to “OpenMRS Data Set.” The reason is that we are clearly drawing data from the OpenMRS database, not drawing a source from the OpenMRS database. As for the Data Set name, I thought it sounded redundant to have “Database” in between “OpenMRS” and “Data Set,” but still wanted it to say “Data Set.” Those name changes are completely up to you, and may vary depending on what your ODA driver does. Finally, click Finish.

When you finish, an editor should come up showing some properties about your new plug-in. Click the “Extensions” tab on the bottom, as this is where we have to do some work. Under “All Extensions,” click the arrow to explore the entry. Click the arrow for the first, “openmrsdb (dataSource).” Again, click the arrow for (properties) underneath that. You should see a list of three properties (property1, property2, property3). Click on property1, and you should see a few boxes come up on the right. We are going to fill in the information for these. Enter this data, leaving the rest of the boxes alone:

property1: name = SERVER; defaultDisplayName = %data.source.server; defaultValue = http://
property2: name = USERNAME; defaultDisplayName = %data.source.username
property3: name = PASSWORD; defaultDisplayName = %data.source.password; isEncryptable = true

These are generally self explanatory. These represent the properties that are saved and will be passed when creating a connection. We will want to specify a server, a username, and a password. However, the default display names I gave them don’t currently mean anything. We’re going to have to set those now.

From your project browser, open the file “plugin.properties” at the root of the project. Add these lines to the end of the file:

data.source.server=OpenMRS Server Path
data.source.username=Username
data.source.password=Password

These will be read by the UI to be displayed when you are setting the connection properties. The reason they are in this file is for internationalization support. You may make other language’s versions of these strings, so that people of other languages may more readily use the driver. You may notice that the other values that were optionally changed earlier (data source and data set names) are in this file. You can change them here if you want to change those names ever again. Go ahead and close that file and the editor for the project information, saving if you haven’t already. If you want to get to the editor again, you can try to open one of the files from the project, such as plugin.xml or MANIFEST.MF, and these will get you to that editor, though possibly under a different tab.

Now, we want to create the UI Plug-in. This is probably even simpler. Create another Plug-in Project, and name this one openmrsdb.ui (or equivalent). It doesn’t really matter what the name is, though having the original name plus “.ui” seems natural to me. Leave the rest of the options the same, unless you want to move this out of your normal workspace. Again we don’t really change anything, though you can change the plug-in name or provider here again. Unlike last time, however, leave checked the box “This plug-in will make contributions to the UI.”

On the next screen, we want to use the template “ODA Data Source Designer.” Note that there are a lot more templates this time, due to the fact that the UI contributions checkbox was checked. I guess there aren’t many plug-ins that don’t have to do with the UI. Click next.

This screen is important. Many of the values here must match up with the values from the previous plug-in. If you didn’t change anything other than what I told you you could, you should be fine. Again at the bottom, you can change the names. I changed them to be the same as the ones I set with the runtime plug-in, just to be sure. Couldn’t hurt to change them, right? Finally, click finish.

Now, you’re done. You’ve got a shell of a plug-in. To see it in “action,” you’re going to want to run the project as an Eclipse application. Right click on one of the projects and select Run As => Eclipse Application. When the new instance of Eclipse starts up, you’re going to want to add a new reporting project and add a new report to the project. Switch to the BIRT reporting perspective, and open the “Data Explorer” view. Right click on Data Sources, and click “New Data Source.” Select “OpenMRS Database” from the list (or whatever it is called. As default, it would be “Openmrsdb Data Source”) and click Next.

Aha! It is the connection properties for our plug-in! Success! You should see something that looks like this:

Except yours won’t have data filled in until you fill it in. It doesn’t matter what you put, as it doesn’t do anything yet (the Test Connection button will always work too :-). There’s not too much more to see, but you can try creating a data set. It will ask for a query in a box. This is not what will eventually be there, as we want our users to see something nicer than a plain old query box. No, the purpose of this driver is to smooth the connection.

That will be a good adapter. But for now, all we have is a shell. It’s the beginnings of a wonderful connection between report designer and database. From here, it’s..well, still uphill. But it’s a good start.

Many thanks to Scott Rosenbaum for writing a nice Primer on ODA Extensions and BIRT, available at EclipseMag.net (registration required). It was the first thing that got me a nice shell, and it’s going to help tremendously in adding functionality.

Over the past few days, I’ve been getting set up.  I’ve mostly got up an Eclipse development setup, and I’ve been reading up on various API documentation.  Not the most glamourous part of this summer’s project, but I’ve got to gain background.

As for the Web service side of things, I’ve been reading on the Javadocs for the packages javax.servlet and javax.servlet.http.  Quite useful information, paired with a reference.  As I read the documentation, I was also viewing some servlet examples that came paired with Tomcat, as well as part of Burke Mamlin’s implementation of a REST module for OpenMRS.

Good news on the Web service front: we may be able to use Burke’s module as the base for the web service module.  Burke’s may have implemented what we need, and I would simply add on handlers for the kinds of URLs that need to be handled.  We’re not sure yet if we’ll have to change some things (e.g., to handle multiple tokens), but for now this seems like a great help.

As for the BIRT side of things, I’ve been trying to read through APIs, but it’s harder to find them.  To some degree, it is described on the DTP Open Data Access Overview, but it says for “detail ODA API documentation, see its Javadoc API Reference Documentation included in the DTP download.”  The DTP download, as far as I can tell, is available from the Data Tools Platform Project download page, but the Javadocs are not well organized, at least if you ask me.

I have the source to the Flat File ODA Plugin, which as far as I can tell is just a CSV File Plugin.  I believe this will be the most indispensable reference I will have in building my ODA adapter.  The CSV driver is “an exemplary implementation of the DTP ODA run-time API,” so I believe that following in its example will be my best option.  It won’t be the easiest to program without a highly organized and accessible API documentation, but hey, that’s half the fun.

While dev.openmrs.org is down for a Python update, I’m going to take time to go over my view of the project as of now.

The basic premise of my project is that OpenMRS lacks sufficient reporting capabilities, and needs to be able to better integrate with reporting tools, such as BIRT. BIRT, Business Intelligence and Reporting Tools, is an open source Eclipse-based reporting system. My project goal is to build a direct link between OpenMRS and BIRT, via two components.

The first component is a module to OpenMRS. This functions as a web service that runs and responds to certain queries. Our service will be a REST service, so that it takes standard HTTP queries and returns some kind of XML document. This is pretty cool in and of itself. The REST module responds to queries like

http://demo.openmrs.org/openmrs/getData?cohort=All_Patients&entity=Patient&token=CD4COUNT&token=AGE

Using an analogy to SQL, this looks up the table (entity) of patients, filters out some with a qualifier (WHERE in SQL, here it is the cohort, and All_Patients actually doesn’t filter out any), and returns the data from the columns (tokens) named CD4COUNT and AGE. I don’t have much experience with SQL, but I’d say that this is a good analogy, and the way I understand the web service. This module is a way to interface with the OpenMRS database without having to actually directly access the MySQL database.

The second component is an adapter for BIRT. In this part of the project, I build an Open Data Access (ODA) adapter that allows BIRT to interface with a new source, the OpenMRS web service. Whereas now BIRT is forced to communicate directly with the OpenMRS database through JDBC, the ODA adapter I build will allow BIRT to communicate transparently with OpenMRS. It is as of yet unsure how complicated the adapter will be, possibly allowing custom building of cohorts, or perhaps just cohorts defined in the web interface.

At the end of the project, generating compelling reports in BIRT should be only as hard as designing any report would be; the interface to OpenMRS will be as transparent as possible. The user will not have to worry about the connection to the database, but will rather only have to worry about making the report pretty.