It's been a while since I last updated on the progress of the Facility Data Module. So without farther adieu here we go:

• The relation between the FacilityDataFormSchema and FacilityDataFormSection is now Many to Many meaning that sections can now be re-used between several form schemas (or reports).
• It now runs without errors on OpenMRS 1.5.x
  

The first item was a big problem that was a sore spot for this project since it wasn't too intuitive, ideally a user should be able to re-use a section. The second item was a freebie since the errors were likely all due to the incorrect modeling.

What's next on my list is the following:

1. Allow for sections to be re-ordered within the schema (this is currently not possible and is a show-stopper.
   
2. Fix up the calendar management pages -- those are horribly ugly, but this is after all a first pass, so that is to be expected right?
3. Currently, it does not validate inputs for Numeric data types -- It need to.
   
4. There still needs to be the ability to analyze the data that exists in the system to submit to funding sources; those pages need to be implemented but that is phase two of the project.
   
5. There are likely other sore spots that exist that I do not see at this point and time that will be fixed at a later date.

That is all for now.

I have made a lot of progress on my Google Summer of Code Project.

First, I've completed the code for the generation of the directory structure for the forms. This is done using the form id, which is the name with all illegal characters removed. The only valid characters left are alpha-numeric and periods. Let me tell you, the regular expression to replace all of those characters was hell to write. But, thanks to the help of Jason Davis, I was able to make it much more readable and easier digest. You've gotta see it to get an idea of how bad it was, so here goes:

   /**
     * Removes all illegal characters, then removes all spaces and makes it all lower-case.
     *
     * @param str the form name
     * @return a viable form id.
     */
    public static String formNameToFormId(String str) {
        return str.replaceAll("[\\$\\(\\)%`~!@#&;:\\^=\\+\\?
\"\\*\\\\////\\\\{\\}'\\]\\[\\|\\s+]", "").toLowerCase();
   
    }

That is pretty much hell on earth to read, and it was hell on earth to type. Here's the *MUCH* simpler version:

   /**
     * Removes all illegal characters, then removes all spaces and makes it all lower-case.
     *
     * @param str the form name
     * @return a viable form id.
     */
    public static String formNameToFormId(String str) {
        return str.replaceAll("[^a-zA-Z0-9.]", "").toLowerCase();

    }

Note, you can clearly see that i'm negating that entire sequence. It does the SAME thing, but is MUCH more readable. That reads: "replace everything that isn't a letter from A to Z *OR* a to z(lower case letters) *OR* from 0 through 9 *OR* a period. Whereas the above just excluded the invalid characters explicitly but was a MONSTER of a regular expression.

Secondly, metadata persistence is written in, I'm using xstream. It's a great tool for XML serialization.

That's it. Oh, by the way, these are the things I earmarked TWO WEEKS FOR! Now, I push domain class model processing up, will work on that tomorrow.

So, it's time for an update. This week got off to a sketchy start, but it's gained momentum. Let me enumerate what I have done thus far with the project. First, I've added AJAX using jquery. I figured I would use jquery mainly because it provided painless AJAX goodness. Prior to even thinking of using jquery, I wrote my own AJAX code using a tutorial that I found while googling. I had to tweak it a bit, but for the most part, it seemed very standard. However, after I wrote the AJAX equivalent code, it didn't feel too elegant. So, first I'm going to show the AJAX code I wrote; then I'll show you the jquery version; finally, I'm going to show you the servlet which handles the AJAX on the server-side.

First, the AJAX I wrote:

function AjaxValidation(url, callback) {
    var req = init()
    req.onreadystatechange = processRequest

    function init() {
        if (window.XMLHttpRequest) {
            return new XMLHttpRequest()
        } else if (window.ActiveXObject) {
            return new ActiveXObject("Mircosoft.XMLHTTP")
        }
    }

    function processRequest() {
        if (req.readyState == 4) {
            if (req.status == 200) {
                if (callback) {
                    chkSyntaxCallBack(req.responseXML)
                }
            }
        }
    }

    this.doGet = function() {
        req.open("GET", url, true)
        req.send(null)
    }
}
function chkSyntaxCallBack(responseXML) {
    var res = responseXML.getElementsByTagName("result")[0].firstChild.nodeValue
    document.getElementById("out").innerHTML = res
}

function checkSyntax() {
    var target = document.getElementById("groovyModel")
    var url = "${pageContext.request.contextPath}/moduleServlet/groovyforms/createGroovyForm?groovyModel=" + escape(target.value)
    var ajax = new AjaxValidation(url, chkSyntaxCallBack)
    ajax.doGet()
}

I warned you that it wasn't too elegant. Understanding this isn't too hard. Here is the call sequence: init() -> doGet() -> processRequest() -> chkSyntaxCallBack(). Not that bad. Now let's see the jquery version.

$(window).ready(function () {
    $("#groovyModel").bind("blur", function () {
        $.ajax({
            type: 'POST',
            data: { groovyModel: $("#groovyModel").val() } ,
            url: "${pageContext.request.contextPath}/moduleServlet/groovyforms/createGroovyForm" ,
            cache: false ,
            success: function(data) {
                var res = $(data).find("result").text()
                $("#out").html(res)

            }
        })
    });
})

Okay, that's much better. A few things are still happening here. When the window is finished loading, I bind my textarea element which has the CSS id of "groovyModel" to the blur event (lost focus). Then you see the AJAX. Now this is very straight forward. We're using the POST method, we're sending whatever value is inside of the textarea at the time the event is fired, we're posting to a servlet, not going to cache, and when we're done, it's printed to the screen. Very straight forward.

Now, like I said, this is all backed on the server-side by a servlet, which is written in Groovy. So here we go:

/**
 * The contents of this file are subject to the OpenMRS Public License
 * Version 1.0 (the "License") you may not use this file except in
 * compliance with the License. You may obtain a copy of the License at
 * http://license.openmrs.org
 *
 * Software distributed under the License is distributed on an "AS IS"
 * basis, WITHOUT WARRANTY OF ANY KIND, either express or implied. See the
 * License for the specific language governing rights and limitations
 * under the License.
 *
 * Copyright (C) OpenMRS, LLC.  All Rights Reserved.
 */
package org.openmrs.module.groovyforms.web

import javax.servlet.ServletException
import javax.servlet.http.HttpServlet
import javax.servlet.http.HttpServletRequest
import javax.servlet.http.HttpServletResponse
import org.apache.commons.logging.LogFactory
import org.codehaus.groovy.control.CompilationFailedException

class CreateGroovyFormServlet extends HttpServlet {
  def classLoader
  static final def log = LogFactory.getLog(CreateGroovyFormServlet.class)
  private static final long serialVersionUID = 066373513262051L

  @Override
  protected void doGet(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    def generateTemplate = request.getParameter("template")
    def generateController = request.getParameter("controller")
    def finalMarkup = request.getParameter("markup")
    def clazz = URLDecoder.decode(request.getParameter("groovyModel"))
    def name = request.getParameter("formName")
    def version = request.getParameter("version")
    def res = this.checkSyntax(clazz)
    if (clazz) {
      if (checkSyntax(clazz)) {
        response.contentType = "text/xml"
        response.setHeader "Cache-Conrol", "no-cache"
        response.writer.write "\n\t$res\n"
      } else {
        response.contentType = "text/xml"
        response.setHeader "Cache-Control", "no-cache"
        response.writer.write "true"
      }
    } else {
      response.contentType = "text/xml"
      response.setHeader "Cache-Control", "no-cache"
      response.writer.write "\n\tPlease fill in the Form Model\n"
    }
  }

  @Override
  protected void doPost(HttpServletRequest request, HttpServletResponse response) throws ServletException, IOException {
    doGet(request, response)
  }

  @Override
  void init() throws ServletException {
    if (log.infoEnabled)
      log.info("Initializing...")
    classLoader = getClassLoader()
  }

  def getClassLoader() {
    def gcl = new GroovyClassLoader(this.getClass().getClassLoader())
    gcl
  }

/**
 * This method is used to relay errors to the user
 * @param clazz the class
 * @return the exception message or null if it was successful
 */
  def checkSyntax(clazz) {
    def sb = new StringBuilder()
    sb << "import org.openmrs.*\n\n\n"
    sb << clazz
    def res = null
    try {
      getClassLoader().parseClass(sb.toString())

    } catch (CompilationFailedException e) {
      res = "Exception: ${e.message}"
    }
    res

  }

/**
 * Check if it is result groovy code.
 * @param clazz the class
 * @return whether or not it is result groovy code
 */
  def isValidGroovy(clazz) {
    def sb = new StringBuilder()
    sb << "import org.openmrs.*\n\n\n"
    sb << clazz
    try {
      getClassLoader().parseClass(sb.toString())
    } catch (CompilationFailedException e) {
      return false
    }
    return true
  }
}

This servlet contains a lot of utility methods. One compiles, one initializes/returns the GroovyClassLoader, and of course doGet(), doPost() and init(). doGet() and doPost() both do the same thing, with doPost() simply delegating to doGet(). The code should be reasonably easy to understand. checkSyntax() returns null if it was parsed cleanly, otherwise it returns the exception message, the stack track wouldn't be useful in my case. It returns an XML tag <result> with "true" if it was successful, the exception message if it was not, and a message stating that the field must be filled in if it's empty or just not passed in. I implicitly import org.openmrs.* to allow for easier access to the OpenMRS domain model classes. The parent classloader for the GroovyClassLoader is set the servlet container's classloader. This gives me access to the classpath when loading Groovy classes.

I still have a bit to do, templating needs to be written in, for the most part it's done. Just have a few problems I'm facing, but I'll get through it. Too many people are relying on me succeeding. I feel that this project could help a lot of people, so i feel pressure to succeed. I still need to write in the "Edit" functionality of the "Manage Groovy Forms" page.

I'm definately making progress. More updates to come, that's for sure.

Okay, week 1 is nearing an end.I accomplished alot of things.

First, I completed the meta-data storage code. I wrote some tests, which revealed some bugs that needed fixing, and they were. I am happy to say, that the metadata storage works!

Writing the test was difficult due to the fact I store the forms in the system using a static List in my container class. I wound up updating to JUnit 4 to get at the @BeforeClass annotation so that the only one set of forms gets loaded into the container class. This helped me greatly. Additionally, the @AfterClass annotation was handy for cleaning up from the tests.

Secondly, Writing the code for interrogating the model was a piece of cake, thanks to groovy. All code for interrogating the model has 50 lines! That includes a model class I wrote to hold the field types and field names. I'll show you, but before I do, I should note that return statements are optional, and the final statement will be returned.

package org.openmrs.module.groovyforms.util

import java.lang.reflect.Field
import org.openmrs.module.groovyforms.metadata.model.GroovyFormsDomainModel

/*
 * Utility class containing methods for class interrogation.
 */
class GroovyFormsClassUtil {
    /**
     * Interrogates the class for all declared fields and
     * stores the type and name in a container class
     * @param the {@link Class#getCanonicalName() canonical name of the class}
     * @see Class#getCanonicalName()
     * @return a reference to a container class containing the type
     */
    static def getModel(fields) {
        def domainModel = new GroovyFormsDomainModel()
        def names = domainModel.fieldNames.&add;
        def types = domainModel.fieldTypes.&add;
        def f = fields.each {Field field ->
            names field.name
            types field.type.canonicalName
        }
        domainModel
    }
}

With groovy, it's so easy and concise (as you can see). Let's explain what's going on. First, I pass in the Field array I get from Field.getDeclaredFields(). Now, groovy adds methods onto the standard JDK classes, one of those methods is a method named each() which takes a closure. Now, back to the point, I pass a Field into the closure. That closure is executed for each element (in this case, a Field). Then I add the name, and the type to a List stored in a container class, now that container class is also written in groovy!

package org.openmrs.module.groovyforms.metadata.model

/*
 * Ths class holds information about the properties of the model. 
 */
class GroovyFormsDomainModel {

    /**
     * The field names
     */
    def fieldNames = []

    /**
     * The field types
     */
    def fieldTypes = []

}

Now, the fields are Lists, not arrays. Anyways, I've gotten off on a tangent here, so let me get back on track.

What I accomplished:


Week 1:

Code to generate the directory structure, serialization of metadata back/forth between XML and POJOs (Plain Old Java Objects), Wrote tests to ensure everything works in that regard. Additionally, I wrote the code to interrogate the domain model which I will generate the forms from.

Next Week

Write up the templates for the view/controller and write code to do the generation of the view/controller. Write some tests to ensure everything generates correctly.

Unrelated to that, my stored-value card from google came today. It feels nice to be $500 richer! This is going to be the best summer, I'm already having fun doing this. It's amazing seeing the whole project evolve into something amazing.

Google Summer of Code 2009 officially ended on August 17, 2009 at 19:00 UTC (12:00 PDT). I had an amazing time this summer. While stressful at times, it was well worth it.

I would like to thank the following people for my success this summer:

My mentor Mike Seaton for helping me throughout various misunderstandings surrounding the project's requirements. I probably made him work more than he had to. I attribute the success this summer to that one factor. This man is quite brilliant and knows his stuff.

My backup mentor Darius Jazayeri, for stepping in and helping at times and providing guidance when necessary. He is truly a brilliant man.

Ben Wolfe for answering any questions I had. He didn't have to help, but he did. I am very grateful.

Paul Biondich and Burke Mamlin for believing in me and giving me a second chance this summer. Did I mention these guys are doctors? Burke, an internist and Paul a pediatrician. Sorry for clumping you guys together.

Finally, without Leslie Hawthorn, Cat Allman, and Ellen Ko from Google, this program would not exist. They deserve a huge thank you. These three women are amazing. Thank you so much!!

Working on a GSoC project was indeed a wonderful experience. To make it more interesting, working as a summer intern at OpenMRS was value added experience. :) (discarding that fact that SL doesnt have any summers. <:o) ).

Starting the project i had so little experience working with the Quartz scheduling framework. Then again this was my first project from scratch. I had to go through all of the software development phases. Well that was a lovely experience. I had a lot of help from OpenMRS developers to complete my project. All helped to test the module and give feedback. without those comments the current schedulerQuartz module wouldn't have come this far. i was lucky enough to have my own Lab machine which helped a lot to test the module and to do Demos.

Justin Miranda, being my mentor was a great benefit to this project. I can remember the hours he spent advising me on how to improve usability. His supervision made the module to have good unique features. I must say that i didnt have any kind of pressure from him while doing this project. (of course we only had less than 3 months to complete it and get it running). you rock!

looking back, OpenMRS is a great place for anyone to do their summer internship for Google Summer of code project. Developers working at OpenMRS knows how to identify your capabilities and only to take what you can give. It has a friendly environment so that a new comer would feel comfortable enough to give his full effort in making a project success.

I would like to thank Justin for supervising my project. And everyone at OpenMRS for helping me to bring the schedulerQuartz to where it is right now.

QuartzScheduler v0.2.0 User Guide

Table of Contents
1 About QuartzScheduler
2 Module Settings.
3 Using the module
3.1 Module Sections.
3.1.1 Scheduler Manager
3.1.2 Scheduler Details
3.1.3 Scheduler Log
4 Creating a new Task
5 Get details of a created Task and modifying

1 About QuartzScheduler

The quartzScheduler module was create as a Google Summer of Code 2009 Project. It’s a scheduler created to give extensive scheduling features to OpenMRS users. The quartzScheduler uses OpenMRSs core schedulers Task classes to schedule its tasks.

Additional information can be acquired emailing the module creator djmlog103[-at-]gmail.com or contacting the developer list dev[-at-]openmrs.org

2 Module Settings

The quartzScheduler module needs a set of parameters to be passed before users start using the module. These set of parameters of parameters are used to setup mail configurations in order for the module to use its email notifications features. You should save the following set of key,value pairs in your OPENMRS_RUNTIME_PROPERTIES.PROPERTIES file. Please note that the module notification service users only stmp transmission.

Eg:
schedulerquartz.smtphost=smtp.orgmail.com
schedulerquartz.smtpport=465
schedulerquartz.smtpdebug=false
schedulerquartz.smtpusername=yourOrganization@email.com
schedulerquartz.smtppassword=password
schedulerquartz.serverhost=www.organization.org/openmrs

3 Using the module
After loading the quartzScheduler module, a new category named Quartz Scheduler will appear in the Administration section.

Figure 1: Loaded Module

3.1 Module Sections

• Scheduler Manager : Create, Edit, Delete Tasks.
• Scheduler Details : Shows information about the Quartz Scheduler.
• Scheduler Log : A detailed module level log.

3.1.1 Scheduler Manager

This section allows users to Create new Tasks, Edit and Delete existing tasks.

Figure 2: Scheduler Manager page

3.1.2 Scheduler Details

The SchedulerQuartz scheduling module is created using Quartz scheduling Framework. This Section Allows you to see on what configurations the Quartz Scheduler is running on.

Figure 3: Scheduler Details page

3.1.3 Scheduler Log

This section is intended for Task developers. This page contains all the log data produced in the module scope. The Scheduler log page is equipped with a tailing feature which would print latest lines of the log file in the given iteration time. To improve usability the module log is printed in reverse order to avoid scrolling. This means that all the current data from the log will be printed on top of the page.

Figure 4: Scheduler Log page

4 Creating a new Task

Creating a new task is quite simple. Go to the Scheduler Manager page. Click on the Create Task button.

Figure 5: Create a new Task

Clicking Create Task will bring a popup window which allows users to create new Tasks. Creating a new Task in the SchedulerQuartz Modules is so simplified that there is nothing to explain in this guide. A detailed explanation about creating tasks is given in the Create Task page.

Figure 6: Create Task page

5 Get details of a created Task and modifying

To get Additional information of a created task, visit the Scheduler manager page. And click on the Tasks title on which you to view additional information.

Figure 7: click to get additional information

Click on the Tasks title will bring a popup window with a detailed page of that Task. To modify the Task just click the “Edit Task” link show in figure 8.

Figure 8: Detailed info on a Task

Created on: 8/13/09 Author: djmlog103

It has come to the last few days of GSoC2009. I couldnt keep up with blogging my work in the past weeks. Here's a list which consist of my weekly progress. (extracted from projects tracking document)

Iteration #1 (Sunday, May 31, 2009)

• Created new basic module for quartz scheduler
• Committed quartzscheduler module to the svn repository
  
• Made module deployable in OpenMRS 1.5
• Integrated code and configuration that has already been written into the new module
• Allow user to schedule a simple hello world task to write a message to the log on some interval

Iteration #2 (Sunday June 7, 2009)

• Got to know more on how quartz work and its post-schedule(what options does it have after scheduling a job) features.
• User stories 35-38 (Edit/new Task page)
• 35. Allow users to enter the task name of the class.
  36. Allow users to enter the description of the schedule.
  37. Allow users to set the priority of the schedule.
  38. Allow users to set the recurrence option for the schedule
  a) Run Once
  b) Seconds
  c) Minutes
  d) Hours
  e) Daily
  f) Weekly
  g) Monthly
  h) Yearly
  i) Cron

Iteration #3 (Sunday June 14, 2009)

• User stories 8-14 (scheduler Manager page)
• 8. Allow users to view class name of the class being scheduled.
  a. Allow users to click on the class name and view a full description of it.
  9. Allow users to view the owner of a particular schedule.
  10. Allow users to view the date of which a particular schedule is created.
  11. Allow users to view the date of which a particular schedule is be stopped running.
  12. Allow users to view the priority of a particular schedule.
  13. Allow users to view the status of a particular schedule.
  a. Started
  b. Stopped
  c. Finished
  d. Failed
  14. Allow users to view the next occurrence time of which the scheduler will start executing.
• User stories (view Task page )
• Allow user to view the status of a particular schedule (Started, Stopped, Finished, Failed).
• Allow user to view the next occurrence time of which the scheduler will start executing.
• Allow user to view the date of which a particular schedule is being stopped running.
• Allow user to view the status of a particular schedule.
• Allow user to view the description of the status of a particular schedule.
• Added more strict validations to the scheduler Form page
  

Iteration #4 (due Sunday June 21, 2009)

• QA test on the Create Task page (Bugs Found):
  
• RunOnce
• Seconds
• when selected the repeat option it always executes one more time than what i put.
  
• Minutes
• Doesnt execute on the given end time. the schedule finished before it.
• when selected the repeat option it always executes one more time than what i put.
• Hours
• when selected the repeat option it always executes one more time than what i put.
• Implement user stories 15-17
• Allow users to execute a particular task immediately.
  Allow users to skip the next occurrence of a task from being executed.
  Allow users to delete a particular schedule.
• Finalize feature list of the 3 pages with Justin.
• Finish the View Task page completely.
• Try to integrate openMRS Jobs, replacing the hello world dummy job.
• Deploy quartz scheduler module to labs
  

Iteration #5 (due Sunday June 28, 2009)

• Scheduler Types QA
  

Iteration #6 (due Sunday July 5, 2009)

• Fixed scheduling bugs in the TabsForm page.
• Allow user to restart the scheduler.
• Allow user to Pause all schedules in the scheduler.
• Allow user to resume all paused schedules.
• Allow user to view a particular tasks, number of remaining executions.
• Allow user to click on the class name and view a full description of it (scheduler manager page).
• Allow user to view a particular schedules list of execution dates
• Allow user to view number of jobs in the Scheduler.
• Allow user to view number of jobs by status (started, finished, failed, stopped).

Iteration #7 (due Sunday July 12, 2009)

• Followed up with some R&D on how to edit quartz JobDataMap on the fly.
• Fixed remaining bugs.

Iteration #8 (due Sunday July 19, 2009)

• Changed DumbJob to TaskWrapper
  
• view_tasks.jsp renamed to taskList.jsp
• tabForms.jsp renamed to taskForm.jsp
• Added the drop down list. other changes will follow after a UAT
  
• Schedule List on the View Task page should be "Next 10 execution times"
• Added Light Green and red. Green shows started tasks. Red shows paused/stopped tasks
  
• The "Run Now" and "Delete" buttons should be on opposite sides. In addition, the "delete" button should be smaller (the "trash" icon) or should be a checkbox, with a Delete Selected button under the table (or in the table footer).
• Added server time to the create task page.
  
• Removed priority from the Create Task and leave that as a detail for the Edit Task page.
  
• The Scheduler Manage page should also show the current time so users can compare the current time against all of the tasks that are in the system.
• Added the scheduler logger page with tailing functionality.
• Added the base to send notification mail
• Added the edit task functionality.

Iteration #9 (due Sunday July 26, 2009)

• Added Notification sending features
• Added properties functionality
• minor UI changes
  

Iteration #10 (due Sunday Aug 2, 2009)

• added the notification Title after the UI change
• UI changes (new functionality should stop at this stage)

Iteration #11 (due Sunday Aug 9, 2009)

• UI
  
  Renamed Scheduler Manager to Task Manager
  Renamed Scheduler Summary to Scheduler Manager
  
  ===== Scheduler Manager===========
  
  Added some spacing between the scheduler Manager buttons
  Added tooltips to Restart/Pause/Resume buttons saying what itll do.
  
  ===== Scheduler Log ===========
  • Changed "Tail Every" to be "Refresh every" with option (Never, 10 seconds, 30 seconds, 60 seconds)
  • Changed "Get Log Now!" to "Refresh Now"
  • added < style="overflow: auto; height:400px"> ... < /div > around the logging contents so the user sees the whole page and only the logging content is scrollable.
  
  
  ===== Task Form ===========
  • Made the dialog box a little wider and taller. Maybe 750 x 550.
  • For the "Often" radio button, add "Every" above the options (like this)
  Every
  ( ) _____ seconds
  ( ) _____ minutes
  ( ) _____ hours
  ( ) _____ days
  ( ) weekday (Monday, Tuesday, Wednesday, Thursday, Friday)
  
  
  • Changed the "Start Date" / "Run Time" to be a single line (like "End at")
  Start at [01] [00] [00] [AM] [08/04/2009 ][cal]
  
  
  • Moved the server time to the "When do you want to start?" section
  When do you want to start?
  -------------------------------------------------
  The current time on the server is: 01:34:59 PM
  Start at [01] [00] [00] [AM] [08/04/2009 ][cal]
  
  
  • Changed "Repeat infinite times" to "Repeat forever"
  
  
  Funtionality
  
  • Added the Edit QuartzTask method
  • mail fix (receive only once)
  • Surrounded the pause/start scheduler with a box
  • Number of lines displayed message in the scheduler log
  • Fixed the Email URL

Iteration #12 (due Sunday Aug 16, 2009)

• Tooltips made more consistent.
• resuming a task would work only if the scheduler is in running state
• Added sorting by title to the scheduler manager datagrid.
• close button (x) was made consistent in the create task page.
• made the module user guide.
• blog and project page update.