Archive

Archive for the ‘programming’ Category

What jar is that friggin class in?!?!?!

March 21, 2009 Andrew Hahn Leave a comment

It seems to me that one of the most common tasks I have to preform when packaging a J2EE project is to find out what jar a class I depend on is in.

I have written (multiple times now) a simple perl script that finds all the jars in a given directory and scans them for a file. Thanks to the power of regular expressions you can pass in either com.foo.bar.Class or com/foo/bar/Class which ever is easier to cut-n-paste from the java.lang.NoClassDefFoundError, or similar, stack trace.

#!/usr/bin/perl

$JAR_COMMAND = "jar";
#Uncomment to use fastjar instead
#$JAR_COMMAND = "fastjar";

$class = "";
$lookin = ".";
if(@ARGV < 1) {
    print "USAGE: \n\tclassfinder [dir to search fo jars]
<packageName>\n";
    exit;
} elsif (@ARGV > 1) {
  $lookin = $ARGV[0];
  $class = $ARGV[1];
} else {
  $class = $ARGV[0];
}

foreach $jar (`find $lookin -name '*\.jar'`) {
    chomp $jar;
    foreach $file (`$JAR_COMMAND -tf $jar`) {
        chomp $file;
        if($file =~ $class) {
            print "$jar:$file\n"}
    }
}

The script relies on the standard unix utility find, it works on unix, linux, osx and cygwin. If you have an easy way to do this without using find I’d love to see it.

Adventures in Grails – Running more than one version of your app

March 29, 2008 Andrew Hahn 1 comment

Update: Posted a patch to the Grails JIRA http://jira.codehaus.org/browse/GRAILS-2771

Grails provides a simple versioning system for your application with the app.version property in application.properties. When you build a .war for deployment the file name is given by "${app.name}-${app.version}.war". For me this is great, I frequently need to run multiple versions of an app to support multiple client development branches. However, when I dropped myApp-0.2.war into tomcat’s webapp dir along side myApp-0.1.war the deployment failed. On tomcat 6.0.16 the error is cryptic and far from helpful SEVERE: Error listenerStart. Tomcat 5.5.26 provided a much more useful error message. It turns out that Grails only includes the version number in the .war file name and not in web.xml where it uses only the app name.

To fix this all you need to do is edit the web.xml of one of the versions and replace the app name with something else. Well, thats not very Groovy! To have Grails do this for you it is a simple change to the Package.groovy script that ships with Grails.

The lines (264-266):


Ant.copy(file:"${grailsHome}/src/war/WEB-INF/web${servletVersion}.template.xml", tofile:tmpWebXml)

                Ant.replace(file:tmpWebXml, token:"@grails.project.key@", value:"${grailsAppName}")

get changed to:


Ant.copy(file:"${grailsHome}/src/war/WEB-INF/web${servletVersion}.template.xml", tofile:tmpWebXml, overwrite:true)

Ant.replace(file:tmpWebXml, token:"@grails.project.key@", value:"${grailsAppName}-${grailsAppVersion}")

The overwrite:true in the copy task is necessary so that when the version is changed, web.xml is updated. This will break you if you rely on hand made changes to web.tmp.xml if your ~/.grails directory. Though, you might not want to do things that way anyway.On a side note, you may want to define more environments in grails-app/config/DataSources.groovy if you have changed your app’s DB schema to prevent errors.

Categories: grails, groovy, programming Tags: ,

Adventures in Grails – Debugging

March 11, 2008 Andrew Hahn 1 comment

Update :

It was pointed out to me on the Grails user list that thegrails-debugcommand can be used to do what I posted below. However, the code below is slightly more flexible in that you can control the port, transport and suspend flag with environmental variables.

Working with Grails so far has been great. However, there is no easy way to (that I could find) to fire up Grails and attach a debugger. So I made a few changes to the startup script $GRAILS_HOME/bin/startGrails to handle a debug argument. This is *NIX systems only (including OS X), I haven’t worked with batch files in so long (and have no need for it) that I did not make the changes to $GRAILS_HOME/bin/startGrails.bat. Perhaps someone that uses that other OS to develop on could port the changes.

The script uses the environmental variables:

  • JPDA_ADDRESS #the port to listen on, default 8000
  • JPDA_TRANSPORT #transport to use, default ‘dt_socket’
  • JPDA_WAIT #if ‘y’ then wait for the debugger to attach, default ‘n’

If anyone of these is not set it’s default value is used.

To start the VM for debugging add ‘debug’ as the first arg to the grails command.

grails debug run-app

The changes are in two places. First:


if [ "$1" = "-cp" ] || [ "$1" = "-classpath" ]; then
  CP=$2
  shift 2
fi

ARGUMENTS=$@

becomes


#Test for and setup debug option
if [ ! $JPDA_ADDRESS ]
then
   JPDA_ADDRESS=8000
fi

if [ ! $JPDA_TRANSPORT ]
then
   JPDA_TRANSPORT="dt_socket"
fi

if [ ! $JPDA_WAIT ]
then
   JPDA_WAIT="n"
fi

if [ $# -ne 0 ]
then
   if [ $1 == "debug" ]
   then
      shift
      GRAILS_DEBUG_OPTS="-Xdebug -Xrunjdwp:transport=${JPDA_TRANSPORT},address=${JPDA_ADDRESS},server=y,suspend=${JPDA_WAIT}"
   fi
fi

if [ "$1" = "-cp" ] || [ "$1" = "-classpath" ]; then
  CP=$2
  shift 2
fi

ARGUMENTS=$@

and in the startGrails function:

startGrails() {
  CLASS=$1
  shift
  JAVA_OPTS="-server -Xmx512M $JAVA_OPTS"
  # Start the Profiler or the JVM
  if $useprofiler; then
      runProfiler
  else
  	if [ $# -eq 0 ] ; then         # no argument given
         exec "$JAVACMD" $JAVA_OPTS \
          -classpath "$STARTER_CLASSPATH" \
          -Dprogram.name="$PROGNAME" \
          -Dgroovy.starter.conf="$GROOVY_CONF" \
          -Dgrails.home="$GRAILS_HOME" \
          -Dbase.dir="." \
          -Dtools.jar="$TOOLS_JAR" \
          $STARTER_MAIN_CLASS \
          --main $CLASS \
          --conf "$GROOVY_CONF" \
          --classpath "$CP"
  	else
         exec "$JAVACMD" $JAVA_OPTS \
          -classpath "$STARTER_CLASSPATH" \
          -Dprogram.name="$PROGNAME" \
          -Dgroovy.starter.conf="$GROOVY_CONF" \
          -Dgrails.home="$GRAILS_HOME" \
          -Dbase.dir="." \
          -Dtools.jar="$TOOLS_JAR" \
          $STARTER_MAIN_CLASS \
          --main $CLASS \
          --conf "$GROOVY_CONF" \
          --classpath "$CP" \
          "${ARGUMENTS}"
  	fi
  fi
}

becomes

startGrails() {
  CLASS=$1
  shift
  JAVA_OPTS="-server -Xmx512M $JAVA_OPTS"
  # Start the Profiler or the JVM
  if $useprofiler; then
      runProfiler
  else
  	if [ $# -eq 0 ] ; then         # no argument given
         exec "$JAVACMD" $JAVA_OPTS \
          -classpath "$STARTER_CLASSPATH" \
          "$GRAILS_DEBUG_OPTS" \
          -Dprogram.name="$PROGNAME" \
          -Dgroovy.starter.conf="$GROOVY_CONF" \
          -Dgrails.home="$GRAILS_HOME" \
          -Dbase.dir="." \
          -Dtools.jar="$TOOLS_JAR" \
          $STARTER_MAIN_CLASS \
          --main $CLASS \
          --conf "$GROOVY_CONF" \
          --classpath "$CP"
  	else
         exec "$JAVACMD" $JAVA_OPTS \
          -classpath "$STARTER_CLASSPATH" \
          "$GRAILS_DEBUG_OPTS" \
          -Dprogram.name="$PROGNAME" \
          -Dgroovy.starter.conf="$GROOVY_CONF" \
          -Dgrails.home="$GRAILS_HOME" \
          -Dbase.dir="." \
          -Dtools.jar="$TOOLS_JAR" \
          $STARTER_MAIN_CLASS \
          --main $CLASS \
          --conf "$GROOVY_CONF" \
          --classpath "$CP" \
          "${ARGUMENTS}"
  	fi
  fi
}
Categories: grails, groovy, programming

Adventures in Grails – WS-Security Part 2

March 11, 2008 Andrew Hahn 2 comments

This post builds on the previous post WS-Secutiry Part 1 by adding inHandlers that populate an acegi security context.

Integrating acegi

It turned out that initial integration of acegi with xfire + WSS was even easier than hooking up WSS for xfire in Grails. Though I can’t claim much original work here. In his blog Propagating Acegi’s Security Context in a WSS UsernameToken SOAP Header via XFire using wss4j Michael Vorbuger provides everything necessary to get it working.

To get it running I added the three classes from Michael’s code acegi-ws-security-xfire-example to the appropriate packages in src/java/ in my Grails app.

  • ch.vorburger.acegiwss.server.PasswordHandler
  • ch.vorburger.acegiwss.server.ForgivingWSS4jInHandler
  • ch.vorburger.acegiwss.server.ValidateUserTokenHandler

and changed the inhandlers to use these classes in XfireGrailsPlugin.groovy.


"xfire.passHandler"(ch.vorburger.acegiwss.server.PasswordHandler) { bean ->
        }

"xfire.DOMhandler"(org.codehaus.xfire.util.dom.DOMInHandler) { bean ->
        }

"xfire.WSS4JHandler"(ch.vorburger.acegiwss.server.ForgivingWSS4jInHandler) {
     properties = ["passwordCallbackRef":ref("xfire.passHandler"),
                    "action":"UsernameToken"]
        }

"xfire.ValidateUserTokenHandler"(ch.vorburger.acegiwss.server.ValidateUserTokenHandler) {}

That makes the SecurityContext avaliable in the service. To see it work I paraphrased Michael’s example in the test service.

import org.acegisecurity.Authentication
import org.acegisecurity.context.SecurityContextHolder

class TestService {

    static expose=['xfire']

    boolean transactional = true

    String serviceMethod() {

        Authentication auth = SecurityContextHolder.getContext().getAuthentication();
		 if (auth == null || auth.getName() == null || auth.getName().length() == 0) {
			 // In a real service, this would be a proper SOAP Fault, NOT an IllegalArgumentException
			 throw new IllegalArgumentException(NOAUTH_FAULT_TEXT);
		 }

       return "You did it ${auth.getName()}!!!"
    }
}

Thats it!

Categories: grails, groovy, java, programming Tags:

Adventures in Grails – WS-Security Part 1

March 1, 2008 Andrew Hahn 4 comments

The next step in rewriting the application was to secure the web services with WS-Security. In this post I get a grails version of the xfire wss example of User Token Authentication up and running. To do this I use (of course) the grails xfire plugin.

After creating a grails project and installing the xfire plugin, the first thing to do is to configure the inHandlers:

to do this add the following to the doWithSpring closure in XfireGrailsPlugin.groovy


"xfire.passHandler"(org.codehaus.xfire.demo.PasswordHandler) { bean ->
        }

"xfire.DOMhandler"(org.codehaus.xfire.util.dom.DOMInHandler) { bean ->
        }

"xfire.WSS4JHandler"(org.codehaus.xfire.security.wss4j.WSS4JInHandler) {
    properties = ["passwordCallbackRef":ref("xfire.passHandler"),
                  "action":"UsernameToken"]
        }
"xfire.ValidateUserTokenHandler"(org.codehaus.xfire.demo.ValidateUserTokenHandler) {
        }

ValidateUserTokenHandler and PasswordHandler are part of the example code distributed with xfire. I just copied them into the correct package in src/java/ in this simple grails app. And then add the inHandlers to the org.grails.xfire.ServiceBean


inHandlers = [ref("xfire.DOMhandler"),
              ref("xfire.WSS4JHandler"),
              ref("xfire.ValidateUserTokenHandler")]

See the complete listing for doWithSpring at the end of this post. Now any service you expose with xfire will require (and print) a username and password in a WSS UsernameToken header. The simple service I used to test this is:


class TestService {

    static expose=['xfire']

    boolean transactional = true

    String serviceMethod() {
       return "You did it!!!"
    }
}

I use soapUI to test, here is the request it generated:


      foo
      bar
      2008-03-01T19:49:03.627Z

This is obviously not a perfect solution. You may not want to secure all the web services in your project or at least not all in the same way. After I finish with this project I will have a more general solution to contribute to the grails xfire plugin.But first, I need to do something with the user credentials I am now receiving. Next up, integrating with acegi through the grails acegi plugin

Full doWithSpring listing:


def doWithSpring = {

        "xfire.serviceRegistry"(org.codehaus.xfire.service.DefaultServiceRegistry) { bean->
            bean.getBeanDefinition().setSingleton(true)
        }

        "xfire.transportManager"(org.codehaus.xfire.transport.DefaultTransportManager){ bean->
            bean.getBeanDefinition().setSingleton(true)
            bean.getBeanDefinition().setInitMethodName("initialize")
            bean.getBeanDefinition().setDestroyMethodName("dispose")
        }

        "xfire"(org.codehaus.xfire.DefaultXFire,
                 ref("xfire.serviceRegistry"),
                 ref("xfire.transportManager")) { bean ->
            bean.getBeanDefinition().setSingleton(true)
        }

        "xfire.typeMappingRegistry"(org.codehaus.xfire.aegis.type.DefaultTypeMappingRegistry){ bean ->
            bean.getBeanDefinition().setSingleton(true)
            bean.getBeanDefinition().setInitMethodName("createDefaultMappings");
        }

        "xfire.aegisBindingProvider"(org.codehaus.xfire.aegis.AegisBindingProvider,
            ref("xfire.typeMappingRegistry")) { bean ->
            bean.getBeanDefinition().setSingleton(true)
        }

        "xfire.serviceFactory"(org.codehaus.xfire.service.binding.ObjectServiceFactory,
            ref("xfire.transportManager"), ref("xfire.aegisBindingProvider")) { bean ->
            bean.getBeanDefinition().setSingleton(true)
        }

        "xfire.servletController"(org.codehaus.xfire.transport.http.XFireServletController,
            ref("xfire")) { bean ->
            bean.getBeanDefinition().setSingleton(true)
        }

        "grails.xfire"(org.grails.xfire.ServiceFactoryBean, "grails.xfire") { bean ->
            bean.getBeanDefinition().setInitMethodName("initialize")
            transportManager = ref("xfire.transportManager")
            grailsApplication = ref("grailsApplication", true)
        }

        "xfire.passHandler"(org.codehaus.xfire.demo.PasswordHandler) { bean ->
        }

        "xfire.DOMhandler"(org.codehaus.xfire.util.dom.DOMInHandler) { bean ->
        }

        "xfire.WSS4JHandler"(org.codehaus.xfire.security.wss4j.WSS4JInHandler) {
            properties = ["passwordCallbackRef":ref("xfire.passHandler"),
                          "action":"UsernameToken Timestamp"]
        }

        "xfire.ValidateUserTokenHandler"(org.codehaus.xfire.demo.ValidateUserTokenHandler) {}

        if(application.serviceClasses) {
            application.serviceClasses.each { service ->
                def serviceClass = service.getClazz()
                def exposeList = GrailsClassUtils.getStaticPropertyValue(serviceClass, 'expose')
                if(exposeList!=null && exposeList.contains('xfire')) {
                    def sName = service.propertyName.replaceFirst("Service","XFire")
                    //
                    "${sName}"(org.grails.xfire.ServiceBean){
                        //
                        xfire = ref("xfire")
                        //
                        serviceBean = ref("${service.propertyName}")
                        //
                        serviceClass = service.getClazz()
                        //
                        serviceFactory = ref("grails.xfire")

                        inHandlers = [ref("xfire.DOMhandler"),
                                ref("xfire.WSS4JHandler"),
                                ref("xfire.ValidateUserTokenHandler")]
                    }
                }
            }
        }
    }

Adventures in Grails

February 8, 2008 Andrew Hahn Leave a comment

I was recently face with the need to rewrite an entire (ok, 3/4 finished) web application from scratch. The app provided SOAP services for external clients, had integrated search (in both the web interface and WS), restricted user access and search results based on profile and needed to be very scalable. We had previously used AppFuse to build the app, so it was using Spring, Hibernate, Lucene, Compass, Acegi,WebWork, CXF (after upgrading XFire) using WS-Security, etc… I needed to maintain the robustness and scalability of those tools while increasing the flexibility and prototyping efficiency. Enter Grails.

I have been using Groovy to do my scripting tasks for the last 4-5 months. I love it. I can write a script with the same ease and power as I can in Perl or Python and have direct access to my domain model and Hibernate DAOs. Once its working, it is no great leap to wire it up in Spring and have it run on a timer in my app. Groovy alone was enough to get me to give Grails a shot. Once there the near instant ability to create a web UI for CRUD with just three total lines of controller code had me hooked. The fact that plugins for XFire, Acegi and Compass/Lucene already exist make it appear to be the perfect platform for what I need to accomplish. Sure the plugins are young, and I may have to do some work to get some of them to the level I need, but that is the beauty of open source.

I plan to blog my experiences as I put the app together. Stay tuned to see how it goes.

Getting Started

I found a wealth of information to get me started using Grails.

Web Services (XFire)

Once I had an idea of how to start building the app with Grails I figured I would get the Web Services running with the XFire plugin. Using the getting started section of the XFire plugin page I quickly had a simple service up and running.
grails install-plugin xfire
grails create-service Test

Add the expose property to the service.
static expose=['xfire']

and
grails run-app
fire off a test SOAP message and IT WORKS!!!!!
Cool, now I need some data in the DB to see how that looks. Enter it by hand? No, I’ll create a controller and use the web UI! OK, now run the app and fire off a test message….

Pttthhhbbbbb!?!?!?!

Unexpected EOF in prolog at [row,col {unknown-source}]: [1,0]

What the….? Adding a controller breaks the web services? My first wrinkle!
Turns out that you have to pay attention to the “Upgrade to Grails 0.5(URL Mapping)” section on the XFire plugin page too. If you don’t add the following constraint to the Grails URL mapping the controller framework mangles the request.

static mappings = {
          "/$controller/$action?/$id?"{
              constraints {
                        controller(matches:/.*[^(services)].*/)
                  }
          }
}

Thanks to Jason Morris-5 on the Grails mailing list, he worked through the same issue just a few weeks earlier and found the answer.Well, can’t beat an active mailing list. Another plus. All in all a very good experience so far.

Getting the sorted and paginated contents of a collection in Hibernate

January 9, 2008 Andrew Hahn Leave a comment

Normally getting the contents of a collection from Hibernate is as easy as retrieving the parent object and calling ‘parent.getCollection()’. But what if you have no interest in the parent and the result set could be huge? I found some help in the hibernate.org ‘Tips and Tricks’ section, but not quite what I was looking for. The solutions given there used filters or pre-defined sorting on the collection.
Sorting
Pagination

The following query retrieves the contents of a collection sorts and paginates them:

public List<Category> getSubCategories(Long categoryId, int start, int size) {
        Session session = getHibernateTemplate().getSessionFactory().getCurrentSession();
        String queryString = "select subCategory from SubCategory as subCategory, Category as category where category.id = " + categoryId + " and subCategory in elements(category.subCategories) order by subCategory.name";
        List<Category> subCategories = getList(session.createQuery(queryString), start, size);
        return subCategories;
    }

I’m using Spring, but any way of getting a Hibernate session should work.

Here Category and SubCategory look like:

public class Category {
    protected Long id;

    protected String name;
    protected Set<SubCategory> subCategories = new HashSet<SubCategory>();

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public Set<SubCategory> getSubCategories() {
        return subCategories;
    }

    public void setSubCategories(Set<SubCategory> subCategories) {
        this.subCategories = subCategories;
    }
}

public class SubCategory {
    protected Long id;
    protected String name;

    public Long getId() {
        return id;
    }

    public void setId(Long id) {
        this.id = id;
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

}

I have intentionally left out the xdoclet tags I use to configure hibernate. The mapping is as simple as it gets.

Simple script for batching tasks in a multi-core/CPU envionment

November 16, 2007 Andrew Hahn Leave a comment

Today I was tasked with keeping all four processors of a machine busy with CPU intensive processes (in this case transcoding video). I wrote a quick and dirty Perl script to keep running ffmpeg as input files became available, but never run more that four at a time. The script looks for .avi files in the directory from which it was executed. It then maintains a stack of file names, forks a child process for each file and moves it when done.

#!/usr/bin/perl

use POSIX ":sys_wait_h";
use Errno qw(EAGAIN);

$secs = 5;

$numProcs = 0;
$maxProcs = 4;

$fileExt = ".avi";

@files = ();
%was_processed = {};

while (1) {
    #print "numProcs = $numProcs\n";
    if($numProcs < $maxProcs) {
	$numProcs++;
	execNext();
    }
    my $reaped = reapNext();
    if($reaped > 0) {
	$numProcs--;
    } elsif ($numProcs == $maxProcs) {
	#print "Sleeping...\n";
	sleep $secs;
    }
}

sub reapNext {
    return waitpid(-1, &WNOHANG);
}

sub execNext() {
    my $file;
    WAIT_FOR_NEW_FILE: {
	getFiles();

	while($was_processed{$file = pop @files}) {}

	if(!defined $file) {
	    print "Waiting for new $fileExt files...\n";
	    sleep $secs;
	    redo WAIT_FOR_NEW_FILE;
	} else {
	  $was_processed{$file} = 1;
	}
    }
    doFork($file);
}

sub getFiles {
    if($#files == -1) {
	push @files, <*$fileExt>;
    }
}

sub doFork {
    my $file = @_[0];
  FORK: {
      if($pid = fork) {
	  #We are in the orig process
	  return;
      } elsif (defined $pid) {
	  #We are in a child process
	  doChild($file);
      } elsif ($! == EAGAIN) {
	  #Something is wonky, but we should be able to recover...
	  sleep 5;
	  redo FORK;
      } else {
	  die "Can't fork: $!\n";
      }
    }

}

sub doChild {
    my $file = @_[0];
    (my $name, my $junk) = split /$fileExt/, $file;
    print "Ripping $file\n";

    system "(ffmpeg -i \"$file\" -acodec libfaac -ab 128k -ac 2 -vcodec libx264 -b 1500k -bf 2 -f mpegts -aspect 4:3 -y \"$name\.ts\" 2>&1) > \"$name\.log\"\n";
    system "(mv \"$file\" ripped  2>&1) >> \"$name\.log\"\n";

    #unlink "\"$name\.ts\.mpeg\"";#delete the ts file
    #sleep $secs + rand $secs;

    exit;
}