Automated black box testing of commandline tools tends to be (but should not be) a bit of a black art. There is a tendency among Java developers to ignore otherwise useful tools and ideas that aren't prepackaged for either Ant or JUnit, and even if I'm working with commandline tools implemented in Java, my preference is for expect, a tool specifically for automating interactive commandline applications. The language is based on Tcl, which isn't too unreasonable to pick up given a few examples, and expect comes with most *nixes (including Mac OS). (It is also available for Windows.) The point of expect is that you can use it to spawn one or more processes, send them text, and tell the tool what to expect (Get it...?) in the way of output. Conveniences like expected response timeouts, regular expressions, and basic control flow (via Tcl) are all natively available in expect.
There are plenty of more or less instructive examples available, but for a concrete use case, an expect script could download and unpack a fresh Tomcat, drop and refresh a database, launch the Tomcat, deploy a web application by copying a WAR into place, and watch the Tomcat logs to ensure that nothing unexpected happens while the same script runs a test client against the web application from another machine. It's also reasonable to write expect scripts that simulate the examples provided in user documentation as part of an integration test suite. (Essentially, expect can do whatever you might normally do with a bunch of shell windows open on your desktop, including logging into other servers and interacting with the remote shell.)
Add a comment.
