Sunday, September 21, 2008

Answering mail #10

Vanina Beraudo wrote:

Hello
My english isn´´t good, but i try speak with you.
Do you know if i can integrate RCP/RAP with XML UI?
Where can i get you demo source sample?
I was looking your blog.It´´Is very interesting.
I will like see runing the demo but in RAP target
Thanks you

Hello Vanina,

I am afraid that the Rich Ajax Platform / Rich Client Platform still require more Java programming than XML specification. I wouldn't be surprised if that would change though. For example, the Eclipse Modeling Framework and the Graphical Modeling Framework can be used to create models from XML Schemas. A powerful application based on EMF/GMF is the SOA Tools BPMN Modeler. (Actually I think this still uses objects in the background, but it is nice anyway ;-))

If you want to create rich AJAX UIs using XML technology, for now, Orbeon is a nice choice. But I can imagine it would be nice to see something similar based on Eclipse technologies, like OSGi. And something that works on both the client and the server side like RAP and RCP. (By the way, XForms do work both on the client and the server side if you have the right plugins... (edit))

The demo I created uses the Jython scripting language to avoid Java development. The Eclipse version on which it was based is a bit outdated now. But you can find it over here.

Sorry I cannot be of any help. Good luck with your efforts on RAP and RCP!

Kind regards,

Adriaan.

Sunday, September 14, 2008

Scala: a misguided approach to XML

In 2001, when Scala was conceived, it might have seemed like a good idea to integrate XML capabilities in a new programming language. XML was emerging technology, Java libraries for XML processing were still young and it seemed like things could be simplified in comparison to DOM and SAX.

But even back then, some XML technologies were already quite powerful and well capable of handling complex problems. The best examples being XPath 1.0 (16 November 1999) and XSLT 1.0 (also 16 November 1999) Their proper usage just wasn't well known yet. Early versions of Cocoon already showed the first ideas of using their power by combining them into XML pipelines.

The fact is: it is not 2001 anymore. Technologies are even more mature. XPath and XSLT are at version 2.0 and XQuery 1.0 is officially a W3C recommendation (all 23 January 2007). While the 1.0 versions were already powerful, the 2.0 versions and XQuery are a lot more mature and backed up by proper research and documentation in a larger set of Recommendation documents.

At the same time in 2008, new dynamic languages are gaining popularity as Java is in an existential crisis heading towards version 7.0. And Scala is one of these languages a language that is becoming more and more popular. And it integrates XML capabilities.

"What could be wrong with adding XML capabilities?" - I hear you ask. This is an XML related weblog after all. What could be wrong with adding XML support?

If you're asking this question, then obviously you haven't seen Scala yet. Or you did, but you're not familiar with proper XML technologies (yet).

A few years ago, it was common to externalize configuration parameters of Java programs XML descriptors. In more recent years, under the pressure of dynamic languages, it became more common to integrate configurations into the code using Annotations and hard-coded values.

And now, using Scala, you can internalize the XML itself, so it becomes easier to output the hard-coded values like this: (Examples taken from scala.xml)

/* examples/phonebook/embeddedBook.scala */
package phonebook

object embeddedBook {

val company = <a href="http://acme.org">ACME</a>
val first = "Burak"
val last = "Emir"
val location = "work"

val embBook =
<phonebook>
<descr>
This is the <b>phonebook</b> of the
{company} corporation.
</descr>
<entry>
<name>{ first+" "+last }</name>
<phone where={ location }>+41 21 693 68 {val x = 60 + 7; x}</phone>
</entry>
</phonebook>;

def main(args: Array[String]) =
Console.println( embBook )

}

But that is not the worst example in this Scala XML manual. Nooooooo!

My real disappointment in this manual started when I read this sentence:

For new developments, it is more straightforward to use the more convenient Scala API rather than the cumbersome XSLT syntax, or (if it really must be XSLT), some Java library.

Personally I do not think of XSLT as cumbersome at all, but I am always open to new ideas and prove my own ideas wrong. So let's take a look at the example for XML transformations that the Scala XML manual came up with: (Examples taken from scala.xml)

object transform {
import scala.xml._ ;
import scala.xml.dtd._ ;
import org.xml.sax.InputSource ;

/* a former version of Scala used regular expression patterns, like
* in the following code. In the future, we plan to reactivate some
* well-behaved regular expressions again
// gimmick: text replacement "scalac" => <code>scalac</code>
def cook(s: String): Seq[Node] = cook1(s) ;
def cook1(s: Seq[Char]):List[Node] = s match {

case Seq( a @ _*, 's','c','a','l','a','c', b @ _* ) =>

Text(cook2( a )) :: <code>scalac</code> :: cook1( b )
case _ => List( Text( cook2( s )))
}
def cook2(s: Seq[Char]): String = {
val r = new StringBuffer();
s.foreach { c:char => val _ = r.append(c); };
r.toString()
}
*/

def transform1(ns: Iterable[Node]): Seq[Node] = {
val zs = new NodeBuffer();
for(val z <- ns) { zs &+ transform( z ) }
zs
}

/** this functions holds "templates" that transform nodes of an input tree
* into an iterable representation of a sequence of nodes of the output
* tree.
*
* It is ok to return a single node, since each node is at the same
* time a singleton sequence. Likewise, the pattern variable x will be
* of type Seq[Node], although here it is always binding a single node.
*/
def transform(n: Node):Iterable[Node] = n match {
case x @ <article>{ ns @ _ * }</article> =>
<source active="ant" title={ (x \ "title" \ "_").toString() }>
<header>
<author>Burak Emir</author>
<keywords>Scala4Ant</keywords>
<style type="text/css"></style>
</header>
<content>
<title><scala/> Ant Task</title>
{ transform1( x \ "_" ) }
</content>
</source>
case x @ <sect1>{ _* }</sect1> =>
<section>{ transform1( x \ "_" ) }</section>
case x @ <title>{ _* }</title> =>
<h>{ x \ "_" }</h>
case x @ <para>{ _* }</para> =>
<p>{ transform1( x \ "_" ) }</p>
case x @ <itemizedlist>{ _* }</itemizedlist> =>
<ul>{ transform1( x \ "_" ) }</ul>
case x @ <listitem>{ _* }</listitem> =>
<li>{ transform1( x \ "_" ) }</li>
case x @ <constant>{ _* }</constant> =>
// an xml:group is a sequence that appears to the scala type system
// as a single node. Here it is used to append a text node with a space
<xml:group><code>{ x \ "_" }</code> </xml:group>
case x @ <programlisting>{ _* }</programlisting> =>
<pre>{ x \ "_" }</pre>
case Elem(namespace, label, attrs, scp, ns @ _*) =>
Elem(namespace, label, attrs, scp, transform1( ns ):_* )
case z =>
z
};

def main(args:Array[String]) = {
if( args.length == 1 ) { // must have one arg
object ConsoleWriter extends java.io.Writer {
def close() = {}
def flush() = Console.flush
def write(cbuf:Array[char], off:int , len:int ): unit = {
var i = 0
while(i < len)
Console.print(cbuf(off + i))
}
}

val src = XML.load(new InputSource( args( 0 ))); //use Java parser

// transform returns an iterable, but we now it is a singleton
// sequence, so we get its first element
val n = transform( src ).elements.next
val doctpe = DocType("html",PublicID("-//W3C//DTD XHTML 1.1//EN","../default.dtd"), Nil)

/** write document to console, with encoding latin1, xml declaration
* and doctype
*/
XML.write(ConsoleWriter, n, "iso-8859-1", true, doctpe)

}
else error("need one arg");
}
}

Talking about cumbersome! The real problem of this code might not even be in the notation. That is just a matter of getting used to Scala syntax. The problem is in its lack of maintainability. This kind of code assumes that XML documents always follow a fixed pattern. And this idea is confirmed by an earlier section of the manual.

The manual (surprisingly adequately) makes a separation between XML that can be considered text with markup and XML that models data structures in a hierarchical tree format. But (disappointingly quickly) after that, the text starts displaying the first signs of a lack of knowledge of XML technologies:
[...]
There are several points of view that can be taken:

1. XML is regarded as text. We ignore the tree structure completely. Some text/regular expression search is used to retrieve or manipulate information. This can get you quite far for small tasks. Go away, use perl :-)
[...]

What ever happened to XPath?

The Scala designers must have suffered from the Not Invented Here Syndrome and defined their own Scala equivalent of XPath, like this: (Examples taken from scala.xml)

package bib;

object bib {

import scala.xml.{Node,NodeSeq};
import scala.xml.PrettyPrinter;

val biblio =
<bib>
<book>
<author>Peter Buneman</author>
<author>Dan Suciu</author>
<title>Data on ze web</title>
</book>
<book>
<author>John Mitchell</author>
<title>Foundations of Programming Languages</title>
</book>
</bib> ;

val pp = new PrettyPrinter(80, 5);

def main(args:Array[String]):Unit = {
Console.println( pp.formatNodes( biblio \ "book" \ "title" ));

// prints
// <title>Data on ze web</title><title>Foundations ...</title>

Console.println( pp.formatNodes( biblio \\ "title" )); // prints the same

Console.println( pp.formatNodes( biblio \\ "_" )); // prints node and all descendant

Console.println( pp.formatNodes( biblio.descendant_or_self )); // prints the same


}
}

Judging from the list of references under the manual, it is more likely that this XPath-surrogate is modeled after XPath 1.0 than XPath 2.0. And judging from the opinions of the author, it seems unlikely that Scala will adopt XML standards in a proper way any time soon.

Probably the most fundamental critique on this approach is that Scala is mixing concerns that are usually separated from each other. Usually, the challenge of software development is to separate concerns that are otherwise mixed. So why would anyone start working the other way around, working in the wrong direction?

In my opinion, the answer to the current Java 7.0 controversy is not in dynamic languages like Scala. I think it is more likely that the answer is in proper standards like XPath 2.0, XQuery 1.0 and XSLT 2.0. Over the years since 1999, these technologies matured into powerful programming platforms themselves. As I am researching these specs more and more, it turns out they are even a lot more interesting than I thought before I started. And I thought I knew these technologies. So a lot more details on these technologies will follow in later posts!

Thursday, September 11, 2008

People: Front-End vs Back-End

A developer who never misses a deadline might be a good developer. He might reduce the quantity or quality of his work to the bare minimum to do so. Quite possibly, the management team values such choices in favor of the deadline more than an increased quality released after the deadline. On the long run, a pile of bare minimum solutions becomes unstable and completely ignores the wishes of the end user. This means that end users may be spending a significant part of their day being frustrated to work with an inefficient application.

In many cases, the management team makes it even worse by purchasing large software systems intended to improve business processes, resource planning and accounting. From management perspective, these products deliver results. On the other hand though, these kinds of products pay little attention to the end user in most cases. 

But there may be good news on the horizon: the latest trend is an increasing attention for usability and rich user interfaces on the internet. Sometimes this is at the cost of standardization, but the real front-end developer knows how to handle such issues properly.

Despite this trend, there remains a major part of the development community who consciously choose to ignore the end user perspective. They expect the end user to adjust to the application instead with intensive training instead of meeting their demands by optimizing usability.

Both kinds of developers can cooperate if they divide the tasks: one works on the user interface and the other implements the back end. The question is whether a poor front end programmer can make a good back end programmer. I have some doubts there. The complexity of the object models to create GUIs is outstanding training material to improve your object orientation skills.

The problem of creating rich clients became worse in the past few years, because the current generation of developers was raised making classic web applications. The majority of young developers has absolutely no experience developing real client side applications providing rich user interfaces by default.

There is good hope that the RIA trend will resolve this issue!

Tuesday, September 09, 2008

Other voices: Unit testing XSLT and XQuery

Just like Java developers use JUnit and .NET developers NUnit, it might be a good idea for XML developers to test their XSLTs and XQueries with a specialized XSLT Unit test suite. The idea is not new, but the implementations are improving. Instead of testing one whole XSLT, it is also possible to test parts of XSLTs. And this using the technology of preference: XSLT! (what else?)

A noteworthy implementation is written by Florent Georges.

Friday, September 05, 2008

Simple Outline XML

For developers who don't like XML, it might be good to know that even developers who do like XML only use it when it is appropriate. For example, XML technologies like XQuery (Recommendation) and RELAX NG Compact Notation have a notation that looks more like script than like XML. But if you want it to be XML, there still is XQueryX (Working Draft) and RELAX NG XML Notation.

For some XML documents, especially tree structures, XML is not always the best human readable format. For these purposes, there are technologies like Simple Outline XML (SOX) which produce XML equivalent notations for documents that can easily be translated back to the original XML format. 

The website on SOX provides this example:

stylesheet> 
    xmlns=http://www.w3.org/1999/XSL/Transform
    version=1.0
    template> 
        match=node()
        copy> 
            apply-templates>
                select=node()>

Tuesday, September 02, 2008

Wish list: Automatic drawings

Working with tools like Subversion and CVS in complex environments, it is easy to get a lot of parallel branches with separate developments and these become hard to maintain very quickly. In general, the best advice is to minimize the number of branches. But in some special cases, you just don't want to because you want to keep your options open. In such cases, you need to keep a proper administration of what happens where. A drawing usually helps, but making drawings takes a lot of time that can be spent on useful things....

Wouldn't it be nice if you could translate an XML file like this....

<?xml version="1.0" encoding="UTF-8"?>
<roadmap main="trunk">
  <branch id="f3" t="7" len="3">
    <label>F 3</label>
    <start level="f2"/>
    <text t="1">tst</text>
    <merge t="2" level="f2"/>
  </branch> 
  <branch id="f2" t="5" len="8">
    <label>F 2</label>
    <start level="f1"/>
    <text t="5">tst</text>
    <merge t="6" level="integration"/>
  </branch>  
 <branch id="f1" t="2" len="5">
    <label>F 1</label>
    <start level="trunk"/>
    <text t="3">tst</text>
    <merge t="4" level="integration"/>
  </branch>
  <branch id="integration" t="4" len="10">
    <label>Integration</label>
    <start level="trunk"/>
    <text t="5">tst</text>
    <text t="6">acc</text>
    <merge t="8" level="trunk"/>
  </branch>
  <branch id="trunk" t="0" len="20">
    <label>TRUNK</label>
  </branch>
  <branch id="release_march" t="14" len="5">
    <label>RL 0803</label>
    <start level="trunk"/>
    <text t="3">tst</text>
    <merge t="4" level="trunk"/>
  </branch>
</roadmap>

...to a picture like this....


...only using code like this?

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:svg="http://www.w3.org/2000/svg" version="1.0">
  <xsl:template match="//roadmap">
    <xsl:variable name="main" select="@main"/>
    <xsl:variable name="width" select="(branch[@id=$main]/@len + branch[@id=$main]/@t) * 3 + 2"/>
    <xsl:variable name="height" select="count(branch) * 5 + 4"/>
    <svg:svg width="{$width}cm" height="{$height}cm" viewBox="0.000 0.000 {$width} {$height}">
      <xsl:apply-templates select="branch">
        <xsl:with-param name="merge" select="'true'"/>
      </xsl:apply-templates>
      <xsl:apply-templates select="branch">
        <xsl:with-param name="merge" select="'false'"/>
      </xsl:apply-templates>
    </svg:svg>
  </xsl:template>
  <xsl:template match="branch">
    <xsl:param name="merge"/>
    <xsl:variable name="start" select="@t * 3 + 1"/>
    <xsl:variable name="end" select="$start + @len * 3"/>
    <xsl:variable name="pos" select="count(./preceding-sibling::*) + 1"/>
    <xsl:variable name="y" select="$pos * 5"/>
    <svg:line x1="{$start}" y1="{$y}" x2="{$end}" y2="{$y}" stroke="#000000" stroke-width="1.000"/>
    <xsl:choose>
      <xsl:when test="$merge='true'">
        <xsl:apply-templates select="merge">
          <xsl:with-param name="pos" select="$pos"/>
          <xsl:with-param name="y" select="$y"/>
        </xsl:apply-templates>
      </xsl:when>
      <xsl:otherwise>
        <xsl:apply-templates select="label">
          <xsl:with-param name="start" select="$start"/>
          <xsl:with-param name="pos" select="$pos"/>
          <xsl:with-param name="y" select="$y"/>
        </xsl:apply-templates>
        <xsl:apply-templates select="start">
          <xsl:with-param name="start" select="$start"/>
          <xsl:with-param name="pos" select="$pos"/>
          <xsl:with-param name="y" select="$y"/>
        </xsl:apply-templates>
        <xsl:apply-templates select="text">
          <xsl:with-param name="pos" select="$pos"/>
          <xsl:with-param name="y" select="$y"/>
        </xsl:apply-templates>
      </xsl:otherwise>
    </xsl:choose>
  </xsl:template>
  <xsl:template match="label">
    <xsl:param name="start"/>
    <xsl:param name="pos"/>
    <xsl:param name="y"/>
    <svg:text x="{$start + 1}" y="{$y + 0.5}" fill="#FFFFFF" text-anchor="start" font-size="0.80" font-family="sans" font-style="normal" font-weight="400">
      <xsl:value-of select="."/>
    </svg:text>
  </xsl:template>
  <xsl:template match="text">
    <xsl:param name="pos"/>
    <xsl:param name="y"/>
    <xsl:variable name="start" select="(../@t + @t)* 3 + 1"/>
    <svg:text x="{$start + 1}" y="{$y + 0.5}" fill="#FFFFFF" text-anchor="start" font-size="0.80" font-family="sans" font-style="normal" font-weight="400">
      <xsl:value-of select="."/>
    </svg:text>
  </xsl:template>
  <xsl:template match="start">
    <xsl:param name="start"/>
    <xsl:param name="pos"/>
    <xsl:param name="y"/>
    <xsl:variable name="level" select="@level"/>
    <xsl:variable name="endpos" select="count(../../branch[@id=$level]/preceding-sibling::*) + 1"/>
    <xsl:variable name="endy" select="$endpos * 5"/>
    <svg:path stroke="#000000" fill="none" stroke-width="1.000" d="M {$start - 5},{$endy} C {$start - 2},{$endy} {$start - 3},{$y} {$start},{$y} "/>
  </xsl:template>
  <xsl:template match="merge">
    <xsl:param name="pos"/>
    <xsl:param name="y"/>
    <xsl:variable name="start" select="(../@t + @t)* 3 + 1"/>
    <xsl:variable name="level" select="@level"/>
    <xsl:variable name="endpos" select="count(../../branch[@id=$level]/preceding-sibling::*) + 1"/>
    <xsl:variable name="endy" select="$endpos * 5"/>
    <svg:line x1="{$start}" y1="{$y}" x2="{$start}" y2="{$endy}" stroke="#000000" stroke-width="1.000"/>
    <svg:line x1="{$start}" y1="{$y}" x2="{$start}" y2="{$endy}" stroke="#FFFFFF" stroke-width="1.000" stroke-dasharray="0.10,0.10"/>
  </xsl:template>
</xsl:stylesheet>


Oh wait! You can! The only thing is that you need to convert the SVG file to a PNG file, but that can be done using several tools!

Monday, September 01, 2008

People: Process vs Information

A developer may think that his favorite solution worked for 3, 5 or 12 clients in a row and therefore is the universal best solution to any problem. Such developers fail to see that these similarities probably resulted from the fact that he is hired as a specialist by the exact companies requiring his specific knowledge. It is very well possible that 98% of the other companies in the world have different requirements, but the specialist never learns about them, because these companies hire other specialists with different experience. And quite likely, these other specialists may also be under the illusion that their solution is the silver bullet to every problem.

For example: There are many discussions about the question whether RESTful web services are better than SOAP based services or the other way around. And in most cases, the answer to this question depends on the problem domain. 

My idea is that both opinions can be explained by the different preferences of two kinds of developers: One of them likes to present large amounts of information from several data repositories and integrate them into an accessible web site. These people tend to work on projects creating public internet sites, requiring high quality templates and appropriate data formats storing and exchanging data.

His counterpart has spent most of his life creating rich web applications running on the local intranet within the walls of a single company. Typically such developer is much less interested in data formats because of the limited reach of the application. Far more important for him, is business process modeling, case management, workflow, security and data integrity.

The information specialist and the process expert may clash with each other for several reasons:

The information specialist spends a lot of time on details to get semantics, maintainability and formatting right. The process specialist may consider that a waste of time, because such things can be generated using his tools. The other way around, the information specialist considers generated code as poor quality. Quite likely, the process specialist is more interested in business modeling than information modeling, cares less about code quality and is better capable of meeting business requirements in a short period of time.

There are roughly two things you can do with these two developer types:

- Separate them and let them work on the assignments they are good at (or prefer)...

Or more interesting:

- Separate them and assign them to each other's work in order to learn to see a bigger picture than they are used to!