The Openness Dilemma
A while ago (circa 2000), Rob Pike wrote a rant about how systems software research is becoming irrelevant, due to the difficulty in getting new operating systems adopted.
Since then, he’s gone on to work at Google, where systems software research is the lifeblood of one of the (if not the) most important Internet sites in history. And he’s happily doing plenty of systems software research that’s become fundamental to the company’s operations.
So his original concern about the irrelevance of operating system research got effectively sidelined, because the action moved from single-machine operating systems to wider distributed systems, especially as used at Google. And Google is as good as anyone at turning research ideas into production practice.
Meanwhile, Jim Waldo at Sun last year wrote another paper — a little less of a rant, but not much — about how systems software design is suffering greatly, largely from the lack of opportunity to learn from experience. Waldo makes good points about the difficulty of teaching system design except through example and experience.
His main concern is that opportunity to learn by doing is very hard to come by. In academia, systems tend to be small and rapidly discarded, due to the need to publish frequently and produce results quickly. In industry, systems tend to be proprietary, encrusted by patents, and impossible to discuss or talk about publicly. This leaves only limited latitude for public construction or discussion of systems large enough and interesting enough to really learn from.
Waldo suggests that open source projects are one of the few ways out of this dilemma. They are in many cases fairly large in scope, they are fully visible to anyone wishing to critique, extend, or adapt them, and they provide not only a code base but (in the best cases) a community of experienced designers from whom new contributors can learn. They therefore are in some ways the best hope for spreading effective education about system design, being unencumbered by either the short-term problems of academia or the proprietary problems of industry.
Recently, coincidentally enough, some Googlers working on the Google lock service — a key part of Google’s distributed infrastructure — wrote a paper describing their experiences building a production implementation of the Paxos protocol for distributed consistency. What’s especially interesting about this paper is how neatly it both decries and embodies the very dilemma Waldo is talking about.
The Google Paxos paper has a lot of extremely interesting technical content in its own right. It’s one of my favorite types of papers — a discussion of problems encountered when trying to take compelling theory and make it into something that really works in a live system. Without that kind of effort, excellent ideas never actually get their chance to make a difference in the world, because until they’re embodied in a real system, they can’t deliver tangible value. So this paper is very useful to anyone working on implementations of the Paxos protocol — it’s exactly the kind of experience that Waldo wishes more people could learn from.
The writers themselves have the following gripes:
Despite the large body of literature in the field, algorithms dating back more then 15 years, and experience of our team (one of us has designed a similar system before and the others have built other types of complex systems in the past), it was significantly harder to build this system then originally anticipated. We attribute this to several shortcomings in the field:
- There are significant gaps between the description of the Paxos algorithm and the needs of a real-world system. In order to build a real-world system, an expert needs to use numerous ideas scattered in the literature and make several relatively small protocol extensions. The cumulative effort will be substantial and the final system will be based on an unproven protocol.
- The fault-tolerance computing community has not developed the tools to make it easy to implement their algorithms.
- The fault-tolerance computing community has not paid enough attention to testing, a key ingredient for building fault-tolerant systems.
As a result, the core algorithms work remains relatively theoretical and is not as accessible to a larger computing community as it could be. We believe that in order to make a greater impact, researchers in the field should focus on addressing these shortcomings.
The ironies here are so deep it’s hard to know where to start. Their implementation itself is not only proprietary to Google (and not open sourced), but it also relies on many other proprietary Google systems, including the Google file system. Hence their work itself is not directly available to the wider community for development and further discussion! Their paper has a number of interesting allusions (such as exactly why they needed to make their local log writing multi-threaded) that are not followed up. Unless they write many more papers, we will never know all the details of how their system works.
They criticize the fault-tolerant systems community for not having provided a more solid experience base from which to build. Waldo’s paper makes it crystal clear exactly why this base has been lacking: where is it to come from? Not from academia; research projects in academia tend to be too short-term and too small in scope to encounter the kinds of issues the Googlers did. And not from industry; Google is not known for open sourcing its core distributed software components, yet Google is arguably ahead of anyone else in this area!
The only alternative would be a true open source project. But large-scale distributed systems are probably among the least likely to achieve real momentum as an open source project, because actually using and testing them requires substantial dedicated hardware resources (many of the failure cases the Google team encountered arise only after running on dozens or hundreds of machines), and those resources are not available to any open source projects I’m aware of.
The Googlers are part of the problem, even while their paper seeks to be part of the solution. To some extent it’s a chicken-and-egg dynamic; without access to a truly large pool of machines, and a truly demanding set of developers and applications, it’s hard to get real-world experience with creating robust distributed infrastructure — but you almost have to be inside a large business, such as Google, in order to have such access at all.
So, unfortunately, it would appear that in the near term the Googlers are doomed to disappointment in their expectations of the research community. Google itself is likely to remain the preeminent distributed systems research center in the world, and the fewer of its systems it open sources, the less assistance the rest of the world will be able to provide it.
One can only hope that several years from now, Google’s applications will have evolved so greatly on top of its base infrastructure that it will no longer consider the fundamental systems it uses — MapReduce, BigTable, GFS, Chubby — to be key competitive differentiators, and will choose to open source them all. Of course, by then Google’s real difficulties will still be with problems the rest of the world wishes they had the resources to encounter….
A coda to this: John Carmack, of id software and Armadillo Aerospace fame, is known for open sourcing his game engines after five years or so. Recently he’s been doing work in mobile games, cellphone programming. Here’s a quote from a liveblog about his keynote at Quakecon last week:
Met with mobile developers at the Apple thing, all talking about how they make mistakes all the time. Carmack: “Can’t the guys who made the mistakes the first time just make the chips right this time?” Other devs: “Yeah, but most of those guys are too rich to care anymore.”
So that’s the other reason the field doesn’t make good progress… proprietary stuff gets built, developers get rich, technology gets sold and eventually back-burnered, and then it all has to get reinvented all over again. Open source: the only way to not reinvent the wheel every five years!
Matthew is born!!!!!
[Crossposted to both my blogs]
I’m overjoyed to announce that at 2:05 AM on Monday, July 30, our son Matthew Thomas was born.
He’s a 9 pound 5 ounce bundle of joy. He’s sleeping well, nursing a lot (gaining his ounce a day — hardly lost any weight after birth), and his big sister loves him, as you can see 🙂
Michelle, my wife, is resting and recovering, along with taking wonderful care of him. I’m taking time off work to care for them both, as well as for our daughter Sophie.
We are incredibly blessed. And what’s more, our family is now complete. After having two oversized kids, with difficult births both times, we’re done… we’re going to count our blessings, quit while we’re ahead, and move from child-birthing to child-rearing for good.
PHEW!!!!!
Thanks to everyone who’s sent congratulations and other good wishes.
What’s more, we’re even getting down to enough of a routine that I can think about blogging and hacking again 🙂 It’d go a bit more quickly if I hadn’t just gotten a sore throat… but anyway, stay tuned.
What’s Gilad Up To?
The legendary “computational theologist” Gilad Bracha is on the verge of another divine revelation.
Gilad, as many know, was for years one of the gurus of the Java language design, delicately dancing on the infinitely thin edge between semantic integrity and linguistic flexibility. Until, that is, he got tired of the need to continually choose between poor alternatives and quit Sun last year. Since then his new blog, the cheerily named Room 101 (one of my must-reads), has been putting forth all kinds of evidence about his new venture.
It’s clear that it’s a startup, and that they’re working on a new language. But what kind of language? And for what purpose?
Start with Gilad’s previously-mentioned-here paper on Objects As Software Services. This paper discusses the possibilities of downloadable software objects — with behavior and state — that can provide offline functionality, transparent synchronization to the server when reconnected, and dynamic upgrade. He goes into some detail on the language properties these objects would benefit from — specifically, reflective tracking of field accesses (via a mirror-based reflective mechanism), and a typing semantics that is flexible enough to allow code upgrade without entering classloader hell.
In more recent posts on Room 101, we’ve seen other pronouncements of what good languages look like. For one, his excellent recent post on why constructors are considered harmful, which is because they are far too tightly coupled to the specific class in which they’re declared. Most of the time you want object creation to be a matter of policy, but constructors are pure mechanism — all you get is the instance of the class you’re specifying, which is totally inadequate in most cases (the majority of cases where you want an object that fulfills some interface, but you want the framework to pick the appropriate class, since it knows more about the classes involved than you do). The canonical solution here is factory methods, but Gilad considers most of the factory method idioms in, e.g., Java to have many of the same problems — you can’t override a static factory class, you can’t extend or replace it. He suggests instead a mechanism for encapsulating object-creation methods within a parameterized class definition, allowing full virtualization of object creation without the overly tight coupling to the class itself. So Gilad’s seeking a language that has the structural purity of Smalltalk, with some of the warts removed.
There’s also his post on why tuples are good, which is relatively straightforward as Gilad’s posts go. Again, tuples give you a single language mechanism that covers a variety of important uses. We’re seeing a real focus here on The Right Thing. It must be a blast for Gilad to be working with a clean sheet of paper after a decade in the immensely constrained Java space.
Finally, there’s his post on why message passing is good. This is not going to be any surprise to anyone who’s enamored of message passing languages, such as Erlang and (at least at the distributed level) E. Pure message passing gives you a degree of isolation and security that more tightly coupled languages can’t match. It also has a very real performance cost — we don’t yet know how to build an optimizer that can make a message-passing loop be as efficient as an inlined C++ function-call loop — but Gilad’s main concerns, as we’ve seen, are linguistic flexibility and expressiveness for use cases including distributed programming and dynamically upgradeable software objects, not achieving the maximum possible performance for typical modern architectures (which has always been one of Bjarne’s key goals).
Also, Gilad recruited Peter Ahé away from Sun, and Peter’s spilling some beans of his own.
So putting all this together, what do we have? Gilad is working on a new language that:
- is a pure message-passing language (a la Self)
- refines Smalltalk’s metaclass structure to allow parameterized factories and basic constructors to be implemented as virtualizable instance methods
- supports a flexible enough typing structure to enable dynamic upgrade of the code of stateful objects
- has a mirror-based reflective mechanism suitable to allow, among other things, logging of all object field accesses for subsequent orthogonal synchronization
- has tuples 🙂
It’s going to be very interesting to play with this language of his once it’s done. And it even looks like he’s hiring, so things may be pretty close to public visibility! If nothing else, his new venture is going to provide a lot of food for thought. New languages always have a hard time achieving critical mass, but with Gilad’s sense of history and design skills behind it, this new one has a much better than average chance.
Offlining Soon (Temporarily)
My wife is within days of delivering our second child. We’ve already got a two-and-a-half year old girl, Sophie, who’s a little bundle of shining happiness — seriously, this kid is one of the cheeriest tots you’re likely to meet, and my wife and I are totally smitten with her. Well, now her baby brother Matthew is on the way, and it’s happening Real Soon Now.
So at some point in the not-very-distant-at-all future, I’m going to be going offline in a big way, possibly for up to a month. Do not fret. I shall return in force, once we’ve gotten the hang of juggling TWO little ones in the house, and once we’ve made darn sure that Sophie feels OK with everything.
Classloader Hell
I really like Java. It’s definitely my favorite language, and the one I’ve written the most code in, by far. Its balance of static typing, great libraries, OS independence, “no surprises” philosophy, and unsurpassed tool support mean that I’m more productive in Java than in any other language I’ve tried.
But that’s not to say it’s perfect. There can never be an objectively perfect language, since there are too many kinds of software that need to be written, and all languages have expressive tradeoffs that affect their usefulness for those purposes. But even on its own merits, Java’s got some major issues.
The biggest issue is the infamously named “classloader hell”. All reasonably experienced Java programmers know it bitterly well: the difficulties of running code within application servers or other kinds of complex deployment situations, where there are multiple sub-projects all loading code at runtime and attempting to operate with each other. (Briefly, for those who don’t know, Java code is loaded by objects named “classloaders”; classloaders define a name space of loaded classes, and a single class, if loaded by multiple classloaders, is considered to be two distinct — and non-interoperable — classes.)
This is one area where Java’s original standard was written in haste. The model of classloaders delegating to each other, the way that classes are considered to be different if loaded from different classloaders, and the many problems this can cause — all are fundamental problems that manifest in a variety of confusing and confounding ways.
The problem gets much worse when you get into code interoperating between multiple systems on a network, each of which may have subtly different versions of the system’s code. The entire Remote Method Invocation standard was predicated on the assumption that Java’s ability to load code over the network would enable code to travel alongside the network messages that reference that code. It feels to me like a case of “wow, Java lets code run anywhere! Just think of what that lets you do for a networking protocol!” The problem was that the versioning issues weren’t well understood, and that has caused bitter pain in actual usage.
It’s not like Sun isn’t aware of the issues. This technical report covers the basics: it’s possible to set up RMI scenarios where perfectly innocent messaging patterns result in systems being unable to send messages to each other at all. Worse, the runtime behavior is difficult to analyze or understand.
These problems aren’t merely abstract. Another Sun paper discusses an entire advanced networking system that was well into development, based on Jini and RMI, when these problems reared their ugly heads and fundamentally destabilized the entire project. The ultimate reason for abandoning the project was its business infeasibility, but it’s clear that Sun’s own technologies — as currently implemented — were deeply broken.
Now, security, versioning, and interoperability are notoriously hard problems, and it’s far from clear that anyone has really great solutions to them. Certainly Sun can’t be too deeply faulted for not getting it right — Java did get many other things right (including OS independence, dynamic compilation, dynamic inlining, garbage collection, and much more). So I don’t mean to pick too deeply on Java; I just mean to highlight some of the biggest outstanding issues with the language.
There’s work going on to fix at least some of this: for example, the Java module JSR, #277, is intended to bring at least some rigor to the entire classloading system. The .NET framework has a better grasp of module loading issues, though I’m not aware of how .NET messaging compares to RMI or whether it’s exposed to any of the same issues. The Fortress language is taking a much more rigorous approach to component versioning and component assembly as well. But most of these seem dedicated to fixing the module loading problems in a single environment; I don’t know of much work on addressing the security and versioning issues associated with network messaging (though JSR 277 does address some of the naming issues that break RMI and Jini).
Still, that’s the great thing about software: there’s always more that needs fixing, and more great things to build! Expect many more posts on this general topic….
Every Two Weeks
[Crossposted to my other blog.]
Blogging’s still new to me. The only thing I really know about it so far is that while there’s an infinity of posts to make, there’s only finite time to make them. And that time is in high demand, mainly from family, but also from all the other great projects that aren’t blog-writing.
So I have to set some kind of deadline for myself. And that is: two weeks. I must, and will, post in both my blogs at least every two weeks.
I’m setting that deadline because I notice my reaction when going to other blogs: if I see the latest post is a month or more old, it’s a sign that the person’s really way too busy to blog, and that the blog’s an afterthought to them. It’s also a sign that who knows when the next post might be?
But if the last post was a week or so ago, that’s still timely. Two weeks is pushing it, but based on my experience so far, it’s the most I can commit myself to.
So, that’s my promise to you: I’ll update both my blogs every two weeks at most. Feel free to flame me if I don’t!
…Except for the next month, because we’re expecting my wife to go into labor sometime in the next three weeks, which means all bets are off 🙂
GWT, JSF, declarative markup, and lunch with Gavin
Just had an interesting lunch with Gavin King (original author of Hibernate and Seam), and came up with a somewhat crazy idea that needs more eyeballs on it.
EXECUTIVE SUMMARY
The GWT toolset lacks a way to do declarative markup. The only way to compose GWT components is through Java code. Joel Webber of the GWT team said at Google Developer Day 2007 that one of his personal ambitions for GWT is to build a declarative markup system.
The Seam toolchain lacks a way to build custom JSF widgets easily. Building a custom JSF widget with plain Javascript is extremely painful. The GWTJSF integration library goes partway towards addressing this, but doesn’t currently support building composable trees of GWT-based widgets.
What if we brought the two together? What if we could create a JSF-based and Facelets-based markup system that enabled composability of independently developed JSF widgets implemented in GWT? This could both leverage an existing declarative markup system (Facelets) for GWT’s purposes, and deliver a much better toolset for creating Seam applications with richer UI functionality.
That’s the basic idea. If you Get It, then please skip to the DETAILS section. Otherwise, read on.
BACKGROUND
The first chunk of lunch was walking through my Seam/GWT example, clarifying what it does and how it does it. Basically, as the page explains, it takes GWT modules and encapsulates them in JSF components. This allows GWT modules to exist within the context of a larger JSF / Seam application, leveraging other JSF page components and directly communicating with Seam components through GWT RPC.
The main use for this, it seemed to us, is taking existing GWT modules and wrapping them as JSF components, to enable their use in a larger JSF application. This was one of the original use cases that motivated the Exadel guys (Sergey Smirnov and Alex Smirnov) to create their GWTJSF integration library.
Where do we take it from here, though? Originally it seemed to us that if all you need is GWT, then this JSF integration isn’t of much use to you. Michael Neale has already landed — in the Seam 2.0 beta — a GWT integration that doesn’t require JSF at all, but just lets your GWT app communicate with Seam components. (He hasn’t, evidently, integrated my RPC refactoring in GWT 1.4, so it doesn’t yet support native GWT-style RPC, but that’ll be easy to do now that GWT 1.4rc1 is out.)
So, is the only use case for GWTJSF really for people who want to take existing GWT components and build them into JSF apps? Well, Gavin saw the code for the GWT module in my demo, and his feeling was, wow, this is a much better programming model than writing custom JSF components in Javascript. So what would it take to make GWTJSF better at creating reusable JSF components?
There are two main things that the current GWTJSF library lacks. One of them is support for Seam conversation propagation. This is important if you want to have GWT components that can participate in a Seam web application which uses both GWT-based AND non-GWT-based components. You want the GWT components to be able to carry the conversation state in RPCs, so they can participate in the Seam server-side conversation state management. (Yes, I know it is totally against the GWT religion to even mention the words “server-side session state.” This isn’t the main point, so don’t get hung up on that.)
But that’s actually not the most interesting possible extension.
THE DETAILS
The current Seam blogging example is basically a tree view of blog posts. Each tree node can be expanded to show the full text of a blog post, with an inline edit button that lets you edit the post’s text right there in the tree.
Here’s the GWT/JSF markup for instantiating this tree-view-blog-editing component:
<!– This is a JSF component encapsulating a GWT module. –>
<bloglist:component id=”main2″>
<!– This widget routes GWT RPC requests to the Seam component named “gwtBlog”. –>
<gwt:gwtListener serviceBean=”#{gwtBlog}”/>
</bloglist:component>
This creates a JSF component wrapping a GWT module. The GWT module exposes an interface which is implemented by the server-side Seam component named “gwtBlog”. So the GWT module uses a single interface to communicate with the server.
The GWT service interface is this:
public interface BlogService extends RemoteService {
public Blog getDtoBlog ();
public List getDtoBlogEntryList ();
public String getFormattedHTML (String wikiText);
public void updateBody (String id, String newBody);
}
This interface conflates operations on the whole list of posts (getDtoBlogEntryList) with operations on individual posts (getFormattedHTML, updateBody).
Overall, this widget is great and all, but it really consists of two orthogonal pieces: a tree view component, and an individual-post-editing component. They’re bound together in the current GWT code. This widget is monolithic; all it does is make a tree of blog posts. It can’t be repurposed by tweaking the JSF page.
When showing this to Gavin, his feeling was that it was not nearly generic enough. The typical Seam way of doing this would be something like (edited for brevity):
<!– Generic tree view, using a post-edit widget for each node. –>
<ice:tree id=”tree” value=”#{tree.model}” var=”item”>
<ice:treenode>
<ice:panelgroup style=”display: inline;”>
<ice:commandlink value=”#{ item.userObject.text}”>
</ice:commandlink>
</ice:panelgroup>
</ice:treenode>
</ice:tree>
This builds a JSF component tree that declaratively nests multiple widgets, using Seam expression language ( e.g. “#{tree.model}”, “#{item}”) to specify where the components get their data.
Now, here’s the rub. What if you could write JSF / Facelets markup that instantiated a tree of GWT widgets this way?
Something like this:
<gwt:tree id=”tree” value=”#{blogEntries}” var=”item”>
<gwt:blogpostedit value=”#{item.blogPost}”>
</gwt:blogpostedit>
What you’d be doing here is writing a JSF page that creates a component tree that is purely instantiated on the client. The only rendering would happen on the client via GWT. The GWT components would (WAVING HANDS MADLY) have some protocol for nesting themselves, and would have some data-binding style coupling to the server components from which they pull data. The RPC could be GWT-style RPC via service interfaces generic to each specific component.
This would essentially turn the JSF UI component tree into a structure that can be either rendered purely on the server (original-style JSF controls), or rendered partially on the server and refreshed using AJAX (the Ajax4JSF / RichFaces style of user interface), OR rendered entirely on the client (GWT-style widgets). This would also enable declarative construction of GWT-based interfaces, using the exact same syntax and UI component tree as used for other JSF interfaces.
This is good for Seam, because it enables a graceful path towards Seam-based JSF apps with lots of custom UI code, without losing the compositional good qualities of Facelets. And it’s good for GWT, because it gives a declarative framework for building GWT applications, with a great story for coupling to server-side business logic.
ARE YOU CRAZY?
This is not quite a half-baked idea yet. In fact I’d say it’s about 10% baked. But it’s definitely interesting, and it definitely bears more thinking about.
There are many issues with it, just a few of which are:
- This doesn’t really let you mix arbitrary non-GWT and GWT JSF widgets; the only allowable children of GWT JSF widgets would be other GWT JSF widgets. (Unless you get really sneaky with partial rendering invoked FROM GWT, which probably isn’t worth it….)
- I have no idea at all what Joel Webber (of GWT fame) has in mind for declarative markup, so this concept may be totally wrong for his use cases.
- On the face of it this idea seems to imply some kind of EL (expression language) parser implemented in GWT, which is no small chunk of work.
- Who exactly implements this is a totally open question; I still want to get back to the offline Blogger client done with GWT/Gears, which has basically nothing to do with this 😉
I don’t claim to have answers to these; this post is just a starting point.
I’m going to post in the GWT developers forum, in the Seam users forum, and in the Ajax4JSF developers forum to see what kind of reactions this idea provokes.
Discuss!
Landed!
My first official self-commit to an open source project.
I’ve just checked in my GWTJSF patches to the Ajax4JSF trunk. You can now get the GWT 1.4-compliant GWTJSF project by following these directions.
No word yet from the Exadel guys; I’m 99% sure I didn’t break anything but they’ll tell me if I did, they’re not shy 🙂 I do hope they will update their labs.jboss.org documentation to mention this project again, now that it actually builds in their tree. We’ll see.
I included a reference to the googlecode repository in the gwtjsf pom.xml, so it should have no problems pulling GWT 1.4.10 (which it needs).
The old GWTJSF examples (which are not on labs.jboss.org anyway yet) no longer work, mainly because the Maven build (as my last post discussed) splits the sources into a separate jar, so the old build files break. I’ll likely land at least one of my modified samples in the Ajax4JSF tree in the next couple of days, so people will have a reference point.
After that I’ll get back to finalizing my Seam/GWT example and submitting that to the Seam project. Hopefully in another week, or two at the most, I’ll have this whole thing wrapped up and then I can get back to the Gears/GWT offline Blogger editor project!
Onwards!
OK, now I see why people dislike Maven
Nothing like using something for yourself to clue you in to the buzz around it.
The buzz around Maven 2 is pretty negative. And now I’m seeing why. As my last post mentioned, I’m working on fixing the Maven build in the GWTJSF source tree. (Right now that source doesn’t build at all.)
I got the basic Maven compile working. Spent tonight on trying to get the tests working. Finally gave up when it turned out there were two critical problems:
- The GWTShell doesn’t like the Maven classloading pattern. Charlie Collins has a patch for this but it didn’t make it into GWT 1.4.
- Running GWTShell requires access to the OS-specific GWT native libraries. The current means for doing that is with system dependencies in Maven, e.g. download the appropriate GWT version and point your Maven POM at it. This is totally contrary to the entire point of Maven, which is supposed to download everything for you. Will Pugh is working on GWT-Maven support for downloading the whole native GWT archive and expanding it locally, but it’s not done yet.
So, fine, I gave up on getting the tests to run. Now I just want to verify that I actually have a working .jar file.
Turns out that I don’t, because the GWT compiler needs not just the .class files but also the .java files to be in the gwtjsf.jar file. The original Ant build for the GWTJSF project had no problem doing this, it was totally obvious and trivial how to do it. So, it should be trivial in Maven 2, right?
HAH.
I found the plugin documentation page for Maven 2. This led to the documentation page for the jar plugin. Hmm, pretty sparse. How about the usage examples? Nope, nothing there about what the actual configuration options are, just some examples of signing (which isn’t at all what I want to do). But look! At the top, it says this:
If you want to use advanced configurations you should have a look at the Javadocs for MavenArchiveConfiguration.
Oh joy! But wait. If you click on that link, what do you get? You get a page titled “Maven – Page Not Found”, which amusingly enough includes a tag saying “Built by Maven”.
That’s what I call bad — when your own javadocs for your automated build tool have automatically generated broken links!
It turns out that I missed a plugin that was actually already in the POM for the project, namely the maven-source-plugin. This generates a separate JAR file containing the sources for the project. That’s not what I want — I want the JAR file to contain both the sources and the classes — but it seems to be the best I can get Maven to give me.
Overall, it really seems that Maven has an incredibly fragmented configuration model, plugins with very, very specific functionality that is hard to override, and documentation that is scattered and broken. Sigh. Oh well, once I land this I’ll do my best to be done with it. Here’s hoping the Maven team can fix some of this.
Edit: I will say that I like what Maven is trying to do. This POM Reference makes a lot of sense… having a higher-level description of your project is not a bad concept. But the execution and the documentation are fraught with peril.
Perforce is FREE
Many people love Perforce. That includes me.
What a lot of people don’t realize, though, is that if you are an individual programmer working on your own personal projects, Perforce is free.
See, you can download the latest release of Perforce and use it in an evaluation mode, where it’s limited to two users and two clients.
If you’re only using it to maintain your own local changes to one or more open source projects (say, GWT and Ajax4JSF and Seam), then that’s all you need. There’s no limit on the number of files or branches, and all Perforce features are 100% functional.
Right now on my Windows PC, I’ve got a Perforce depot which has:
– a complete download of the GWT Subversion tree
– a complete branch of that GWT tree, with my changes in it
– a complete download of the Ajax4JSF Subversion tree
– a complete branch of THAT Ajax4JSF tree, with my changes in it
– a copy of the Seam booking example
– a branch of the Seam booking example, with my changes in it
Setting my Windows Perforce client to use Unix line endings means that Perforce doesn’t screw up the Subversion metadata.
The workflow is:
– check out from Subversion
– check in to Perforce
– branch in Perforce
– edit like mad in the branch, committing at will
– once done, integrate back to the Subversion copy
– check out the whole Subversion copy for edit
– svn update, svn patch, svn commit
– revert unchanged files in Perforce
– re-commit to Perforce
The truly wonderful part is that I can take changes from the Subversion copy to my dev branch, or vice versa, as circumstances warrant. Basically, it’s the full power of Perforce branching, used to manage my personal development alongside the concurrent development of everyone who’s landing in Subversion.
I can’t recommend it too highly. Totally brain-saving compared to keeping multiple local Subversion working copies or whatever. Try it, you might love it.