Java is a mature proven language for non-GUI applications.  What it means is that it does what you expect it to and there is a large body of open source software you can leverage.  Carlos E. Perez's enumeration of Open Source Web Crawlers Written in Java is a good example.

C# and .NET, on the other hand, has a long way to go still and there is no easy to extend IDE like Eclipse for developers to rally around.  Working with .NET at this point is like working in a new town destined to grow, maybe like Chicago was around 1840.  As for my involvement with .NET, I enjoy the rough life.