http://igraphicsgroup.com/blogwp

 Thanks!

Right now, we have several systems running virtual machines.  We have an older ESXi server that runs a few older development systems.  We also have a subversion server running on a Linux VM, and a mail server running under Hyper-V.

Essentially, we have four systems running different hypervisors: ESXi 3.5, VMware Workstation, Hyper-V and VMware Server 1.0.

This will be changing next week.  A new project is forcing us to expand our testing capabilities, and the Dell Outlet had a very nice server at a good price!

We picked up a dual processor T710 with 48GB of Ram to run VMware vSphere 4.1; the Essentials version is only $611.  We’ll put a few TB of storage on it, and be able to migrate all the existing VM’s, as well as all the new ones to it. 

Our existing ESXi server will be transitioning to the backup server role.  I’ve been wanting to try out Openfiler, and this will give me a chance.  After another purchase at Newegg, we have an external enclosure, and 12 TB of RAID storage for backup data.  We’ll use external drives to archive the data.

This project will allow us to repurpose (or retire) three existing servers and add 10 new “testing” VMs, which will be available all the time!  It also allows us to consolidate our backups to a single backup server.

We’re still unsure what third party backup tool to use with vSphere.  There are many options, with differing price points.  We’re going to start with Veeam FastSCP, since it’s free (and what we are using now). 

Over the next several months we will use the 30 day free trials to test the following tools:

1. Veeam Backup & Replication
2. Quest vRanger
3. PHD Virtual Backup

Right now I’m leaning towards Veeam due to it’s ability to restore single files from a VM.  To be honest, if it works well enough, I won’t even test the other tools.

The logical cleanup started as well.  I’ve set up a new Domain Server using openSUSE 11.3 with LDAP.  This was my first experience with openSUSE, and now that it is done I can recommend it.

Major thanks to Masim Suguanto and his tutorial which was a great help with the initial setup: http://vavai.net/2010/03/30/how-to-samba-pdcopenldap-on-opensusesles-part-1/.

Also, it was good to read up on the Samba group mapping http://www.samba.org/samba/docs/man/Samba-HOWTO-Collection/groupmapping.html before starting.

But, getting Windows Server 2008 and 2008 R2 on the Domain was a bit of work!

How to get Server 2008 R2 to operate with a Samba 3.5 Domain Server:

1) Update the registry on the Server 2008 System: http://wiki.samba.org/index.php/Windows7

2) Ensure that the Windows and Linux have the same date and time (run “net time \\Server /set” on the Windows Server).

3) Modify the Server 2008 local policy’s for Network Security: LAN Manager authentication level to “Send LN & NTLM – use NTLM2 session security if negotiated”.

4) Modify the Server 2008 local for Network Security: Network security: Allow Local System to use computer identity for NTLM (Disabled): http://wiki.ssystems.de/doku.php?id=samba_trust_w2008r2_harald_strack.

5) I also set the AllowNT4Crypto registry setting, though I’m not sure if that was needed: http://technet.microsoft.com/en-us/library/cc974327%28WS.10%29.aspx

Reading this, it sure seems easy now!  Though, it took me the better part of two days to get the Samba Domain Controller, LDAP, and Server 2008 all working together.  I like the new database style configuration used by openSUSE 11.3, as well as the GUI configuration tools. 

Migrating the local profiles to domain profiles was not as bad as I expected.  I ran the “User Profile Wizard 3.0” and it seemed to work nicely.

The Next Step:

With the new domain server in place, my next step will be to migrate all my utility and test VM’s from an ESXi (3.5) Server, VMware Server for Linux, and VMware Workstation to a new Dell Server.  I’ve gotten very tired of having all the VMs scattered among several different systems, and want to consolidate them in one place.

Once it is running, the old ESXi server is going to become the new backup server (openfiler seems to be a good choice).  Well, once I plug in a new eSATA card to allow me to hook up all the external drives I’ve been using, that is.

Previously, I had been building Qt with the vc9 (2008) mkspec, and -no-webkit -no-script -no-scripttools.

To pad this post a bit, here’s a link to another blog post where somebody is using Qt, F#, and .dot/mono: qt-from-fsharp.  I have to admit that I *like* building GUIs using XAML, but for cross-platform applications Qt is the way to go.

I’m excited to use the new C++ features.  Auto, lambda, and static_assert are my current favorites.

The What's New in Visual C++ 2010 has a list of all the new language goodies.

Qt doesn’t have a mkspec yet.  I believe that 4.7 will officially add VC 10 to the supported list, and remove VC 7.1 (sadly, since I think it was the best compiler Microsoft has released).

Last night our deployment server died.  It had been running (fairly) problem free for over 6 years.  We used it to host our internal FTP, Bug Tracking and Wiki software.

The silver lining is that we can migrate to a hosted service.  This would save us from needing to buy a new Server.  More importantly, we wouldn’t have to take care of it, manage software updates or back it up.

This would also let us move a few projects to a hosted svn server.  Our external developers would appreciate the speed boost.

I’m leaning towards using xp-dev.

First, CE isn’t the same as Windows Mobile -- CE may be customized quite a bit for the device it is targeted at.  This makes it a little challenging to build the software; especially when the client isn’t sure what the hardware folks are doing.

Second, the “recommended” tools and SDK are extremely old (based on VC 6).  Luckily, you can use the newer tools (VC 2008) as long as you’re careful about your build settings and the System API’s you are using.

Lastly, it’s amazing how dependant I’ve become on the FPU.  This project was targeted at an ARM CPU.  These don’t typically have FPUs (though it is an option).  Most of the rendering code I write uses floating point.

It’s not that ARM processors are slow, they are rather quick and I appreciate their shifting ability – it’s that floating point is emulated with a software library.  This makes it imposable to use in time sensitive areas.

It reminds me of writing for the 80386.

So, after benchmarking the rendering code and discovering just how slow the math was, it was time to implement a fixed point library.  Years ago, when Intel was still developing their XScale, they had an optimized fixed point library for the ARM.  This seems to be long gone from the Internet – I looked for it for several hours and sent several emails to Marvell without any luck.

As it turns out, my ‘C’ fixed scale code was reasonably fast, and the VC2008 compiler did a decent job of optimizing it.  After tuning the routines to prevent over and underflows the module performed quite well.

The only magic is to use 64 bit integers to multiply and divide – if you don’t have those, it’s a lot tougher.

The result was a fast software renderer, with optional support for DirectDraw and GAPI blit’ing.  It was a lot of fun to get into “old school” software mode.

I have been thinking about my goals for the year. 

1. I will complete porting my main codebase to Visual Studio 2010.
2. I will complete my F# tour.
3. I will completely clean my office.

It isn’t that my office is a huge mess, rather, it has a lot of older items stored in it.  For example, I can count 6 blank VHS tapes sitting on a shelf.  Getting ride of all the old items, and reorganizing the equipment that I actually use would allow me to gain a huge amount of space.

With that complete, I would be able to buy a few metal racks and finally put all the systems in one place.  Last year, I “found” two operating routers when I was moving a printer.  I’m not sure what bothered me more; that I had forgotten about them or that they were buried so deeply under cables that they were not visible.

My goal is to have everything on racks, visible, with cables zip tied neatly.

I was enjoying my progress with F#, and can’t wait to get back into it.  The immutability coupled with threading excites me a great deal.

The Visual Studio 2010 release seems to have corrected the flaws introduced with the 2005 and 2008 releases.  My C++ benchmarking has shown it to be almost as fast as Visual Studio 2003, and a bit faster when PGO was used.  Once the mobile SDKs are available, I may be able to migrate all my active projects to it.

My two goals are to:

(1) Try the new features of Silverlight 4; printing and privileged applications.  These were the key issues that prevented me from hosting applications in Silverlight.

(2) Begin the process of porting a large project from Visual Studio 2003.  VC 2005 and 2008 tried to “fix” the MS runtime libraries via Windows SxS.  In my mind, the result was far worse!  VC 2010 goes back to the older model, where the application will load the DLL’s in the application directory (no fuss, no muss).

Because I’m in a C++ mood, I’m starting with the port.  The first step will be to build Boost – the trunk has support (for 2010), so it’ll just launch bjam and let it churn.  Qt doesn’t have a mkspec for 2010 yet, so I’ll just use the 2008 settings.

Assuming nothing goes wrong, I’ll start the basic porting this afternoon.

I’m reading “Pro C# 2008 and the .NET 3.5 Platform”, by Andrew Troelsen (ISBN-13: 978-1-59059-884-9), and can give it a good recommendation.

For my first project, I have decided to implement a Hilbert R-tree and see how well it compares with the overlapping Quad-Trees that we have now.

R-trees use nested dynamic rectangles to store information.  Because they are created based on the data which is stored in them, the structures should provide better utilization, that is no empty (wasted) nodes.

The difficulty with R-trees is that it is difficult to decide how to split data into child rectangles.  The goal is to put an equal number of objects into each of the new child rectangles and minimize overlap.

The Hilbert R-tree solves this problem by using a 2D Hilbert curve to index the data.  A Hilbert curve is a continuous fractal space-filling curve (wiki) which will visit all points of a k-dimensional grid without overlapping.

By partitioning the data using the Hilbert number of it’s center point, we can easily split a node into several child nodes.

The Hilbert R-tree also balances itself via node reinsertion (like the R* tree).

In any case, this seemed like an interesting task to use for exploring C#.

I started by porting my existing overlapping quad tree to C#.  Then I implemented a Hilbert R-tree, without the reinsertion logic, to see how well the splitting logic worked.

Testing Results

Using random data, with a limited size (to better emulate typical mapping data), the quad tree was about 4x faster for inserts.

The R-tree had about %25 fewer nodes.

Search time for the quad tree was slightly better.  I believe that this is due to the large amount of overlapping of the child nodes in the R-tree.

I modified the R-tree splitting to partition the data based on the largest distances between the Hilbert values, which increased the node count, but increased the search speed to roughly the same as the quad tree.

Next Step

The operation of spatial trees is incredibly dependant upon the data.  My tests are using randomly generated data, which is interesting, but ultimately useless.

Next, I’ll extract several hundred thousand objects from a dataset, and use them for performance testing.

I can also recommend “Pro Silverlight 2 in C# 2008”, by Matthew MacDonald.  It covers the major parts of the client side.

The Silverlight control does a reasonable job of displaying raster data.  I enhanced my sample to show a birds’ eye view, and allow pushpins to be added to the map.

Sample

I used the MapCruncher to try and construct a mosaic from a set of raster data which I generated.  Don’t misunderstand, for a Beta Product, it did a reasonable job with a handful of raster images, but when I tried to run a batch job with several thousand – not so good.

Part of the issue is that my test data is in an interesting projection.  I have been using it to test my rendering pipeline, and never actually received the projection data.  In any case, I fudged a projection function that looked reasonable.

After five attempts to run batches of images, I took a step back and just rendered a single huge PNG file for each map layer.

The results are pretty good. 

The primary negative is that the MapCruncher output is huge!  My source data was 70MB.  The MapCruncher data size was 850MB.

I would consider using MapCruncher for small amount of data, but nothing large. 

For bigger raster datasets, I would consider using the ERDAS Image Web Server.  I spotted a blog entry which has some asp.net files that would allow IWS to handle Virtual Earth requests.

For my vector and polygon data, I would create a web service to generate raster data on demand for dynamic map layers.  This would be fun, since I could change the display properties of the different objects types and simply re-render the map tile.

Conclusion

The Silverlight Virtual Earth control shows a great deal of promise for consumer, and light weight applications.  I was easily able to generate custom raster base maps and load vector data via a web service. 

Being forced to use the Mercator projection limits its usefulness for more traditional applications.

First, I needed the Virtual Earth Silverlight Map Control CTP.

Then I downloaded the MapCruncher.  This tool will translate raster data into the Virtual Earth Mercator projection.  I used this with a PNG file I generated to make my sample content.  MapCruncher requires that you to select the common points between your data and the Earth.  Hopefully there is a way to automate this, as it would be quite tedious to go this by hand for 20,000 images!

The Map Control was very easy to use in my sample project; just add a reference, and the XAML code. 

On startup, I set my initial view and add my LocationRectTileSource (which points to the data that I generated with the MapCruncher).

Note: most of this code was cut and paste from the Map Control SDK.

// Create a Tile Layer that will display our custom Map Imagery Tiles

var customTileLayer = new MapTileLayer();

// Define the Bounding Rectangle

LocationRect boundingRect = new LocationRect(

    new Location(34.79576153, 139.35607910),

    new Location(34.71226694, 139.45220947)

);

myMap.View = new MapViewSpecification(new Location(34.75, 139.4), 13);

// Create a LocationRectTileSource

LocationRectTileSource customTileSource = new LocationRectTileSource();

// Set the Uri for the custom Map Imagery Tiles

customTileSource.UriFormat = "http://…/Layer_NewLayer/{0}.png";

// Set the Min and Max Zoom Levels that the imagery is to be visible within

customTileSource.ZoomRange = new Range<double>(0, 18);

// The bounding rectangle area that the tile overaly is valid in.

customTileSource.BoundingRectangle = boundingRect;

// Add the Tile Source to the Tile Layer

customTileLayer.TileSources.Add(customTileSource);

// Set the Tile Layer Opacity to a desired value

customTileLayer.Opacity = 1.0;

myMap.Children.Add(customTileLayer);

When the page opens, it shows my raster data.  I left the Mode Selector enabled, to allow the Virtual Earth information to be shown as well.  This data is *not* correctly geo-referenced, nor is it scaled.

View my sample Silverlight Mapping Tool.

It ran nicely on Windows and MacOS X; I haven’t tried it under Moonlight yet.

The Map SDK also offers the ability to overlay vector data.  My next test will be overlaying polyline and push-pin data onto the raster map.

I have been unable to find a way (on the client side) to convert text to a path, so that I may render it onto my canvas. 

Yes, I know about the TextBlock.  But I can’t seem to figure out how to get it to use the same RenderTransform which works so nicely with my PolyLines.

My very gross solution for my sample is to store the original X, Y, and Height as DependencyProperties in the TextBlock.  I choose this method, because I cannot derive from a sealed TextBlock class (thank you Silverlight .Net).

// Add a Public Property to my TextBlocks

public static readonly DependencyProperty wxProperty = 

    DependencyProperty.Register(

        "wxProperty", 

        typeof(Double), 

        typeof(TextBlock),

        new PropertyMetadata(null));

It seems that the more that I try to do with Silverlight, the more limitations I’m finding.  I guess this is a good thing, as that was the goal of this project.

When I change the transform via a zoom or pan operation, I enumerate the children of the canvas and set the correct coordinates.

//  Transform the TextBlocks

Point pnt = new Point();

Point[] pntY = new Point[2];

Double d;

foreach(Object obj in myCanvas.Children)

{

    if(obj is TextBlock)

    {

        TextBlock tb = obj as TextBlock;

        pnt.X = (Double)tb.GetValue(wxProperty);

        pnt.Y = (Double)tb.GetValue(wyProperty);

        pntY[0].X = 0;

        pntY[0].Y = 0;

        pntY[1].X = 0;

        pntY[1].Y = (Double)tb.GetValue(whProperty);

        pnt = trans.Transform(pnt);

        pntY[0] = trans.Transform(pntY[0]);

        pntY[1] = trans.Transform(pntY[1]);

        d = pntY[1].Y - pntY[0].Y;

        tb.FontSize = d;

        tb.SetValue(Canvas.TopProperty, pnt.Y+d);

        tb.SetValue(Canvas.LeftProperty, pnt.X);

    }

}

I’ll make a class to hold the 3 Doubles with getter’s and setter’s and all that nice stuff later on.  This was mostly a way to see how it would work.

I must admit, it does work.  I suppose that the performance hit wouldn’t be too bad, since this is basically what the framework is doing when I change the transform.

In any case, my sample now renders text on top of the vector data.

The Silverlight Test Application.

Visually, it looks very nice.  Rendering performance seemed a bit slow though.  I’m not going to benchmark it until I get text rendering working.

My cross platform testing was very encouraging!

1. Windows x86 and x64 worked fine.
2. Mac OS X 10.5 worked the same as Windows, including the Install to Desktop.
3. Linux had issues with several of the APIs that I was using for OOB and Network detection.  Once I removed them, the actual application code ran fine.  But, the rendering speed was very poor.

The most recent head scratcher was the decision that Microsoft made to not allow the WebClient to get data from outside of the Domain.  There are samples that show how to set up a proxy (using asp).  But, strangely enough, nothing for developers who wish to use a different web server.

This isn’t a show stopper, just an aggravation.

I started by watching several of the videos on the Silverlight Site.  These covered most of the basic issues that I had questions on:

1. Out of Browser Mode
2. Disconnected Mode
3. Install to Desktop
4. Automatic Update of the Desktop

The current version of the Visual Studio 2008 Silverlight package makes it quite easy to add these functions.

I did run into an issue with the Moonlight Beta, when I added the Out-of-browser metadata to the XML, Moonlight didn’t like it.

Could not parse element Deployment.OutOfBrowserSettings, attribute (null), error: Unknown element: Deployment.OutOfBrowserSettings.
not implemented: (ErrorEventArgs.lineNumber)

Test App Version 0.01

As a starting point, I have a (very) small set of vector data.  It exists in a bounding rectangle of 0,0, 1000,1000.

When the Application starts, I initialize a TransformGroup with a translate, scale, rotate and translate.

trans = new TransformGroup(); 

trans.Children.Add(new TranslateTransform()); 

trans.Children.Add(new ScaleTransform()); 

trans.Children.Add(new RotateTransform()); 

trans.Children.Add(new TranslateTransform()); 

I hook the Canvas OnSize event, and use that to adjust the transformgroup to ensure that my canvas “world” data is mapped into the canvas pixel space.  There is likely a better way to handle this, but I haven’t found it yet.

((TranslateTransform)trans.Children[0]).X = -1 * (wx /* 1000 */ / 2); 

((TranslateTransform)trans.Children[0]).Y = -1 * (wy /* 1000 */ / 2); 

double sx = myCanvas.ActualWidth / wx; 

double sy = myCanvas.ActualHeight / wy; 

double s = Math.Min(sx, sy); 

((ScaleTransform)trans.Children[1]).ScaleX = s; 

((ScaleTransform)trans.Children[1]).ScaleY = s; 

// Skip rotation 

((TranslateTransform)trans.Children[3]).X = (myCanvas.ActualWidth / 2); 

((TranslateTransform)trans.Children[3]).Y = (myCanvas.ActualHeight / 2);

I hope that there is a better way to cast the children!  This is ugly.

When my data is added to the canvas, I assign my transform group to the objects RenderTransform.

Line l = new Line(); 

[Set stroke and coords] 

l.RenderTransform = trans; 

myCanvas.Children.Add(l); 

So far, so good.  When the canvas is resized, my transform is updated, and the data is rendered correctly.

Here is a link to my Test Silverlight Application.  After it loads, press the “Load Data” button and resize the window to see the transform magic.

My next step is to generate some “real” data and see how quickly I can render a few thousand polygons.