From the Labs -- MetaCarta Labs Weblog

Oftentimes, it’s not entirely clear to users of software like OpenLayers, TileCache, and FeatureServer what MetaCarta does.

Recently, John Frank, MetaCarta’s CTO (and my boss) put together a short (1m30s) screencast, demonstrating the power of what MetaCarta really does: The Power of Geographic Search. Schuyler graciously provided narration to the video.

This video demonstrates MetaCarta’s powerful text + geographic search engine, indexing millions of documents and responding in the blink of an eye. Using this search tool, it’s possible to do a full text and geographic search over millions of documents, finding everything written about an area even if you don’t happen to know the names of all the several hundred thousand places nearby.

As Schuyler’s voiceover says: “MetaCarta provides a powerful filter for finding everything written about any place.”

MetaCarta Labs on a Stick includes this video, and also a geographic search demonstration, showing how to use the GeoSearchCore download available from the MetaCarta Developers Downloads to add geographic text search to any OpenLayers Map. Of course, if you find this interesting, the next step is to read more about it on the official GTS Product Information page, and of course, contact us –  but even if you don’t, now you know a bit more about what MetaCarta does.

So, right after FeatureServer was released, I realized I finally had the full stack I had an interest in handing out to users:

• Web Client, OpenLayers
• Tiles, via TileCache
• Vector drawing and saving, via FeatureServer

However, I know that one of the things that makes the use of things like FeatureServer a bit more difficult is that it depends on the installation of a number of Python modules for full usage. The GeoJSON service, for example, requires simplejson to be installed, and if you’re on Windows, even getting Python up and running in a sane way can be a bit of a challenge.

The solution? MetaCarta Labs on a Stick.

A collection of distributions for the latest in MetaCarta Labs software, including OpenLayers, TileCache, FeatureServer and GeoSearchCore, MetaCarta’s geographic text search addon for OpenLayers. A built in HTTP server that will run on Windows, OS X and Linux with nothing more than double clicking an icon or starting up a shell script. A set of tile data that will allow you to browse your local vectors over vmap0 or blue marble satellite data — without even being connected to the net.

And when I say ‘will run on Windows’, I mean it. I took a freshly formatted Windows machine, plugged the USB stick in, double clicked the ‘run_windows.bat’ icon, and I had a web server running — with no connection to the internet, I was able to draw features, save them in FeatureServer, read them as GeoRSS, and edit them as KML.  (Note that this is with the USB drive version, which is different from the ‘preliminary distribution’ on the website.)

I’m going to have 14 of these things with me in California next week. (The 15th is staying on my keyring. ) At least some of them are likely going to be used as prizes for a competition that I’ll announce during my OpenLayers presentation at Where 2.0, on Tuesday at 4:30.

You can try out a less complete version of the Labs on a Stick distribution via the preliminary download, but for a demo of the ‘real thing’, be sure to find me at Where 2.0 or WhereCamp.

I’d like to point out that this is not ‘try before you buy’ software, as was mistakenly reported last night: the software on this distribution is provided free of charge for all eternity. FeatureServer, TileCache and OpenLayers are all open source software released under BSD-like licenses. The GeoSearchCore distribution has a slightly more restrictive terms of use, but not one that limits the use of the software — it just means that to get your search results, you have to come to MetaCarta. (Who else does super-fast geographic text search anyway?)

Everyone can thank James Fee for egging me on to actually get this out the door — I was going to spend the weekend on it, but somebody had to be impatient… but it seems like he’s pretty happy with what he’s got.

MetaCarta Labs is proud to announce the release of FeatureServer, an open source, Python based RESTful feature server.

FeatureServer allows you to store points, lines, and polygons in a number of backends, and get the data out all at once or one piece at a time. You can get the data out as KML, JSON, GeoRSS, GML/WFS or even as HTML.

FeatureServer is primarily designed as a lightweight vector feature storage companion to the new OpenLayers vector capabilities.

A demo of FeatureServer in OpenLayers is available at the FeatureServer Demo page. You can see features from this demo in:

• HTML
• KML
• GeoRSS (Atom, simple)
• JSON

FeatureServer is the first project we’ve released under our copyright-only open source license, a move designed to protect both MetaCarta and project contributors. We’re moving forward with getting this license approved by OSI, and plan to release future open source software under this license.

Chris Holmes from the Open Planning Project has published a TileCache tutorial on Codehaus. The tutorial covers installation, configuration, integration with OpenLayers, performance tuning, and cache seeding, making it a more in-depth complement to Christopher Schmidt’s recent screencap video. We hope you find it helpful!

Getting started with MapServer can be a daunting task. There’s a magical configuration file that’s non-intuitive to edit, and a lot of knowledge of things like projections and so on is required to edit the configuration files for anything but the most trivial data. One of the ways I’ve tried to help this has been distributing mapfiles for maps I create so that people can learn from them, but there’s only so much this helps alleviate the pain of learning the mapfile syntax.

This is the reason why I was extremely excited to see QGIS’s support for converting a QGIS project to a map file. Finally, something that could take the headache out of creating mapfiles by hand. Perhaps this could help create the GUI that users needed to create the maps they wanted, and get them onto the web.

Today, I put together a 4 minute video demonstrating exactly this functionality. It shows how to download GIS data, open it in QGIS, style it according to an attribute, and export that view to a map file. It then shows how to load that map file into OpenLayers, so you can view it in the browser.

You can view the video online at OpenLayers: Mapping Your Data (requires Flash), and the wiki page that goes along with it in the OpenLayers Trac: MappingYourData.

Questions, comments, and suggestions are always welcome.

— Christopher Schmidt, Video Maven

From an update I sent to the OpenLayers mailing list on the vector work we’ve done this week:

• Changed:
  • Feature.js -> Feature.Vector. There are now two features — onedesigned for use with Markers, popups, etc. and one designed foruse with geometries.
  • Parser/Writer classes now in a single ‘Format’, which has read()and write() functions which switch between strings and collectionsof features. This means that ot create a layer which is like theGML layer (accept a URL, put features on the map) but which parses, for example, GeoRSS, requires only writing a Format() classfor it, and then changing a single line of code.

    This also means that it is possible to take any vector layer andserialize it to GML with a single function call. (var g = newOpenLayers.Format.GML(); gml = g.write(layer.features);)

  • Layer.GML and Layer.WFS now use the Format class.
• Major changes:
  • Attributes() parsing moved into the Format() class. This makes iteasier for other formats, but may be a performance hit. (However, Ihave run one hundred features through the GML parser and not hadany speed problems: the speed problems are typically in renderingrather than raw XML parsing). This means that feature.attributes isnow just a dict, rather than an object with an XMLNode attached.This means that all attribute parsing is done in one location,which makes it much simpler across the code.
  • Style.js is going away. Instead, there is anOpenLayers.Feature.Vector.defaultStyle (similar toOpenLayers.Marker.defaultMarker) which encapsulates the defaultstyle: various other styles will be stored where appropriate,usually in the controls.
  • MouseListeners seemed to boil down into a concept of ‘handlers’: there are several types of events on the map which should be encapsulatedin handlers which have various actions. (”draw box”, “draw point”,etc.) So, there are now ‘handlers’ which handle the action ofdrawing a point. Controls can then turn these handlers on and off:this will allow trivial building of controls like the toolbar,because each handler has an idea of whether it is activated ordeactivated, a toolbar only needs to encapsulate mechanisms forturning these functions on and off.

    http://dev.openlayers.org/sandbox/vector-2.4/examples/draw-point.html

    Checking the box turns the point drawing control on. Uncheckint it turns it off.

    This also changes the ‘box’ mechanism such that you can easily tkaea box and do whatevery ou want with it. Handlers return the featurethat was drawn with them, essentially: so when you draw a pointwith the DrawPoint handler, it returns the point. When you draw abox with the zoombox handler, it reutrns the box.

    If you look at the source of:

    http://dev.openlayers.org/sandbox/vector-2.4/examples/custom-control.html

    You can see the draw() of the control creates a ‘Box’ Handler(Handler.Box), and assigns the ‘done’ callback to call notice().This means that when the Box handler is done with drawing the box,it will call the notice function — with the bounds of the box. Thecontrol can then do what it wants with this. In some cases, (mousedefaults) this might be to zoom to the bounds described by thatbox. In this case, it just ‘alert()’s the pixel coordinates of thebox.

    This solves a whole host of ‘how do I change the behavior of’questions: it encapsulates the Handling of mouse events at a lowerlevel than registering on the map, and the result is that somewhathacky code (like that in MouseDefaults) moves down a level, makingwhat MouseDefaults will soon be a very simple control which justsets up a set of handlers.

• Testing:
  • Currently, the geometry classes have more than 150 tests. Erik hasdone an excellent review of these classes, and has pronounced themcomplete… I think
  • Renderer, Format and Layer tests are coming up the pipeline.
• Documentation
  • The planned route for API documentation is JSDoc, and the JSDoccomments have been kept together for most of the code (though a fewclasses that I wrote need better docs )
  • There are examples demonstrating how to add simple features tolayers, and that API is unlikely to change. Very little API changehas happened at all betweeen the vector and vector-2.4 branch, soif you were using one, you should be fine with the final versionthat goes to trunk.
• Future plans:

  The desire is to get this stuff as done as possible this week suchthat we can walk away from the Cambridge OpenLayers Hackathon witha working branch, and hten soon after, merge it to trunk. We arehoping to get everyone on board for a 2.4 release in the next 2weeks with vector stuff — even if some features and functionalityare not complete — so long as there is a consensus that theexisting API is solid and unlikely to change. This will leteveryone work on continuing development of the vector code on trunkrather than on a branch.

I just want to offer a huge thank you to all the people who have madethis possible. The work on the vector branch is the only thing that hasmade this development possible, and a lot of people took part inmaking that happen. Camptocamp’s participation especially took this codefrom being a great idea to being a great tool. The vector branch is acore example of the widespread diversity of the Open Source communityparticipation that has led OpenLayers to become such a great project,and I am extremely grateful — both as a project contributor andpersonally — for all the work that has been done and all the greatfeedback we’ve gotten.

Thank you, thank you, and thank you again. It is because of developmentlike this that OpenLayers is going to continue to move onward and upwardinto becoming the best mapping API out there.

This week is going to be an OpenLayers HackFest here in the MetaCarta office in Cambridge. In addition to the normal developers here in the office — Schuyler and myself — we’ve got our errant hacker Erik Ezureau up from south of the border, and Tim Schaub, in town from the far west. (Of course, when you live on the East Coast, anything west of New York is supposed to be Far West).

The big goal as part of this development time is to integrate the work that has been done by a number of contributors to the Vector branch of OpenLayers. This development has been geared around creating in-browser vector-based display and editing, including display of WFS lines, polygons, points, etc. This is one of the main features which is holding OpenLayers behind other mapping APIs: adding this functionality using client-side rendering will definitely increase the number of applications which can be built on top of OpenLayers.

I’m looking forward to meeting Tim, who has been one of our most prolific contributors over the open development lifetime of the OpenLayers project. With contributions both technically and socially, Tim is one of the most important contributors to the project today, and having him together with the other three developers that we have sprints with in Cambridge can only amplify the quality and quantity of code coming out of this sprint.

Many thanks to MetaCarta for making this happen, and also to the developers of the vector branch, who have put so much hard work into making the idea of client-side feature rendering a possibility. Without the hard work of people like Cameron Shorter, Bertil Chapuis and Pierre Giraud and other developers from Camp To Camp, and the numerous contributors of both testing and commenting on the code and process, there is a lot of really solid work that doesn’t look like it will be particularly difficult to merge back into trunk. They’ve done an excellent job in making it this far, and this week we hope to carry the baton home, by bringing the code back to trunk with documentation, tests, and support.

I’m hoping to keep up blogging as we go through this sprint to keep the outside world appraised of what’s going on: I aim for as much transparency as possible. Hopefully we’ll be diligent enough to keep those of you who are interested informed.

– Christopher Schmidt, MetaCarta Labs OpenLayers Hacker

For the first time yesterday, the MetaCarta Labs WMS servers served more than 1 Million tiles in a 24 hour period.

A big part of this has been a large influx of visitors to the OpenLayers homepage from a link from meneame.net, a Spanish language equivilant of Digg. The OpenLayers home page has served about 10 times as many tiles as it does on an average day as a result: yesterday, 360,000 tiles were served for this page.

Our participation in FlashEarth has kept us serving a healthy 200,000 tiles a day. This is pretty standard — some days it’s higher, some days it’s lower, but that’s the average.

Both FlashEarth and the OpenLayers homepage serve WMS-C tiles, against our TileCache server. There’s also one other large tile user who doesn’t use this server: the WarWide OpenLayers-based war strategy game. Currently, they are using our slower WMS server to the tune of 300,000 tiles a day.

It’s not really any sweat for us: with the optimizations we have in place, the load on the tile server never really gets above 1 or 2. TileCache is a huge help in this regard: I don’t expect we would be able to serve the full load of rendering all the WMS images if we were rendering them all each time. In fact, I’d like to communicate with the people at WarWide that they should change to WMS-C, but my emails so far have gone ignored. The WMS-C server is on average about 10 times faster than the non-prerendered tiles, and I’d like to see everyone see as good of performance as is available for their use case.

Still, a million tiles a day — an average of 15 requests/second for the 24 hour period — is nothing to sneeze at.

TileCache 1.4 has been released.

Changes in this version include:

• Added wsgi handler, from Ehud Shabtai.
• Added standalone HTTP server, using wsgi handler, from Ehud Shabtai.
• Added fastcgi implementation, using wsgi handler, from Ehud Shabtai.
• Improved documentation, including patch from Eric Lemoine, Bill
  Woodall.
• Improved support for running as CGI under IIS, from Suki Hirata.
• Support for ‘maxResolution’ layer option.

TileCache is available from:

• TileCache Homepage
• .tar.gz
• .zip

About TileCache:
TileCache provides a Python-based WMS/TMS server, with pluggable caching mechanisms and rendering backends. In the simplest use case, TileCache requires only write access to a disk, the ability to run Python CGI scripts, and a WMS you want to be cached. With these resources, you can create your own local disk-based cache of any WMS server, and use the result in any WMS-C supporting client, like OpenLayers, or any TMS supporting client, like OpenLayers and worldKit.

You may have noticed that we replaced the link to the Labs GeoParser APIwith a link to MetaCarta Web Services at developers.metacarta.com. Thesenew web services are faster, more robust, better documented, and generallymore capable.

The terms of use is currently non-commercial. In the near future, we willcreate a small-site-friendly pricing model.

We are actively improving the services and welcome your feedback.

Here’s a quick tour:

Search

Search allows you to query MetaCarta’s curated collections of documentsand display the results on maps using AJAX clients.

If you would like us to add your site to our curated collections, email usdirectly. We will provide APIs for adding your content in the future.

Location Finder

Location Finder looks up coordinates for locations in a structuredgazetteer. It includes the ability to include geometries, and provides information useful for building maps of these places.

GeoTagger

GeoTagger extracts and resolves geographic references from unstructureddocuments.

Query Parser

Query Parser extracts the best geographic reference from a search query,so you can direct a user to a geographic search interface.

Accounts

You can sign up for an account, and then create API keys and monitor the number of hits generated by yourapplications using those API keys.

We look forward to your feedback.

– John Frank, MetaCarta CTO