VectorMap District is now a layer in OpenSpace. See here for instructions on how to add it to a map. It doesn't appear by default - the default layers are listed here. For a description of all the available OpenSpace layers, including VectorMap District see here. VectorMap District appears between the 1:50K layer and the StreetView layer.
I've updated the cycle count map to include VectorMap district. Apparently VM can be configured and styled by the user but I've not found out how yet.
Incidently, with the latest version of OpenSpace, the OS have fixed the problem with Internet Explorer 8.
Sunday, 3 October 2010
Friday, 13 August 2010
Experiments with geoJSON and OpenSpace
I was asked recently to provide an online map of cycle count locations. The information was provided as point data using Ordnance Survey Eastings and Northings. For a change from Google Maps I decided to use the Ordnance Survey's OpenSpace mapping service.
For a further change, I decided to serve the point data to the map using geoJSON. geoJSON is a format for encoding geographic features. It is not as verbose as XML and unlike KML you can specify your co-ordinate reference system.
A further advantage of JSON in general is that it avoids the cross domain problems caused by making XMLHttpRequest calls in your JavaScript (see here for a description) i.e. where your web page is on a server in one domain and you want to call XML data from a server in another.
OpenLayers, which OpenSpace is built on, has functionality to allow the reading and parsing of geoJSON data.
The geoJSON was generated on-the-fly from data in an SQL Server table using an ASP.NET script. The data can be seen here:
http://www.gmtu.gov.uk/openspace/cycledata.aspx
Although it is less verbose than XML, geoJSON is, for me, less readable for the human eye.
The application can be seen here:
http://www.gmtu.gov.uk/openspace/cyclecount.aspx
If you look at the source code for the above page you can see that the JavaScript is fairly straightforward - read in the geoJSON data and loop through it, creating markers.
The only issue was with viewing the map in Internet Explorer 8. OpenSpace has problems with IE8, see this discussion.
Anyway a meta tag in the page head will force IE7 Compatibility mode which solves the problem.
For a further change, I decided to serve the point data to the map using geoJSON. geoJSON is a format for encoding geographic features. It is not as verbose as XML and unlike KML you can specify your co-ordinate reference system.
A further advantage of JSON in general is that it avoids the cross domain problems caused by making XMLHttpRequest calls in your JavaScript (see here for a description) i.e. where your web page is on a server in one domain and you want to call XML data from a server in another.
OpenLayers, which OpenSpace is built on, has functionality to allow the reading and parsing of geoJSON data.
The geoJSON was generated on-the-fly from data in an SQL Server table using an ASP.NET script. The data can be seen here:
http://www.gmtu.gov.uk/openspace/cycledata.aspx
Although it is less verbose than XML, geoJSON is, for me, less readable for the human eye.
The application can be seen here:
http://www.gmtu.gov.uk/openspace/cyclecount.aspx
If you look at the source code for the above page you can see that the JavaScript is fairly straightforward - read in the geoJSON data and loop through it, creating markers.
The only issue was with viewing the map in Internet Explorer 8. OpenSpace has problems with IE8, see this discussion.
Anyway a meta tag in the page head will force IE7 Compatibility mode which solves the problem.
Tuesday, 8 December 2009
Geoprocessing in the cloud
Interesting article in GeoInformatics (December 2009) entitled "Geoprocessing in the Clouds". The authors implement the 52North Web Processing Service on Google's and Amazon's cloud services.
They compare response times for both implementations and find that both scale well.
A more detailed paper describing the project can be found on the 52North website.
They compare response times for both implementations and find that both scale well.
A more detailed paper describing the project can be found on the 52North website.
Two contrasting views of GeoWeb processing
Now my MSc is done and dusted, I've decided to keep the blog going, focussing on the OGC's Web Processing Service.
I came across a couple of quotes recently. First from Jack Dangermond, ESRI CEO:
"So I think even 45 years ago when we were first inventing computer mapping at Harvard, our first attempts were to make beautiful maps. They weren't very beautiful; they were pretty crude. But very quickly thereafter, we moved from mapping into analytics, spatial decision-making to support land use planning and business planning of various sorts. So I think if you're asking me where will the web and the geospatial world move, it will be from simple map displays to analytics on the web. It'll move from simply a basemap with overlays on it to the notion of multiple distributed services that do spatial analysis, but combine various kinds of spatial data together and serve it back out as visualization communications."
full article...
And from an article on the Chrome JavaScript engine on the Spatial Miscellany blog:
"Chrome isn’t the only web browser to recognize the importance of working with JavaScript heavy web sites, for example, much work is being done on a new JavaScript engine for Firefox. Perhaps in the future, as this approach gains support, spatial analysis functionality can move from the server to the client, which would make for a more engaging web mapping experience?" full article...
So processing via services or on the client...?
I came across a couple of quotes recently. First from Jack Dangermond, ESRI CEO:
"So I think even 45 years ago when we were first inventing computer mapping at Harvard, our first attempts were to make beautiful maps. They weren't very beautiful; they were pretty crude. But very quickly thereafter, we moved from mapping into analytics, spatial decision-making to support land use planning and business planning of various sorts. So I think if you're asking me where will the web and the geospatial world move, it will be from simple map displays to analytics on the web. It'll move from simply a basemap with overlays on it to the notion of multiple distributed services that do spatial analysis, but combine various kinds of spatial data together and serve it back out as visualization communications."
full article...
And from an article on the Chrome JavaScript engine on the Spatial Miscellany blog:
"Chrome isn’t the only web browser to recognize the importance of working with JavaScript heavy web sites, for example, much work is being done on a new JavaScript engine for Firefox. Perhaps in the future, as this approach gains support, spatial analysis functionality can move from the server to the client, which would make for a more engaging web mapping experience?" full article...
So processing via services or on the client...?
Thursday, 16 April 2009
WPS implementation (nearly)
My WPS system now consists of a ASP.NET application wrapped around a ASP.NET Web Service. The web application takes the input and handles the getCapabilities and DescribeProcess requests. It also, after some error checking passes on the Execute request to the Web Service which processes the request asynchronously.
The WPS getCapabilities response can be seen here:
http://www.gmtu.gov.uk/wps/?service=WPS&version=1.0.0&Request=GetCapabilities
The DescribeProcess response can be seen here:
http://www.gmtu.gov.uk/wps/?service=WPS&version=1.0.0&Request=DescribeProcess&Identifier=DBSCAN
The DescribeProcess output is not yet complete as there is no description of the input and output data format. There is some error trapping so if you enter:
http://www.gmtu.gov.uk/wps/?service=WPS&version=1.0.0&Request=DescribeProcess&Identifier=noProcess
An error message will be returned.
The Execute process should return an XML response document that allows any client to monitor progress. In my implementation, for each request the WPS creates a file nnnnn.xml, where nnnnn is a unique integer. Both the wrapper application and the web service write to this file. When the Execute request has been parsed and all the inputs are present and correct, the wrapper creates the file with a ProcessAccepted message. When the Web Service is called, it overwrites the file with a ProcessStarted message. If there are any errors in processing then a ProcessFailed message is written. Finally a ProcessSucceeded message is written to the ResponseDocument. The idea is that the client can poll this document to check the process of the request.
This is sort of working but I have occasional errors “file being used by another process” when I try to overwrite the file or sometimes when reading it. I think this is due to the web server software (IIS) locking any file it has recently served. I’ll need to think of a way around this.
The WPS getCapabilities response can be seen here:
http://www.gmtu.gov.uk/wps/?service=WPS&version=1.0.0&Request=GetCapabilities
The DescribeProcess response can be seen here:
http://www.gmtu.gov.uk/wps/?service=WPS&version=1.0.0&Request=DescribeProcess&Identifier=DBSCAN
The DescribeProcess output is not yet complete as there is no description of the input and output data format. There is some error trapping so if you enter:
http://www.gmtu.gov.uk/wps/?service=WPS&version=1.0.0&Request=DescribeProcess&Identifier=noProcess
An error message will be returned.
The Execute process should return an XML response document that allows any client to monitor progress. In my implementation, for each request the WPS creates a file nnnnn.xml, where nnnnn is a unique integer. Both the wrapper application and the web service write to this file. When the Execute request has been parsed and all the inputs are present and correct, the wrapper creates the file with a ProcessAccepted message. When the Web Service is called, it overwrites the file with a ProcessStarted message. If there are any errors in processing then a ProcessFailed message is written. Finally a ProcessSucceeded message is written to the ResponseDocument. The idea is that the client can poll this document to check the process of the request.
This is sort of working but I have occasional errors “file being used by another process” when I try to overwrite the file or sometimes when reading it. I think this is due to the web server software (IIS) locking any file it has recently served. I’ll need to think of a way around this.
Monday, 2 March 2009
Final architecture?
After several weeks of exploring different solutions including remoting and using a Windows Service I have settled on using a Web Service.
The service takes as input a value for Eps (in metres) and a url pointing to a WFS service.
Currently I have the web service running on one physical server and the WFS service running on another. The service saves the results in a KML file on the server, which can be accessed via the web. The clustering alogorithm may take several minutes to run (depending on the amount of source data) so the service needs to be called asynchronously.
Firstly, the server code that executes the DBSCAN algorithm has to be encoded as a sub rather than a function since it cannot return a value to the client if we are running asynchronously. The sub definition needs the following attribute set:
SoapDocumentMethod(OneWay:=True)
The timeout has to be increased in the service application, using a setting in the Web.config file:
httpruntime executiontimeout="1200"
The service can be called from a web client. A reference to the service is added to the client and, for the page that calls the service, the sync attribute needs to be set in the @Page directive:
Async="true"
The client can then call the service:
Dim ws As New WPSservice.Service
ws.DBSCANasyncAsync(Eps, Url)
Where ws is a reference to the web service and DBSCANasyncAsync is the sub in the web service.
I’ve got this working as it should. The client calls the service asynchronously and control returns to the client page straightaway whilst the service continues to run in the background. When complete it writes out a results file to the server. This is all well and good, but since the client does not wait for the service to finish, how does the client know when the job has been completed, where the results are, if there are any errors? This is the next stage, I need to setup a mechanism for polling the service for the results and reporting any errors back to the client. I need to start getting my system looking like a Web Processing Service as specified by the OGC.
The service takes as input a value for Eps (in metres) and a url pointing to a WFS service.
Currently I have the web service running on one physical server and the WFS service running on another. The service saves the results in a KML file on the server, which can be accessed via the web. The clustering alogorithm may take several minutes to run (depending on the amount of source data) so the service needs to be called asynchronously.
Firstly, the server code that executes the DBSCAN algorithm has to be encoded as a sub rather than a function since it cannot return a value to the client if we are running asynchronously. The sub definition needs the following attribute set:
SoapDocumentMethod(OneWay:=True)
The timeout has to be increased in the service application, using a setting in the Web.config file:
httpruntime executiontimeout="1200"
The service can be called from a web client. A reference to the service is added to the client and, for the page that calls the service, the sync attribute needs to be set in the @Page directive:
Async="true"
The client can then call the service:
Dim ws As New WPSservice.Service
ws.DBSCANasyncAsync(Eps, Url)
Where ws is a reference to the web service and DBSCANasyncAsync is the sub in the web service.
I’ve got this working as it should. The client calls the service asynchronously and control returns to the client page straightaway whilst the service continues to run in the background. When complete it writes out a results file to the server. This is all well and good, but since the client does not wait for the service to finish, how does the client know when the job has been completed, where the results are, if there are any errors? This is the next stage, I need to setup a mechanism for polling the service for the results and reporting any errors back to the client. I need to start getting my system looking like a Web Processing Service as specified by the OGC.
Tuesday, 3 February 2009
Output to KML file and more thoughts on implementation
So far I have written the results from DBSCAN as INSERT statements to be run against a postgreSQL database. This is not going to be much use for a client application, so for the next step I’m going to generate a KML file showing the clusters (I’m going to omit noise points to keep the output filesize low). KML has now been adopted as a OGC standard and can be used as input to Google Maps and Google earth.
One problem is that my data is in OSGB co-ordinates whereas KML uses WGS84 lat/long. I've used the Grid InQuest co-ordinate translation program which comes with a DLL file that can be used in Visual Studio. My VB.NET application now outputs the results to a KML file. The code for generating the KML file was adapted from here.
Some sample results:
http://www.gmtu.gov.uk/nmg/citycentre.kml
http://www.gmtu.gov.uk/nmg/heatonsnorth.kml
If these urls are pasted directly into the browser, Google Earth will be fired up (if installed). To use the kml files in Google Maps you can paste the URL into the Search Maps text box on the Google Maps page.
Here is the Heatons North data in Google Maps:
View Larger Map
The number on each point is the clusterID (note that noise points have not been included.
As it stands the KML displays the individual points in each cluster. I perhaps need to think about generalization and generate polygons to display the clusters.
How does this fit in with my eventual implementation? My current thinking is this:
To develop An ASP.NET web application that acts as a “wrapper” around a VB.NET service/application. The wrapper is what the client sees and is in effect the WPS. This is the technique as recommended by Michael and Ames in their early evaluation of WPS. They suggest that the geo-processing takes place in a separate process. This will stop a badly performing process from bringing down the entire WPS (On Windows Server I can put the geo-processing tools in a separate application pool). My ASP.NET application will handle the GetCapabilities and DescribeProcess functions. It will also manage the Execute process by calling the necessary geo-processing service. The geo-processing service will take input from the wrapper and write its results to a results file (e.g. a KML file as above) It will also write to a log file, reporting its progress. The log file can be queried by the ASP.NET wrapper (on request of the client) allowing it to check on progress. The relevant section of the OGC WPS specification is:
"The mandatory Execute operation allows WPS clients to run a specified process implemented by a server, using the input parameter values provided and returning the output values produced. Inputs can be included directly in the Execute request, or reference web accessible resources. The outputs can be returned in the form of an XML response document, either embedded within the response document or stored as web accessible resources. If the outputs are stored, the Execute response shall consist of a XML document that includes a URL for each stored output, which the client can use to retrieve those outputs. Alternatively, for a single output, the server can be directed to return that output in its raw form without being wrapped in an XML reponse document.
Normally, the response document is returned only after process execution is completed. However, a client can instruct the server to return the Execute response document immediately following acceptance by the server of the Execute request. In this case, the Execute response includes a URL from which the response document can later be retrieved during and after process execution. The server can be instructed to provide regular updates to a measure of the amount of processing remaining if the process is not complete. This allows the client to determine the process status by polling this URL. "
The response document will be in effect my log file (an XML document), which while processing is ongoing will provide status updates when queried and when processing is complete will contain URLs to any output e.g. a KML file.
I think this method means generating a separate XML document each time the process is called (each file named with a unique reference number) unless I generate the response document dynamically from ASP.NET.
I need to examine further the approaches taken by 52 North and deegree in their WPS implementations.
One problem is that my data is in OSGB co-ordinates whereas KML uses WGS84 lat/long. I've used the Grid InQuest co-ordinate translation program which comes with a DLL file that can be used in Visual Studio. My VB.NET application now outputs the results to a KML file. The code for generating the KML file was adapted from here.
Some sample results:
http://www.gmtu.gov.uk/nmg/citycentre.kml
http://www.gmtu.gov.uk/nmg/heatonsnorth.kml
If these urls are pasted directly into the browser, Google Earth will be fired up (if installed). To use the kml files in Google Maps you can paste the URL into the Search Maps text box on the Google Maps page.
Here is the Heatons North data in Google Maps:
View Larger Map
The number on each point is the clusterID (note that noise points have not been included.
As it stands the KML displays the individual points in each cluster. I perhaps need to think about generalization and generate polygons to display the clusters.
How does this fit in with my eventual implementation? My current thinking is this:
To develop An ASP.NET web application that acts as a “wrapper” around a VB.NET service/application. The wrapper is what the client sees and is in effect the WPS. This is the technique as recommended by Michael and Ames in their early evaluation of WPS. They suggest that the geo-processing takes place in a separate process. This will stop a badly performing process from bringing down the entire WPS (On Windows Server I can put the geo-processing tools in a separate application pool). My ASP.NET application will handle the GetCapabilities and DescribeProcess functions. It will also manage the Execute process by calling the necessary geo-processing service. The geo-processing service will take input from the wrapper and write its results to a results file (e.g. a KML file as above) It will also write to a log file, reporting its progress. The log file can be queried by the ASP.NET wrapper (on request of the client) allowing it to check on progress. The relevant section of the OGC WPS specification is:
"The mandatory Execute operation allows WPS clients to run a specified process implemented by a server, using the input parameter values provided and returning the output values produced. Inputs can be included directly in the Execute request, or reference web accessible resources. The outputs can be returned in the form of an XML response document, either embedded within the response document or stored as web accessible resources. If the outputs are stored, the Execute response shall consist of a XML document that includes a URL for each stored output, which the client can use to retrieve those outputs. Alternatively, for a single output, the server can be directed to return that output in its raw form without being wrapped in an XML reponse document.
Normally, the response document is returned only after process execution is completed. However, a client can instruct the server to return the Execute response document immediately following acceptance by the server of the Execute request. In this case, the Execute response includes a URL from which the response document can later be retrieved during and after process execution. The server can be instructed to provide regular updates to a measure of the amount of processing remaining if the process is not complete. This allows the client to determine the process status by polling this URL. "
The response document will be in effect my log file (an XML document), which while processing is ongoing will provide status updates when queried and when processing is complete will contain URLs to any output e.g. a KML file.
I think this method means generating a separate XML document each time the process is called (each file named with a unique reference number) unless I generate the response document dynamically from ASP.NET.
I need to examine further the approaches taken by 52 North and deegree in their WPS implementations.
Subscribe to:
Posts (Atom)