SANY Software Components
SANY developed a number of reusable software components for building SensorSA applications, including the basic services, value added services with one or more service interfaces, frameworks for building SensorSA applications, and smart sensor components.
Following links give an overview of the reusable components developed within SANY IP:
OGC SWE Services
Sensor Web Enablement is a standardisation effort driven by the Open Geospatial Consortium. Through its liaison with ISO TC211, the OGC is closely involved in the development of often legally binding services published by ISO.
To date, OGC has successfully established its Observation and Measurement specification as a new work item in ISO. In the future, other SWE standards will follow. SensorSA inherits following SWE services and data models:
-
Sensor Model Language (SensorML) – defines a data model and encoding to describe processes and processing components associated with the measurement and post-measurement transformation of sensor observations.
Observations and Measurements (O&M) – defines a data model and schema for encoding measurements and observations.
Sensor Observation Service (SOS) – defines a service model and interface encoding for the provision of sensor measurements and observations, from simple sensors to complex sensor systems, both physical and virtual.
Sensor Planning Service (SPS) – defines a service model and interface encoding for the execution of sensor tasking and parameterization requests.
Sensor Alert Service (SAS) – defines a service model and interface encoding that enables subscription for and notification of situations of interest based upon continuous evaluation of incoming sensor observation streams.
Web Notification Service (WNS) – defines a service model and interface encoding for distributing incoming information to registered users via various communication protocols. It is often used for supporting asynchronous communication and routing urgent messages to whole groups of users according to their communication preferences.
Consequently, all existing SWE-compliant software components can be used in SensorSA Networks. In SANY, the reference implementations of the OGC SWE services developed by 52 North were used both "as is", and as the basis for own developments (e.g. within Cascading SOS). In addition, SANY partners also developed several prototype SWE services with SOAP and REST bindings.
Sensor Observation Service
The Sensor Observation Service (SOS) is the primary interface for accessing sensor data within the SANY architecture. The SOS is a standard Web service interface for requesting, filtering, and retrieving observations and sensor information. This is the intermediary between a client and an observation repository or near real-time sensor channel.
Within SANY, the SOS is used for Web enabling the sensor systems of insitu sensor networks. The SOS is interrogated from the individual application services, e.g. a spatial fusion service, a settlement prediction model service, a temporal fusion service or from the SSE workflow, and is the key building block to facilitate the interrogation of sensors and visualisation of measurements. The SOS implementations are based on the specification of the Open Geospatial Consortium, which comprises three different profiles:
-
The core profile includes the following mandatory operations:
-
GetCapabilities for requesting a self-description of the service. It provides Service Provider information, the list of supported operations, and other information about the service.
GetObservation, for requesting O&M encoded sensor data, i.e. this operation actually sends back the observation data requested by the user.
DescribeSensor for requesting SensorML encoded metadata about the sensors contained in a SOS instance; the SensorML data which is sent back describes the arbitrarily detailed characteristics of the sensors
and sensors systems used
-
RegisterSensor for putting new sensors into the SOS
InsertObservation for inserting sensor observations
-
GetFeatureOfInterest, for requesting GML encoded representations of features of interest
GetResult for periodically polling sensor data
GetObservationByID for retrieving observations by passing the IDs of the observations
In the SANY implementation pilots, three Sensor Observation Service implementations have been deployed: a) an internal development of SolData, b) the Open Source 52° North Sensor Observation Service and c) the Fusion SOS of Fraunhofer IITB, whose observations are coverages generated by a fusion process. All SOS implementations are of the 1.0.0 version of the OGC SOS standard.
Sensor Planing Service
Service The Sensor Planning Service (SPS) is the standard interface for all sensor, model and process tasking operations, whereby the latter two can also be handled with the Web Processing Service. SANY uses the Open Source 52° North SPS implementation as well as the Fraunhofer Fusion SPS that tasks fusion processes. Both are of the 1.0.0 version of the OGC SPS standard.
The following operations are specified within the SPS standard:
-
GetCapabilities for requesting a self-description of the service
DescribeTasking for requesting information that is needed for preparing a valid task, e.g. information about the necessary parameters.
GetFeasibility for checking if a task with certain parameters can be executed or not, e.g. if the sensor is busy it might not be possible to successfully submit a task.
Submit for sending a task that shall be executed by a sensor to the SPS.
GetStatus for checking the status of a task, e.g. completed, cancelled.
Update for updating the parameters of a task.
Cancel for cancelling a task.
DescribeResultAccess for retrieving information where the results of a task, e.g. the observations, can be accessed.
In addition to the operations specified by the OGC, the 52° North SPS offers additional functionality, which allows the administration of SPS instances. This includes:
-
Registration of new sensor plug-ins and instances
Unregistering sensor plug-ins and instances
Updating a registered sensor Getting detailed status descriptions of sensor instances
Updating information about the services providing access to the data collected by a sensor instance
The modular plug-in architecture allows the flexible integration of any kind of sensor data into an SPS instance. It offers an open, well-documented interface that can be used for easily developing plug-ins for connecting new sensors to your SPS. Through its plug-in concept, the integration of new sensors is fully supported and offers the flexibility to adapt an implementation to the specific requirements of your use case.
Web Notification Service
The Web Notification Service (WNS) is mainly used to support asynchronous communication patterns, where message-originating services have to deliver messages to clients on protocols other than HTTP.
Since SANY mainly analyzed alternative asynchronous interaction patterns using WS-Addressing and WS-Notification standards, the relevance of WNS as a message-indirection service for SANY was rather low.
The WNS used in SANY is an Open Source implementation done by 52° North and based on the OGC WNS Best Practice Paper version 0.0.9. It includes the following set of operations defined in the WNS specification:
- GetCapabilities for requesting a self-description of the service
- Register for allowing clients to register themselves to the WNS by proving information about their communication endpoint (e.g. their email address). The registration of single users as well as of user groups is supported.
- Unregister for removing a client from the WNS
- UpdateSingleUserRegistration for allowing a client to provide a new communication endpoint (e.g. a new email address)
- UpdateMultiUserRegistration for adding or deleting members from a registered group
- DoNotification for submitting a message to the WNS, which will be forwarded to the specified receiver
The WNS allows the integration of a broad range of communication means for sending notifications. Adding a new communication channel just requires the implementation of a new handler, which forms the bridge between the WNS
business logic and the communication system. By default XMPP and SMTP support are available. Furthermore a handler is available which supports sending SMS, fax and phone messages via the commercial HTTP-to phone/fax/ SMS service 'ecall.ch'.
Thus, the available Web Notification Service implementation already provides a broad range of initially supported communication protocols and a flexible architecture for easily integrating additional notification channels, which even qualifies the WNS to be used within applications setups based on OASIS standard 'WS-Notification'.
Web Service s Notification
Due to the increasing demand for more flexible and dynamic services, communication patterns are required that effectively allow for the decoupling between the notification publisher and subscriber.
The Web Services Notification (WSN) aims to standardise the way in which Web services interact by using ‘Notifications’ or ‘Events’: These specifications provide a standardized way for a Web service, or other entity, to disseminate information to a set of other Web services, without having to have prior knowledge of these other Web Services. They can be thought of as defining ‘Publish/Subscribe for Web services’.
These specifications have many applications, for example in the arenas of system or device management, or in commercial applications such as electronic trading. Another approach called WS-Eventing has been followed by the W3C, but it is not a W3C recommendation yet.
Sensor Alert Service
The Sensor Alert Service (SAS) defines an interface that allows nodes to advertise and publish observational data or alerts and corresponding metadata respectively. It also allows clients to subscribe to this data – or any other data that is produced by the SAS based on incoming messages from sensors – within specific thresholds. This observational data might be a single observation result, a complex observation result or even an alert in its nature. Within SANY the SAS has been used by end-users to subscribe to alerts and set alert conditions for the sensors of their choice, provided those sensors publish events through the SAS.
The SANY Sensor Alert Service implemenation is composed of four components:
- SAS server;
SAS client;
Multi User Chat (MUC) program;
Jabber server that deals with XMPP messages.
Two protocols are used for the communication between the sensor, the server and the client. The sensors use SOAP over HTTP to advertise themselves, and the XMPP protocol to publish their data. The client sends the subscriptions and receives the answer on SOAP over HTPP, and receives alert messages from the client on XMPP. The SAS uses the Extensible Messaging and Presence protocol (XMPP) to provide the push-based notification functionality, used for instant messaging. Communication between the MUC and the client or server is done over XMPP. The Web service SOAP bindings are document literal with a wrapped parameter style.
According to the OGC SAS Best Practice Paper version 0.9, the following operations are currently defined:
-
GetCapabilities for requesting a self-description of the service
Sensors advertise to the SAS the data they publish. In return they receive the information where (to which multi user chat) they can publish their data. Thus, three operations for managing such advertisements are implemented:
-
Advertise for allowing sensors to inform a SAS about the data they publish and returning the information where they can publish their data to
CancelAdvertisement for cancelling an advertisement
RenewAdvertisement for renewing an advertisement (in order to avoid that the advertisement expires)
Finally, three operations are available, which allow clients to subscribe to the information they are interested in and for managing these subscriptions
-
Subscribe for allowing clients to subscribe to the information they want to receive
CancelSubscription for cancelling a subscription
RenewSubscription for renewing a subscription (in order to avoid that the subscription expires)
When subscribing to certain information at a SAS you are able to use the filtering options defined in the SAS specification. This comprises
-
Spatial filtering, within a bounding box or at a certain feature.
Sensor based filtering, i.e. by sensor id.
Content based filtering, i.e. smaller than, greater than, equal to, not equal to.
Map and Diagram Service
Have you ever felt the need for more graphic functionalities in a WMS? The "Map and Diagram Service" (MDS) specifications define extensions to the OGC WMS and SLD standards with cartographic features such as diagrams, patterns and custom symbols expressed in Scalable Vector Graphics.
Diagram and other proportional symbol overlays are geo-referenced, and (technically) indistinguishable from regular WMS layers. As a consequence, all WMS client applications are compatible and can be used for viewing thematic layers that can be overlayed on regular cartographic materials. However, the creation of thematic maps on the fly requires dedicated MDS clients that are able to use the extended diagram parameters.
MDS defines several types of diagrams, including pie charts, bar charts, and line charts. Contours (see figure) are another important feature for the effective and intuitive representation of sensor data and interpolated fusion results.
A reference implementation of the Map and Diagram Service specifications is being implemented in the "QGIS map server", which is offered under GPL licence and available for download at:
Download: QGIS Server home page
The development of this service started in ORCHESTRA IP , and continued in SANY IP . The result is an open source application which is mature enough to compete with existing WMS implementations. If you need an application capable of cartographic symbolization while remaining compatible with existing WMS clients, then this is the application you have been looking for. Nevertheless, the usual disclaimer "this application is provided as is, with no express or implied warranty..." applies.
SensorSA Catalogue Service
Catalogue services play an important role for the discovery of resources. Conventional catalogues usually contain meta-information about available data and service resources. A typical user query to a conventional catalogue could include ‘give me all services supporting standard interface x’ or ‘give me all datasets in a specific region, where the responsible party is y’.
A catalogue used for the discovery of sensor related meta-information needs to address additional requirements. Typical queries for such a catalogue differ from the conventional ones. Some examples may be: ‘give me all 'temperature' observations in Austria of May 2009’ or ‘give me all entries supporting a specific sensor type’. Looking at these queries it is clear that additional search criteria and specific meta-information are needed, which reflect the needs from the sensor domain.
SANY addressed these challenges in developing a meta-information schema for the catalogue which follows the Observation and Measurement Model (O&M) from the OGC (Cox). This model is used by Sensor Observation Services, which provide the meta-information necessary to answer the queries above. Besides conventional catalogue resource types (data and service) SANY defined the following new meta-information resource types according to the O&M Model for the catalogue:
-
The ‘Feature of Interest’ representing the observation target
The ‘Observed Property’ describing the phenomenon to be observed (e.g. temperature)
The ‘Procedure’ representing a specific sensor, sensor system(s) or algorithm(s) used by a system.
Supported Search Criteria
Following illustration shows the resource types of the so called SANY Application Schema for Meta-information. Each resource type supports mandatory meta-information sections (table of contents and core elements) containing common meta-information elements like ‘title’, ‘keywords’ or ‘source url’. Further meta-information can be provided by specifying optional sections. This has been used for the description of the new resource types. Additionally the ‘Procedure’ resource type supports a SensorML section for a detailed description of sensors.
To support the possibilities of the SANY meta-information schema for the discovery process, the SensorSA Catalogue supports following search criteria ("queryables"):
-
‘FeatureOfInterest’ allows search for specific feature of interests, like a specific test region.
‘ObservedProperty’ allowa search for general phenomena, like ‘urn:ogc:phenomenon:temperature’.
‘Procedure’ supporting the possibility to search for general sensor types (e.g. accelerometer) and sensor instances.
‘DatasetType’ allows search for specific resource types like ‘Feature of Interest’, ‘Observed Property’ or ‘Procedure’.
Harvesting and Semantic Annotation
SensorSA Catalogue supports automatic creation of meta-information. Since the meta-information schema was designed according to O&M, which is also used by the SOS, the Catalogue relies on SOS operations GetCapabilities and DescribeSensor for automatically harvesting of the meta-information necessary to create SANY meta-information documents.
In addition to automatic creation of SANY meta-information documents, the SensorSA catalogue can also harvest INSPIRE meta-information. In this case the information provided by the SOS is not sufficient for the creation of
instance documents compliant with INSPIRE schemas and must be extended. For more information on this, please refer to chapter 8.1.3 of the SANY Book.
To overcome problems with the discovery of unharmonized URNs used in phenomenon or feature of interest descriptions of an SOS, the principle of semantic annotation has been tested. In the test example a SOS provides links to an ontology and a lifting schema, which describes the relation between the ontology concepts and the SOS phenomenons. In order to provide flexibility, the W3C recommendation ‘Semantic Annotation for WSDL and XML Schema’ (SAWSDL) has been used (Farell, Lausen).
The harvesting operation infers from the phenomenon to the related ontology concept and includes the concept into the created meta-information document. The catalogue client provides the user access to the used ontology. The advantage is, that a user using the ontology concepts for his search will get more results than a user performing a search with a-priori knowledge of phenomenons available in the catalogue: a search for the observable property ‘relativeHumidity’ leads to results of a specific SOS. A search for ‘rf ’ leads to results of another SOS using this observable identifier for
the very same phenomenon. But a search using the ontology concept ‘relative_moisture’ related to meteorology leads to results of both SOS.
SensorSA Security Framework
As an open architecture, SensorSA does not specify what any particular sensor or service does to protect itself. What the SensorSA does include, are security provisions to control access to services that are considered part of the SensorSA. The focus of the Security Framework is on access control. In a nutshell, access to a particular service is controlled in accordance with a policy specified for that service.
SensorSA security framework uses the SAML tickets to define Identities (individual users), Roles (attributes of Identities, indicating their function - e.g. "administrator" role), and Groups (sets of Identities). The Access Control Policies are specified using (Geo)XACML XML dialect.
The SensorSA Security Framework provides the software components that manage policies and identities, and enforce the policy rules. This includes:
-
The Identity Management & Authentication Service is responsible for the management of identities, their authentication, and the management of credentials and issuing of sessions. An instance of the Identity Management and Authentication Service acts as both authentication provider and identity provider. The service supports the management of groups (of identities) as a special kind of identity.
The Policy Management and Authorisation Service supports the management of policies, acting as policy administration point by allowing the management (select, create, update, delete) of (Geo)XACML policies,
as well as policy information point. Moreover, as an instance of the authorisation service interface it acts as policy decision point by providing a decision on whether some identity (e.g. a user or a service) is authorised to access a certain resource. The Policy Enforcement Service handles the necessary interaction (authentication and authorisation) to obtain the required access control decision and is independent of the controlled service (generic).
The Service Proxy mimics the controlled service and delegates the service request to the Policy Enforcement Service.
In addition to the services supporting the Service Access Control Pattern the Profile Management Service manages profiles and their relations to identities.
SensorSA security framework is published under GPL, and can be downloaded from the "Downloads" section of the SANY-IP web site.
Time Series Toolbox
AIT decided to continue the TS-Toolbox development after the project end, and publish most of the components and several example applications under terms of the GPL. Please follow this link for more information.
The Time Series Toolbox (TS Toolbox) is a set of software components and application programming interfaces that simplify the task of building applications that record, process, store and publish time series of observations.
In SANY the TS Toolbox components are used in the following applications: the Cascading SOS Service, the SensorSA Data Acquisition System, Generic Time Viewer, and in the Universal Data Pump – a simple application that provides a convenient way for transporting data from one application, service or file for which a TS-Toolbox data connector exists to another.
The TS Toolbox contains software components for the following functional areas:
-
Data connector components implementing access to data using various protocols and data formats
Core components interfacing with the connector components and providing specific additional functionalities like data processing or caching
Frontend components implementing interface functionality (user interfaces or software interfaces)
The functionalities implemented by TS Toolbox components provide application developers with higher-level building blocks than typical general purpose libraries, and allow rapid development of fully fledged applications. The TS Toolbox also includes example applications that can be either used as they are, or as a basis for developing more complex applications. At a time of publishing this information, the TS-Toolbox included following components:
Core Components
The TS Toolbox currently includes two reusable implementations of ‘core’ components:
-
Formula 3, a concise, text-oriented, high-level language for manipulation and transformation of time series data, enabling users to efficiently implement processing logic.
A caching component, which allows temporary storage of the data within an application and offers an easy way for including pre-fetching and caching capabilities in applications and services.
Frontend Components
The ‘frontend’ TS Toolbox components provide interfaces to users or other applications. Currently, three frontend components exist:
-
The SOS frontend component simplifies the task of developing applications with an OGC Sensor Observation Service compliant interface. It provides an implementation of a SOS service on top of the Time Series API, and is based on the 52° North SOS implementation.
The Data Pump frontend component implements functionality for transporting time series data from one data connector component to another. As such it can be easily used for creating applications to import and export data between applications.
The GTV frontend component implements functionality for building GUI client applications for accessing and displaying time series data.
Data Connectors
The ‘data connector’ TS Toolbox components provide access to data using various protocols and data formats; this includes three general purpose data connector implementations:
-
SOS connector: by using this connector implementation, applications can be interfaced to OGC SOS compliant services. Currently the SOS connector supports reading data from a SOS service, and storing data in a SOS service using the transactional profile of the SOS specification.
CSV connector: many legacy applications implement functionality to export data in simple comma- or tab-separated text files. The CSV connector component allows seamless integration of this type of data in TS Toolbox-based applications.
AnySen connector: an implementation of a data connector fetching data from a sensor driver that interfaces with physical sensors. The AnySen connector implements a flexible configuration scheme allowing it to be adapted to different vendor protocols.
The TS Toolbox also includes one example of a ‘legacy connector’, which is used to access the air quality data stored in the proprietary data acquisition system 'UWEDAT' monitoring systems (not included in GPL edition). Legacy connectors allow much tighter integration of legacy applications, e.g. real time access; storing altered data back to original service etc., than exporting and/or importing data to integrate those applications.
Cascading SOS
The Cascading SOS software developed by AIT is distributed under terms of the GPL, as a part of the Time Series Toolkit, . More information on Time Series Toolkit and Cascading SOS can be found in the Downloads section of SANY-IP.-
"Custom View to Data"
"Protocol Translation"
"SOS Proxy"
"Load Balancing"
"Value Added SOS"
"Sensor Data Store"
Georeferenced Timeseries Viewer
The Georeferenced Timeseries Viewer (GTV) is a generic desktop application and a toolbox for building specialized applications capable of presenting a common and combined view on time series data stemming from different sources, such as sensors, simulation models or data fusion outputs.
The GTV is implemented in Java on a richt client platform. It is an expert tool for the daily work of decision makers mainly in environmental authorities. The GTV is used as the basis for the ‘Data inspection client’ in Air Quality Monitoring (SP4) Pilot, and the design strongly reflects the requirements inherent to the air quality monitoring domain. Nevertheless, the GTV can be easily adopted to the needs of other environmental domains. The main design goals of GTV were:
-
to develop an expert tool capable of accessing and visualization of all data used within the SANY project;
to assure the GTV provides efficient and reliable support for domain
experts inspecting large amounts of data. to assure the GTV is easily extendible in order to answer the future user
requests for additional functionality
The main GTV components are a set of connectors to remote systems and a set of viewer windows, which can be combined and configured in a flexible way. Both, connectors and viewers can be easily added at runtime without the need to recompile or reconfigure the application.
GTV Scripting
The most interesting GTV feature is the possibility to process the data on the fly using the build in scripting features. In addition to evaluating the data from one or more sources, the script decides which visualization component(s) are invoked to display the result. For example, the data from two sources can be compared and the differences visualized using the symbols on a map or colour-coded tables.
In the future, the existing processing component may be replaced by more powerful TS Toolbox Formula 3 processor component, and the performance improved through inclusion of the caching component.
Viewer Components
Within the scope of SANY, the AIT developed three basic viewer components for visualization of the data in tabular form, as x-t graph or on a map. Additional GTV connectors and viewers can be easily added at runtime without the need to recompile or reconfigure the application.
Depending on the acutal configuration, the views may be either independent, or connected and capable of dynamically synchronising their context. For example, the symbols on a map (wind speed and direction) can reflect the time chosen by the user in one of the other two graphs, and browsing through data in tabular form can result in animated maps.
GTE testbed
The Georeferenced Timeseries Editor (GTE) is an example how to use the TimeSeries Toolbox V4.x.
Moreover GTE adds viewer components to the TimeSeries ToolBox and is itself also usable as a toolbox to implement applications.
There are 3 files you need if you want to play with the GTE:
- GTE_QuickStart.pdf tells about the installation and usage of the GTE application.
- gte_2009-11-18.zip is the GTE application. Just unpack and start
bin/gte. Details are in the document above. - GTE_Profiles_2009-11-18.zip contains some example profiles mentioned in GTE_QueickStart and a readme file.