AI on geo data is receiving high attention currently. However, literature that AI models are primarily reliable on their training data (e.g., 85%) and drastically unreliable outside it (e.g., 20%). The illustration shows a field fruit classification in the Netherlands, applying a model produced by Wageningen Research. It is clearly visible how the model hallucinates over water (bottom left), providing incorrect results.
In this case, a geographical restriction (e.g., based on cadastral data) would already be sufficient to constrain the model to valid situations. However, in general the challenge can be of arbitrary complexity and often not decidable at all.
Currently, (i) there is no general method for characterizing the validity of models in terms of spatial, temporal, and content-based criteria; (ii) there is no method for using a validity characterization to protect specific user requests; and (iii) there is no method to automate these mechanisms within a general spatiotemporal geodata infrastructure. Published models – for example on HuggingFace with currently 827,859 models – at best have a textual description of their applicability, often overly optimistic and generic, and currently not machine-readable.
We summarize the capability of a server to recognize invalid data/model combinations and react appropriately as "model fencing" to express that AI models during their inference get constrained to their individual application conditions ("comfort zone"). FAIRgeo attempts to structure the field and find decision criteria for selected situations.
PS: we see the FAIRgeo interpretation of "FAIR" as compatible with the widely used expansion as "Findable, accessible, interoperable, reusable" - it is all about more usability and trust of (big) data.
The vision of FAIRgeo is to increase the reliability of "AI-as-a-Service" particularly in the context of information infrastructures for the automated analysis of large-scale geo-raster data such as satellite image time series and weather forecasts.
To this end, automated methods are to be researched that evaluate the reliability of model evaluation before executing specific processing requests containing calls to AI models – and, if the situation is deemed insufficiently reliable, inform users appropriately (in extreme cases by rejecting the request but also possibly by hiding unreliable parts, such as water areas in the example above). This requires suitable machine-readable descriptions of the AI models, which are also subjects of investigation.
Expected benefits center around increased reliability of ML models on geo datacubes, without additional overhead on users:
Building on results from BMWi AI-Cube and EU FAIRiCUBE, rasdaman allows WCPS datacube queries to contain ML inference, like in the following schema:
for $s2 in (Sentinel_2),
$m in (CropModel)
return encode( nn.predict( $s2[…], $m ), “tiff“ )
As it is possible now on principle to embed any model into any query the question of model trust becomes even more pressing.
Concretely, the server should be able to decide whether a given model and a given spatio-temporal evaluation region provide sufficient quality when applied, and consequently allow, disallow, or adjust the query.
This primarily involves evaluating metadata, but potentially also further information from the server (which requires that this data is machine-readable). This is likely to lead to the definition of new metadata structures, which needs to be aligned with current research on Analysis-Ready Data (ARD).
These works will closely follow the relevant standards for multi-dimensional raster data / data cubes, especially ISO/OGC Coverage Implementation Schema, which is also used in EU INSPIRE; if necessary, insights into sensible extensions of the coverage structure may be brought into standardization.
|
Constructor University Bremen gGmbH c/o Prof. Dr. Peter Baumann Campus Ring 12 28759 Bremen Germany |
Contact: Peter Baumann,
Images credits: tbd
All external links implemented on this site have been checked for conformance with the spirit and intent of this site, and have found conformant at this time. However, the maintainer of this site has no control over the (change of) contents of the referred websites and, therefore, does not assume any responsibility for the contents of referred websites.
The controller responsible for the described data collection and processing is named in the Constructor University imprint.
When you visit our website, the Web server temporarily stores usage information on our server machine (running in Germany) for statistical purposes in the form of a log in order to improve the quality of our website and to fulfill funding agency statistics reporting obligations. We do not transfer your personal data to third parties without your express consent.
When it comes to processing your personal data, the GDPR grants you certain rights as a website user: