Links and Resources

1. Metadata ("Data about data")

What is Metadata?

Metadata is the information we create, store and share to describe things, objects, data or services. And it helps us to obtain the knowledge we need about these things, objects, data, or services.

Every day we use metadata in order to find out about thing we are interested or use. We listen to music; post photos on Instagram; locate video on YouTube; connect with others via social media; and store contact lists on our mobile devices. All of this content comes with metadata—information about the item’s creation, name, topic, features, and so on. Metadata is key to the functionality of the systems holding the content, enabling users to find items of interest, record essential information about them, and share that information with others.

This video below introduces a metadata, and discusses descriptive and structural metadata types (for more information visit the webpage of authors.)

5 minutes about metadata

Metadata encodings

Metadata can be found in a variety of forms and encodings. In traditional information systems design, it might be stored as fields in relational database tables. A collection of metadata in this context is known as a record. 

XML. In the 2000s, XML (eXtensible Markup Language) emerged as a commonly used encoding, transfer, and occasional internal system storage mechanism for metadata. Metadata in XML exists as sets of files, called XML documents. XML defines elements, tags that signify that the values inside them have a certain meaning. Elements can also have other elements inside them, and it is from this feature that XML documents gain their structure. The use of XML is not limited to descriptive metadata; many different types of metadata can be stored in XML documents. To explore more about XML go to XML Tutorial or XML -Overview

XML supports multilingualism of metadata by providing a predefined attribute to indicate the language in which an element’s value appears.

RDF (Resource Description Framework) is a set of W3C specifications designed for metadata on the Semantic Web. Whereas XML models information as a tree, RDF models it as a graph, with small bits of information each connected to other small bits of information. No one entity or piece of data has primary importance in a graph; the network of information can be accessed equally at any point.  An RDF graph is best viewed as a whole, or as a simplified subset used in a given context for a given purpose. If you would like to explore RDF visit RDF Tutorial

2. Geospatial metadata

Geospatial metadata (also geographic metadata, or simply metadata when used in a geographic context) is a type of metadata that is applicable to objects that have an explicit or implicit geographic extent, i.e. are associated with some position on the surface of the globe. 

Metadata documentation has increased attention due to the evolution from Monolithic GIS to Web-GIS. Web-GIS allows spatial data sharing across distributed infrastructure. In this context spatial data documentation has become more important than ever as it allows the users the possibility to find the available spatial data shared by data providers to evaluate where the discovered data fulfill the requirements of their application in terms of scale, accuracy, actuality etc.

The ISO/TC 211 defined metadata by data about the content, quality, condition and other characteristics of spatial data or services and gives answer to the following questions:

• What spatial data (or service) are available? – Title, short description
• Who is responsible for data and metadata generation?  - Publisher/Owner
• Where (geographic location)? – Geographic coordinates (box)
• Why were the data collected? – Data collection purpose
• When were the data acquired? – Temporal information
• How to access the available data? – Data access and lineage

Metadata serve a wide range of applications, apart from data description also the organisation and maintenance of data. Complete metadata descriptions of the structure, contents and accuracy of all datasets is an important requirement for database design.

3. Metadata standards

 With an increasing focus on geodata sharing and geodata integration into corporate information systems, the development of metadata standards has become a priority. A few of the existing metadata standards are listed below:

• Metadata "in the Dublin Core style", is metadata designed for interoperability on the basis of Semantic Web or Linked Data principles. Metadata in this style uses Uniform Resource Identifiers (URIs) as global identifiers both for the things described by the metadata and for the terms used to describe them. 
• 
ISO 19139/19115 is the metadata standard dedicated to the spatial data documentation (ISO 19139:2007 and ISO 19115:2005) and identifies an extended set of metadata elements required to describe geospatial data. Explore yourself.
• 
ISO 19119 describes the structure of a metadata model for spatial web service instances to help searching, discovery and using available services (ISO 19119:2003) .
• 
FGDC (Federate Geographic Data Committee) standard is one of the earliest attempts to devise a standard for geospatial metadata. In 1994 the first version of this standard known as Content Standard for digital Geospatial Metadata (CSDGM) has been adopted. The second version was adopted in 1998.
• 
INSPIRE Metadata Implementing Rules define the guidelines for documenting the spatial data shared across EU Countries
National profiles: besides the acknowledged international standards, national metadata profiles are developed in order to document the spatial data according to the national rules and regulations: e.g. data access policies might be different from one country to another and therefore additional metadata elements might be required to document the new data sharing policies

There are two main rules that apply to metadata:


• a discrete resource should have a discrete metadata record (1:1 rule)
• 
metadata must be exchangeable and parseable (Metadata has be expressed into a language that can be understood by computers).


As you already know, data documentation is encoded into XML Language. XML encoding is not enough for consistent data documentation. Therefore, a XML Schema defining the metadata elements is also required. This schema defines metadata elements and has been specified by ISO/TC211.  Here you can have a look at ISO/TC 211 Metadata Schema and learn more about it.

4. National metadata profiles

National metadata profiles: besides the acknowledged international standards, national metadata profiles are developed in order to document the spatial data according to the national rules and regulations: e.g. data access policies might be different from one country to another and therefore additional metadata elements might be required to document the new data sharing policies.

5. Metadata Editors

Metadata creation and publishing implemented in three steps as metadata generation, validation and publishing. 

Metadata generation is a process of filling all information for mandatory elements. Metadata validation is assessing compliancy of the generated metadata with Standard. In addition, metadata publishing stored the generated and validated metadata in catalogue.

Metadata can be published using an online metadata editor or desktop metadata editor.

Desktop Editors

• GeoNetwork - GeoNetwork is a catalog application to manage spatially referenced resources. It provides powerful metadata editing and search functions as well as an interactive web map viewer.
• GeoCat - GeoCat Bridge, it is extension for Esri ArcGIS Desktop, which designed to make easy the process of publishing geospatial data and metadata on the internet.
• CatMDEdit is a metadata editor tool that facilitates the documentation of resources, with a special focus on the description of geographic information resources.
• The EPA metadata editor was developed by the Environmental Protection Agency (EPA).

Today, an increasing number of geoportal applications offer the possibility to generate metadata using the metadata editor hosted on the application. Metadata publisher fills in information about data or services to create metadata. Usually, if users need assistance, hints (help buttons) provide more information about what values should be filled in each field. 

• USGS Online Metadata Editor - A tool designed to help you document your dataset by asking questions about your data. Using this freely available tool you can: login and start new records or upload and edit existing ones; see all records you have created or uploaded in the past; save records and return later to complete them; and save completed records to your desktop.   Currently the tool supports XML output in the Federal Geographic Data Committee (FGDC) Content Standard for Digital Geospatial Metadata.
• ArcGIS online and ArcCatalog - Esri provides support for multiple metadata management options to meet specific needs or standards profiles. It supports these metadata standards. ISO 19115, FGDC Content Standard for Digital Geospatial Metadata (CSDGM), Dublin Core, INSPIRE and DCAT
• INSPIRE online Metadata Editor - The INSPIRE metadata editor allows data suppliers to document their assets following the INSPIRE metadata specification

Generated metadata is then added to a metadata repository (called also catalog). Exiting editors permit metadata publishers to validate their records before sending them to a catalog. 

Explore the comparison table of ISO Geospatial Metadata Editors: https://www.fgdc.gov/iso-metadata-editors-registry/editors

6. Metadata catalogue

The metadata catalogue offers the possibility to systematically search and find the documented spatial data and services. It stores and manages the indexed and searchable metadata “against which intelligent geospatial search can be performed within or among SDI communities” (GSDI Association, 2009). The mechanism to classify, register, describe, search, maintain and access information on geodata and services is defined differently within geospatial community: repository, metadata registry spatial data directory, clearinghouse etc.

If metadata is valid, then it is stored in the catalogue under approved status. Otherwise, additional editings of the metadata items are required before they are accepted as valid metadata entries.

Metadata can be published using an online metadata editor embedded on the same platform where the Catalogue Service is running or can be generated using desktop Editor and then imported into the implemented catalogue service.

To allow metadata query and harvesting across distributed platforms, the catalogue services have to be aligned to the existing cataloguing standards. Some of the available standardization efforts dedicated to the metadata catalogue services are outlined below:

• ISO 23950 Search and Retrieve Protocol (ANSI Z39.50) has been developed for the library community for accessing virtual catalogs;

• OGC Catalogue Services Specification;

• ebRIM (ebRIM (Electronic business Registry Information Model) is a meta model developed by the Advancement of Structured Information Standards (OASIS) for documenting and managing metadata items in a web catalog (OGC, 2007).

Catalogue services are required to support the discovery and binding to registered information resources within an information community.

7. OGC Catalog services (CSW)

OGC® Catalogue Service for Web (CSW)- supports the ability to publish and search collections of metadata for data, services, and related information objects. Metadata in catalogues represent resource characteristics that can be queried and presented for evaluation and further processing by both humans and software. 

OGC Catalogue interface standards specify the interfaces, bindings, and a framework for defining application profiles required to publish and access digital catalogues of metadata for geospatial data, services, and related resource information.

There are two types of CSW: 


• Transactional CSW allows users to publish, edit and delete metadata; 
• 
Read Only CSW supports only metadata search and retrieval.


CSW enables metadata query using metadata core (mandatory) elements.

Searching in distributed catalogs

The process of searching in distributed catalogs involves the following steps: users formulate as accurately as possible their data queries. User ‘s request is posted to a Catalog Gateway (interface) responsible for registering the metadata content of different catalog services. This interface has to implement the ‘harvest’ operation in order to gather the metadata records stored in different distributed catalogs.  Thus, search request is transmitted to all registered catalog services. Each catalog service manages metadata entries and returns discovered metadata items to the users. 

Figure. Distributed search with common information model (Source: OGC Catalog services (CSW)- general model)

Thus an OGC CSW client should be able to query any OGC CSW catalogue, regardless of the underlying information model, using the elements defined in the common record schema and understand the response. With this model it is possible for an OGC CSW Client to query an OGC CSW AP ISO Catalogue.