Biodiversity, the wealth of life on Earth, is crucial for the functioning of ecosystems and the provision of services that sustain human well-being. But as human activities continue to threaten biodiversity, it is more important than ever to understand and monitor this valuable resource. One key tool in this effort is the sharing and gathering of data to provide insights into the distribution, ecology and characteristics of different species and ecosystems. Since ecological patterns can only be identified through the aggregation of many individual observations (Underwood et al. 2000) sampled data should be stored systematically and well documented in easily accessible, centralized data repositories to gain a comprehensive understanding of our planet's biodiversity (von Wettberg & Khoury 2022). Therefore, Orr et al. (2022) identified the interoperability of biodiversity data as one of the bottlenecks of the Post-2020 Global Biodiversity Framework which is still under development as a tool of the Convention of Biological Diversity (CBD) to significantly lower biodiversity losses by 2050. To take this into account the DFG (German Research Foundation), one of the largest national research funding bodies in Europe, has now set out extensive requirements on the handling of research data when approving research proposals (DFG 2022).

In general, two different types of scientific database concepts can be distinguished (adapted from Porter 2000) for the storage of biodiversity data. “Wide” databases collecting as much data as possible from the relevant scientific field and “deep” databases that specialize in certain types of data. An example of a “wide” database in the biodiversity context is the GBIF database infrastructure. From primitive unicellular organisms to slime moulds, invertebrates, and vertebrates to the fossil Tyrannosaurus rex, any occurrence record or checklist or sampling event dataset of any species that has ever lived on our planet can be stored there (Telenius 2011, see also data types in the GBIF Integrated Publishing Toolkit (IPT) User Manual). This concept of a mega-diverse data aggregator leads to an enormous heterogeneity of the spectrum of available datasets. Another limitation is the data quality, i.e. the heterogeneous use of the standard information elements, e.g. within Darwin Core and ABCD (Access to Biological Collection Data), within and across the single datasets, which is regularly criticized in studies (Beck et al. 2013, Ferro & Flick 2015, Maldonado et al. 2015, Yesson et al. 2007, Zizka et al. 2020). Therefore, a more complex analysis of the data is not easy and requires careful data selection and several data cleaning and data preparation steps, which can be time-consuming.

“Deep” databases focus on a specific type of data, in the biodiversity context, for example occurrence or sampling-event data restricted to specific taxonomic groups, single habitats or geographic regions or traits of specific taxa. Examples are Edaphobase (Burkhardt et al. 2014) focusing on soil zoological data, Critterbase (Teschke et al. 2022) for benthic taxa, GlobalAnts for ant trait data (Parr et al. 2017) or the Azorean Biodiversity Portal (Borges et al. 2010). Since the datasets here are limited to taxonomic groups, habitats or regions, the data naturally exhibit an increased homogeneity compared to data in wide databases. Also improved are data quality and data validation, as the total data flow is considerably smaller and therefore much more controllable.

The ARAMOB data repository

Here we present the database or data repository ARAMOB of the “deep” kind, restricted to spiders (Araneae) as a taxonomic group and focussing on Central Europe as the geographic region (although so far with a very strong predominance of data from Germany) and systematically sampled community data (spider assemblages). Spiders are an appropriate model group for this kind of database project, as they have several advantages. First, arachnologists are well-organized worldwide. There is the International Society of Arachnology (ISA) and many continent and country-related societies, e.g. the European Society of Arachnology (ESA) and the German language-based Arachnologische Gesellschaft. Second, there is a strong and commonly agreed taxonomic backbone – the online World Spider Catalog (2023). Taxonomy and faunistics of this group have been extensively studied in Central Europe, resulting in a plethora of online resources available to the arachnological community, e.g. araneae Spiders of Europe (Nentwig et al. 2023) and the Picture Gallery of Belgian and French spiders (Oger 2023) for identification, the Atlas of the European Arachnids (Arachnologische Gesellschaft 2023a) for occurrence data and the World Spider Trait database (Pekár et al. 2021). However, a systematic collection of existing sampling-event data on spiders was so far lacking, despite a large known quantity of dark data, which is mainly opportunistically collected data (Bowser 1986, Michener 2015) from small-scale studies, not available after publication of the results (Heidorn 2008, Reichman et al. 2011). Dark data are maintaining the situation of severe impediments in invertebrate conservation (Cardoso et al. 2011).

The ARAMOB data repository is one outcome of the DFG-funded project “Semantic enrichment and mobilization of data in distributed repositories for taxonomy and ecology of spiders” (2016 - 2019, grant number: 316372061). The main objective of this project was to create a curated data repository for spider assemblage (i.e. sampling-event based) data that can be used by researchers to analyse the ecology, habitat preferences and temporal changes in population sizes of these organisms. To this purpose, research groups from Staatliches Museum für Naturkunde Stuttgart (SMNS), the IT Center of Staatliche Naturwissenschaftliche Sammlungen Bayerns (SNSB) and the Institute for Environmental Research (IFER) of RWTH Aachen University, under the leadership of Staatliches Museum für Naturkunde Karlsruhe (SMNK), have joined forces. The SNSB and SMNS are established GFBio Data Centers (see Web links and abbreviations) with a focus on biodiversity data (Weibulat et al. 2023).

The volume of data that could be mobilized and made accessible to date with the help of workflows developed in the project (and reported elsewhere) shows the mass of biodiversity data that is still waiting to be tapped. The ARAMOB project, in collaboration with its partners, has likely amassed one of the largest and most easily accessible collections of systematically collected spider datasets in Central Europe, with community data available from more than 1170 sites with over 450.000 individuals from 660 species. This is particularly noteworthy given that only three institutions contributed their previously immobilized data to the ARAMOB data repository. Other notable collections of similar sizes are the dataset of Hänggi et al. (1995), which is not publicly accessible, and the Atlas of the European Arachnids (Arachnologische Gesellschaft 2023a) which incorporates mainly occurrence data for understanding spider distribution and is already frequently used by scientists (Narimanov et al. 2021, Purgat et al. 2021, Štokmane & Cera 2018, Wersebeckmann et al. 2021, Wiśniewski et al. 2018). Most of the atlas data were not systematically collected, posing challenges for their application in addressing ecological research questions requiring quantitatively comparable datasets. Datasets originating from non-systematic and non-standardized sampling approaches are often deposited in natural history museums, such as unsystematically collected collection specimens or observation data from citizen science portals like inaturalist.org (iNaturalist 2023). For niche modelling, biogeographic, conservation or taxonomical research, this kind of data still represents immeasurable value (Casas-Marce et al. 2012, Rowe 2005, Sillero et al. 2021). The SMNK and SMNS regularly deliver such data to the Atlas, or deliver the data in a machine-readable format to GBIF (see Web links and abbreviations).

ARAMOB was intended to be a supplement to the existing Atlas and GBIF by providing high-quality data packages from studies under a specific research hypothesis or question that can be used in meta-studies. Currently, data exchange between the two arachnological data collections (Atlas, ARAMOB) is done manually, but plans to automate this process in the future are being considered. ARAMOB is focused on opportunistic biodiversity data (Bowser 1986, Michener 2015). This is usually a self-contained dataset that has been collected over a short or long period of time, to address a specific research hypothesis or question. Although these data have well-documented information on sampling sites and protocols, environmental variables, and other relevant details, like their collection and compilation are influenced by the scientific question being addressed. Opportunistic datasets are often generated in small-scale research projects that are completed within a defined timeframe, typically three to six years. Heidorn (2008) outlined the usual manpower behind this kind of projects as follows: “… with one lead researcher with part-time commitment to the project and perhaps two or three graduate students or part-time staff scientists”. The collection, processing and evaluation of the data is also performed with the same care as in long-term studies. However, various reasons lead to the situation that the data from these kinds of projects are often not accessible at the end. One of the main reasons can be found in the reward system of the academic environment. Scientists get recognition and funding when they publish findings derived from their raw data in highly condensed scientific articles. Making the raw data publicly available does not initially benefit the responsible scientist. This is where data papers come into focus.

To our knowledge ecological data on spiders are collected regularly and in large quantities (not only in Germany), but there is a pressing need for action to ensure that these collected datasets are not left unused or stored locally after the conclusion of research projects and theses, published or not. Only biodiversity data published and stored in standardized formats will be usable for science in the future. And it is crucial to store such data in a structured manner that retains its complexity and critical metadata for cross-study analysis, while also making it easily accessible to researchers and decision makers. As a case study, the research data repository ARAMOB, which focuses on systematically collected spider assemblage data, can be utilized to evaluate its advantages, challenges, and potential applications. Members of the Arachnological Society, and all other arachnologically interested scientists, may carry out such work.

Which software is used by ARAMOB?

Various types of database management systems exist, with relational databases historically holding a predominant position as the most extensively utilized (Porter 2018). In Germany, the German Federation for Biological Data (GFBio e. V.) recommends two systems, Diversity Workbench (Triebel et al. 1999) and BEXIS2 (Chamanara et al. 2021), for the management of biodiversity data (Diepenbroek et al. 2014, Lotz et al. 2012, Karam et al. 2016). Diversity Workbench (DWB) is a modular, free (GPL 3 licence) virtual research environment aimed at managing and analyzing bio- and geodiversity data and was started in 1999 at the SNSB IT Center and has been continuously developed ever since (Triebel et al. 1999). It supports a variety of data types including Taxon data, Biodiversity and Occurrence data, Environmental Biological and Ecological Data, Non-Molecular Analysis Data and Molecular Sequence Data, and allows users to customize data entry forms and the database structure to suit their specific needs. To cope with the possible volume and large heterogeneity of biodiversity data, there are different modules within the DWB framework depending on the managed type of information (Triebel et al. 2007).

The utilization of Diversity Workbench has the advantage of facilitating standardization of data through the specification of terminologies, ontologies and taxonomies at an early stage of the data life cycle (Recknagel & Michener 2018), thus promoting the creation of FAIR data (Harjes et al. 2020, Karam et al. 2016, Schneider et al. 2019). This data can subsequently be published via a BioCASe (see Web links and abbreviations) data pipeline as an output of the DiversityCollection module, based on the ABCD (Access to Biological Collection Data) standard schema, and accessed via GBIF and NFDI for increased FAIRness (see Harjes et al. 2020, Novotný et al. 2022, Weiss et al. 2018). Modern data standards such as Darwin Core and ABCD have emerged to further standardize and improve the management and sharing of biodiversity data (Holetschek et al. 2012, Wieczorek et al. 2012). Both standards provide a common framework for describing and sharing primary biodiversity data, allowing for greater interoperability and accessibility of data across different platforms and systems. Given the significance of data security and the requirement of Microsoft licences, the operating environment of the Diversity Workbench server incorporates PostgreSQL cache databases. Before external release, data in the primary operating environment must be transferred to those cache databases.

Data quality in ARAMOB

To ensure the quality of the data in the ARAMOB database, only datasets sampled systematically and in a methodologically consistent manner can be included. In a first step, structures, templates, and workflows were developed to mobilize data from a wide variety of sources, e.g., publications, studies, project reports, or theses, and to transfer them into a consistent terminology. When possible, data was enriched with additional information, such as coordinates or habitat types, if not available beforehand. By minimizing the heterogeneity and inconsistency that can plague biodiversity databases, ARAMOB aims to provide a freely accessible ‘clean’ research environment that allows robust and reliable analyses with community datasets. In the further course of this work, specific methods, queries and algorithms were developed to automate the data extraction from the database and convert it into formats that are suitable for statistical analysis.

The concept of data quality may seem intuitive, but it encompasses numerous nuances that require closer examination. A dataset that comprises a species name, date, and coordinates, generated by a photo identification app, for instance, may have a high level of data quality (see RfII 2019), if all information is accurate and may be suitable for, e.g., modelling species distributions. Nonetheless, this type of dataset may not be usable for meta-analysis with systematically collected community data. Hence, data quality is primarily contingent upon the user's specific requirements for the analysis. According to Veiga et al. (2017), assessing biodiversity data quality involves using a set of quality metrics to determine its suitability for a specific use. However, even though data may have the same quality level, it may still have varying levels of suitability for different purposes. To determine the fitness of data for use, three crucial interrelated components must be considered: the intended use, the type of relevant data, and the criteria for determining ‘fitness’ for that data in the context of the use. In ARAMOB, the primary focus is on systematically collected spider assemblage data obtained through standardized methods. Thus, when referring to data quality here, the emphasis is placed on not just the accuracy of the data, but also on determining the additional information necessary in community data to facilitate meta-analyses and enhance their validity.

How can research data become and remain sustainable in the long term?

One early initiative to support the needs of researchers and institutions in managing and sharing their biodiversity and environmental data was the German Federation for Biological Data (GFBio), which emerged from a DFG-funded project and later became an established association (GFBio e. V.) in 2016 (Diepenbroek 2015). The goal of GFBio e. V. is to provide a centralized infrastructure to help scientists managing and sharing biodiversity data along the data life cycle (Fig. 1).

Fig. 1:

Idealized, schematic data life cycle. GFBio Training Materials: Data Life Cycle Fact-Sheet: Data Life Cycle: Plan. Retrieved 27 Jun 2022 from  https://www.gfbio.org/training/material/data-life-cycle/plan/.

This included the development of a portal for accessing data, as well as tools for data management and analysis (Authmann et al. 2015, Diepenbroek et al. 2014). GFBio also serves as a facilitator for various data centers, providing a platform for researchers to submit their data for permanent storing and archiving. Later, in 2019 the National Research Data Infrastructure (NFDI) was founded with a broader scope and the goal of creating a sustainable infrastructure for research data management in Germany across all domains, not limited to biodiversity. It aims to create a network of research data infrastructures to support the storage, management, and preservation of research data in a variety of disciplines (Hartl et al. 2021). To date, the NFDI is split into 26 different consortia, each of which is focused on a specific research discipline. For biodiversity and environmental data, this is done by the NFDI4Biodiversity consortium, supported by 49 partner organizations from (citizen) science and government (Glöckner et al. 2020). This also includes GFBio e. V., whose services and tools will be integrated into the broader infrastructure being developed by NFDI4Biodiversity, providing researchers with a comprehensive set of data management tools and services (Weber et al. 2021).

NFDI4Biodiversity carries out a bottom-up approach, by involving different actors like research institutions and projects, natural science societies and government agencies in this process (Weber et al. 2021). One of the so-called Use Cases is no. 16 by the Arachnologische Gesellschaft e. V. (AraGes), SMNK and SNSB. In this way, the ARAMOB project continues to be integrated as a use case for data provisioning within the NFDI4Biodiversity consortium (Glöckner et al. 2020). This integration aims to enhance the interoperability of ARAMOB and Atlas data, enabling the seamless transfer of data between the NFDI4Biodiversity tools and services currently under development. In order to ensure data quality and long-term access of the ARAMOB database as a data provider at the SMNK, a partnership has been established between the SMNK and the AraGes with involvement of the SNSB and SMNS respectively as data centers. This agreement for distributed responsibility intends to ensure permanent data storage through the integration of the arachnological data stock into the technical infrastructure of the three museum institutions, realized as an autonomous Diversity Workbench (DWB) Data Repository ARAMOB, administrated and technically curated by two GFBio data centers: SMNS (data in DiversityCollection, DiversitySamplingPlots and DiversityProjects) and SNSB (freely available thesauri and name services, data in DiversityTaxonNames and DiversityScientificTerms as well as DiversityGazetters – via database server with open access). The scientific curation and the public data portal and website ARAMOB is under the responsibility of the SMNK. The set of explorative data analysis tools (see below) was developed and integrated by the working group at the RWTH Aachen university.The AraGes society provides expertise to aid in the collection, organization and quality assurance of ecologically relevant data for ARAMOB (Fig. 2).

Participation and benefits of ARAMOB for researchers

The main, and most significant, advantage for scientists lies in the provision of high-quality data packages dedicated to research purposes. By adhering to the aforementioned stringent quality criteria, these datasets become readily usable for meta- and cross-studies, enhancing the robustness and reliability of scientific investigations and results.

Fig. 2:

Distributed responsibility for service, tool and data management, archival and publication of ARAMOB datasets.

Moreover, this initiative extends its benefits to (citizen) scientists, professionals from public authorities, and nature conservation organizations, empowering them to evaluate the dataset using a specially designed explorative tool package independently and interactively (Bach et al. in press). This sophisticated tool package enables comprehensive data analysis, encompassing various essential parameters. For instance, researchers can focus on study of phenology, ecological parameters (including the factors shading and humidity), habitat preferences, altitudinal distribution patterns, and encounter companion species of the diverse spider species documented within the database. By facilitating such thorough and customizable analyses, this comprehensive approach serves as a valuable asset in advancing our understanding of spider species diversity and ecological dynamics. Consequently, the implementation of this system fosters collaborative research efforts and empowers researchers and stakeholders alike in making informed decisions concerning biodiversity conservation.

There are currently two options for making your own data available through the ARAMOB data repository. Institutes may either become part of the ARAMOB repository through their own Diversity Workbench database or contribute their datasets through the SMNK. A new and attractive way to make data from systematic ecological studies (results published or not) available is the publication of a data paper in Arachnologische Mitteilungen/Arachnology Letters (Raub et al. 2023).

Access to the ARAMOB data repository

Data from the ARAMOB data collection can be accessed at  https://aramob.de/en/data/data-exploitation. The data portal is bilingual (German, English) and includes information on the former project, background and networks of data management and – central – the part where the data can be accessed. Directions for use help users to effectively work with the filter and search functions and explore the resulting data collection. Search results are shown as a map of sampling localities, a study data list (including number and gender of the individuals of a species per sampling event, sampling dates, localities, habitat types, sampling method and the specimen-ID) or a species list. The lists can be exported as CSV files for individual data management and further analysis. A set of metadata of all project-based datasets included in the search results are offered to facilitate the selection of appropriate data for specific analytical approaches and interpretation of the results. As mentioned, the available data were quality-checked by us, however we cannot give a guarantee for correctness and accuracy. The responsibility for a data selection, the results and consequences of use have to be taken by the user. All the data presented on the ARAMOB website can be used under a Creative Commons 4.0 International Licence (CC BY 4.0). Downloaded/analysed data collections have to be cited as: “ARAMOB data accessed via  https://aramob.de [Date of access]”.

The complete dataset can also be analysed with a set of explorative data analysis tools (Bach et al. in press) accessible at  https://aramob.de/en/data/statistic-tools/. Operating on a server, it removes the necessity for users to install or download extra software. Furthermore, members of the AraGes retain the ability to access the data within the Diversity Workbench virtual environment, which offers additional information such as methodological metadata and the full range of tools integrated in the complex software.

Concretely, for ambitious users, ARAMOB offers: