Introduction

Audience: This document is targeted toward those who wish to use or

share information based on the Darwin Core terms using text files. It
provides technical details on how to construct these files and
complementary metadata files that describe their content.

This document provides guidelines for formatting and sharing Darwin

Core terms [TERMS] in fielded text formats, such as one or more
comma separated value (CSV) files. Data conforming to the Simple
Darwin Core [SIMPLEDWC] (CSV format and having the first row include
Darwin Core standard term names) can be shared in a single file, while
a non-standard text file can be understood using an [XML] metafile to
describe its contents and formatting.

More complex structure can be shared in multiple related files. The

description of content and relationships between files can be achieved
using the metafile. This guideline makes recommendations for the
simple case of a core file, upon which Darwin Core records are based,
and extensions that are linked to records in that core file. Specifically,
extension records have a many-to-one relationship with records in the
core file. For example, a core file might contain specimen records, with
one specimen per row in the file, while an extension file contains one or
more identifications for those specimens, with one identification per row
in the extension file, and with an identifier to the specimen for each
identification row. This example would allow many identifications to be
associated with each specimen.

1.1 Simple Example Metafile Content

A simple comma separated values (CSV) data file with the following
content:
ID,Species,Count
123,"Cryptantha gypsophila Reveal & C.R. Broome",12
124,"Buxbaumia piperi",2
can be described with the following Darwin Core metafile:
<?xml version="1.0" encoding="UTF-8"?>
<archive xmlns="[Link]
xmlns:xsi="[Link]
xmlns:xs="[Link]
xsi:schemaLocation="[Link]
[Link]
<core
rowType="[Link]
rwinRecord" ignoreHeaderLines="1">
<files>

<location>[Link]
on>
</files>
<field index="0"
term="[Link] />
<field index="1"
term="[Link] />
<field index="2"
term="[Link] />
<!-- A constant value has no index, but applies to all
rows -->
<field term="[Link]
default="urn:lsid:[Link]:collections:1"/>
</core>
</archive>

These same data could be understood without the metafile if the first
row of the CSV file contained the term names:

type,institutionCode,collectionCode,catalogNumber,scientificN
ame,individualCount,datasetID
PhysicalObject,ANSP,PH,123,"Cryptantha gypsophila Reveal &
C.R. Broome",12,urn:lsid:[Link]:collections:1
PhysicalObject,ANSP,PH,124,"Buxbaumia
piperi",2,urn:lsid:[Link]:collections:1

1.2 XML versus Fielded Text

Many resources exist on the web describing the advantages of

Extensible Markup Language [XML] over less structured content such
as fielded text. The Darwin Core Text Guide (this document) is not
meant to promote the use of fielded text over XML for data exchange,
but rather to provide recommendations for how to handle such data files
when necessary.
Two scenarios that might benefit from the use of fielded text are:

The transfer of large numbers of Darwin Core records and related

data from one database to another. Typically databases are very
efficient at exporting and importing comma separated text files.
The description of legacy data existing in a fielded text format,
such that it might be automatically understood and loaded into
another system. It could be that this system would then serve the
data in another format such as XML.

2. Metafile Content

The text metafile schema [TEXTSCHEMA] provides technical details for

the structure of a metafile by defining the elements and attributes
required to describe the contents and relationships between text files.
These elements and attributes, with descriptions and specifications for
their use in a metafile, are described in the following table.

2.1 The <archive> element

The <archive> element is the container for the list of related files (one
core and zero or more extensions). The <archive> element has just one
attribute, metadata.
Attributes
Attribute Descriptio

metadata Contains a qualified Uniform Resource Locator (URL) defining the location o
metadata is not prescribed, but a standardized format such as Ecological M
(FGDC), or ISO 19115 family is recommended.

Elements
Element

<core> An <archive> must contain exactly one <core> element, representing th

upon which records are based.
If extensions are being used, each record in the core data must have a u
order to associate extension records with the core record.

<extension> An <archive> may define zero or more <extension> elements, each rep
general file attributes described below, every extension entity must have
extension itself does not have to have a unique ID field and many rows c
2.2 The <core> or <extension> element

Attributes
Attribute Description

rowType A Unified Resource Identifier (URI) for the term identifying the cla
data represented by each row, for example,
[Link] for Occurrence records
[Link] for Taxon records. Additional
may be referenced by URI and defined outside the Darwin Core
specification. The row type defaults to the ambiguous
SimpleDarwinRecord. For convenience the URIs for classes defined
Darwin Core are listed below:
Simple Darwin Record
[Link]
Occurrence
[Link]
Event
[Link]
Location
[Link]
GeologicalContext
[Link]
Identification
[Link]
Taxon
[Link]
ResourceRelationship
[Link]
MeasurementOrFact
[Link]

fieldsTerminatedBy Specifies the delimiter between fields. Typical values might be ","
for CSV or Tab files respectively.

linesTerminatedBy Specifies the row separator character.

fieldsEnclosedBy Specifies the character used to enclose (mark the start and end of
field. CSV files frequently use the double quote character ("), but t
default is no enclosing character. Note that a comma separated va
that has commas within the content of any field must have an enc
character.
 Attribute Description

encoding Specifies the character encoding for the data file. The encoding is
extremely important, but often ignored. The most frequently used
encodings are:
UTF-8
8-bit Unicode Transformation Format.
UTF-16
16-bit Unicode Transformation Format.
ISO-8859-1
Commonly known as Latin-1 and a common default on syste
configured for a single western European language.
Windows-1252
Commonly known as WinLatin and a common default of lega
versions of Microsoft Windows based operating systems.

ignoreHeaderLines Specifies the number lines to ignore from the beginning of the file.
can be used to ignore files with column headings or preamble com
for example.

dateFormat When verbatim dates are consistent in format, this field can be us
indicate the format represented. It is recommended to use the dat
dateTime and time for field formats wherever possible, but where
verbatim dates are required, a format may be specified here. This
be considered a 'hint' for consumers. It is recommended that cons
support the minimum combinations of DD MM and YYYY with the
separators / and -. Examples:
DDMMYYYY
For dates of the form 21121978
DD-MM-YYYY
For dates of the form 21-12-1978
MMDDYYYY
For dates of the form 12211978
MM-DD-YYYY
For dates of the form 12-21-1978
YYYYMMDD
For dates of the form 19781221
Elements
Element

<files> <core> or <extension> element must contain one <files> element to locate
Element

<id> If extensions are being used, the <core> must contain an <id> element tha

<coreId> If extensions are being used, the <extension> element must contain a <cor
record identifier (the matching <id> in the core file).

<field> A <core> or <extension> element must contain one or more <field> eleme

2.3 The <files> element

The files element must contain one or more <location> elements, each
defining where a file resides. Each core or extension entity can be
composed from one or more files. If an entity has data in more than one
file, use the <location> element multiple times, once for each file that
makes up the entity.

Elements
Element

location Specifies the location of the file being described, which may take either o

A web accessible URL such as "[Link]

A filepath relative to the location of the metafile such as "specimen

2.4 The <field> element

The field element is used to specify the location and content of data
within a file. There must be one field element for every term being
shared for the entity, whether explicitly or through the use of a default
value for all rows in the file.
Attributes
Attribute Descripti

index Specifies the position of the column in the row. The first column has an ind
then the term and the default may be used to define a constant value for a

term A Unified Resource Identifier (URI) for the term represented by this field. F
Attribute Descripti

term="[Link] Terms outside of the

Dublin Core Metadata Initative, for example, dcterms:modified would be te

default Specifies value to use if one is not supplied for the field in a given row. If n
for all rows for a field that is not in the data file.

vocabulary A Unified Resource Identifier (URI) for a vocabulary that the source values
machine readable definition like SKOS, RDF or at least some simple text or
[Link] [Link]
[Link] .

3. Implementation Guide

3.1 Extension example

The following example illustrates the use of extensions. In this example

there are three files in the archive, all of which are located in the same
directory as the metafile. The [Link] file acts as a core file of Taxon
records. The [Link] file is extended by two other files, [Link] and
[Link]. The [Link] file contains records of a type specified in
an external definition at [Link] and
consists of Dublin Core and Darwin Core terms, while the [Link]
file contains records of a type specified at
[Link] and consists of Darwin
Core terms plus an additional term for threatStatus. Both extension files
are related to the core file by the taxonNameID fields. Presumably, this
archive contains information about whale species, type specimen
records for those species, and lists of countries and the threat status for
those species.
<?xml version="1.0" encoding="UTF-8"?>
<archive xmlns="[Link]
xmlns:xsi="[Link]
xmlns:xs="[Link]
xsi:schemaLocation="[Link]
[Link]

<core encoding="UTF-8" fieldsTerminatedBy="\t"

linesTerminatedBy="\n" ignoreHeaderLines="1"
rowType="[Link]
<files>
<location>[Link]</location>
</files>
<id index="0" />
<field index="0"
term="[Link] />
<field index="1" term="[Link]
/>
<field index="2"
term="[Link]
<field index="3"
term="[Link]
<field index="4"
term="[Link]
<field index="5"
term="[Link]
 </core>

<extension encoding="UTF-8" fieldsTerminatedBy=","

linesTerminatedBy="\n" fieldsEnclosedBy='"'
ignoreHeaderLines="1"
rowType="[Link]
<files>
<location>[Link]</location>
</files>
<coreId index="0" />
<field index="1"
term="[Link]
<field index="2"
term="[Link]
<field index="3"
term="[Link]
<field index="4"
term="[Link]
<field index="5"
term="[Link]
</extension>

<extension encoding="UTF-8" fieldsTerminatedBy=","

linesTerminatedBy="\n" fieldsEnclosedBy='"'
ignoreHeaderLines="1"
rowType="[Link]
<files>
<location>[Link]</location>
</files>
<coreId index="0" />
<field index="1"
term="[Link]
<field index="2"
term="[Link]
<field index="3"
term="[Link]
</extension>
</archive>

4. Database Example

4.1 MySQL

It is very easy to produce fielded text using the SELECT INTO

outfile command from MySQL. The encoding of the resulting file will
depend on the server variables and collations used, and might need to
be modified before the operation is done. Note that MySQL will export
NULL values as \N by default. Use the IFNULL() function as shown in the
following example to avoid this.
SELECT
IFNULL(id, ''), IFNULL(scientific_name, ''),
IFNULL(count,'')
INTO outfile '/tmp/[Link]'
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY '\n'
FROM
dwc;

5. Tools

For tools and applications, including a Java-based application to read

Darwin Core text archives, see the Darwin Core Tools and
Applications page [TOOLS].

Except where otherwise noted, content on this site is licensed under

a Creative Commons Attribution 4.0 International License.

Copyright 2011-2015 - Biodiversity Information Standards -

TDWG - Contact Us