R/recordTable.R
recordTable.RdGenerates a record table from camera trap images or videos. Images/videos must be sorted into station directories at least. The function can read species identification from a directory structure (Station/Species or Station/Camera/Species) or from image metadata tags.
recordTable(
inDir,
IDfrom,
cameraID,
camerasIndependent,
exclude,
minDeltaTime = 0,
deltaTimeComparedTo,
timeZone,
stationCol,
writecsv = FALSE,
outDir,
metadataHierarchyDelimitor = "|",
metadataSpeciesTag,
additionalMetadataTags,
removeDuplicateRecords = TRUE,
returnFileNamesMissingTags = FALSE,
eventSummaryColumn,
eventSummaryFunction,
video
)character. Directory containing station directories. It must either contain images in species subdirectories (e.g. inDir/StationA/SpeciesA) or images with species metadata tags (without species directories, e.g. inDir/StationA).
character. Read species ID from image metadata ("metadata") of from species directory names ("directory")?
character. Where should the function look for camera IDs:
'filename', 'directory'. 'filename' requires images renamed with
imageRename. 'directory' requires a camera subdirectory within
station directories (station/camera/species). Can be missing.
logical. If TRUE, species records are
considered to be independent between cameras at a station.
character. Vector of species names to be excluded from the record table
integer. Time difference between records of the same species at the same station to be considered independent (in minutes)
character. For two records to be considered
independent, must the second one be at least minDeltaTime minutes
after the last independent record of the same species
("lastIndependentRecord"), or minDeltaTime minutes after the
last record ("lastRecord")?
character. Must be a value returned by
OlsonNames
character. Name of the camera trap station column. Assuming "Station" if undefined.
logical. Should the record table be saved as a .csv?
character. Directory to save csv to. If NULL and
writecsv = TRUE, recordTable will be written to inDir.
character. The character delimiting hierarchy levels in image metadata tags in field "HierarchicalSubject". Either "|" or ":".
character. In custom image metadata, the species ID tag name.
character. Additional camera model-specific
metadata tags to be extracted. (If possible specify tag groups as returned
by exifTagNames)
logical. If there are several records of the same species at the same station (also same camera if cameraID is defined) at exactly the same time, show only one?
logical. If species are assigned with metadata and images are not tagged, return a few file names of these images as a message?
character. A column in the record table (e.g. from
a metadata tag) by to summarise non-independent records (those within
minDeltaTime of a given record) with a user-defined function
(eventSummaryFunction)
character. The function by which to summarise
eventSummaryColumn of non-independent records, e.g. "sum", "max"
(optional)
list. Contains information on how to handle video data (optional). See details.
A data frame containing species records and additional information about stations, date, time and (optionally) further metadata.
The function can handle a number of different ways of storing images, and supports species identification by moving images into species directories as well as metadata tagging. In every case, images need to be stored into station directories. If images are identified by moving them into species directories, a camera directory is optional: "Station/Species/XY.JPG" or "Station/Camera/Species/XY.JPG". Likewise, if images are identified using metadata tagging, a camera directory can be used optionally: "Station/XY.JPG" or "Station/Camera/XY.JPG".
If images are identified by metadata tagging, metadataSpeciesTag
specifies the metadata tag group name that contains species identification
tags. metadataHierarchyDelimitor is "|" for images tagged in DigiKam
and images tagged in Adobe Bridge / Lightroom with the default settings. It
is only necessary to change it if the default was changed in these programs.
minDeltaTime is a criterion for temporal independence of species
recorded at the same station. Setting it to 0 will make the function return
all records. Setting it to a higher value will remove records that were
taken less than minDeltaTime minutes after the last record
(deltaTimeComparedTo = "lastRecord") or the last independent record
(deltaTimeComparedTo = "lastIndependentRecord").
removeDuplicateRecords determines whether duplicate records
(identical station, species, date/time, (and camera if applicable)) are
all returned (FALSE) or collapsed into a single unique record (TRUE).
camerasIndependent defines if the cameras at a station are to be
considered independent. If TRUE, records of the same species taken by
different cameras are considered independent (e.g. if they face different
trails). Use FALSE if both cameras face each other and possibly
TRUE ).
exclude can be used to exclude "species" directories containing
irrelevant images (e.g. "team", "blank", "unidentified"). stationCol
can be set to match the station column name in the camera trap station table
(see camtraps).
Many digital images contain Exif metadata tags such as "AmbientTemperature"
or "MoonPhase" that can be extracted if specified in metadataTags.
Because these are manufacturer-specific and not standardized, function
exifTagNames provides a vector of all available tag names.
Multiple names can be specified as a character vector as: c(Tag1,
Tag2, ...). The metadata tags thus extracted may be used as covariates in
modelling species distributions.
eventSummaryColumn and eventSummaryFunction can be used to
extract summary statistics for independent sampling events. For example, you
assigned a "count" tag to your images, indicating the number of individuals
in a picture. In a sequence of pictures taken within 1 minute, most pictures
show one individual, but one image shows two individuals. You tagged the
images accordingly (count = 1 or count = 2) and run recordTable. Set
eventSummaryColumn = "count" and eventSummaryFunction = "max"
to obtain the maximum number of count in all images within
minDeltaTime minutes of a given record. The results is in a new
column, in this example count_max. You can also calculate several
statistics at the same time, by supplying vectors of values, e.g.
eventSummaryColumn = c("count", "count", "camera") and
eventSummaryFunction = c("min", "max", "unique") to get minimum and
maximum count and all unique camera IDs for that event. Note that
eventSummaryColumn and eventSummaryFunction must be of same
length.
Argument video is a named list with 2 or 4 items. 2 items
(file_formats, dateTimeTag) are always required, and are
sufficent if IDfrom = "directory". In that case, no digiKam tags will
be returned. To return digiKam tags, two additional items are required
(db_directory, db_filename). This is essential when using
IDfrom = "metadata". When using IDfrom = "directory", it is
optional, but allows to extract metadata tags assigned to videos in digiKam.
This workaround is necessary because digiKam tags are not written into video
metadata, but are only saved in the digiKam database. So in contrast to JPG
images, they can not be extracted with ExifTool. It also requires that
inDir is in your digiKam database.
The items of argument video are:
file_formats | The video formats to extract (include "jpg" if you want .JPG image metadata) |
dateTimeTag | the
metadata tag to extract date/time from (use exifTagNames to
find out which tag is suitable) |
db_directory | The directory
containing digiKam database (optional if IDfrom = "directory") |
db_filename | The digiKam database file in db_directory
(optional if IDfrom = "directory") |
See the examples below for for how to specify the argument video.
The results of a number of other function will depend on the output of
this function (namely on the arguments exclude for excluding species
and minDeltaTime/ deltaTimeComparedTo for temporal
independence):
detectionMaps |
detectionHistory |
activityHistogram |
activityDensity |
activityRadial |
activityOverlap |
activityHistogram |
surveyReport |
Custom image metadata must be organised hierarchically (tag group - tag; e.g. "Species" - "Leopard Cat"). Detailed information on how to set up and use metadata tags can be found in vignette 2: Species and Individual Identification.
Custom image metadata tags must be written to the images. The function cannot read tags from .xmp sidecar files. Make sure you set the preferences accordingly. In DigiKam, go to Settings/Configure digiKam/Metadata. There, make sure "Write to sidecar files" is unchecked.
Please note the section about defining argument timeZone in the
vignette on data extraction (accessible via
vignette("DataExtraction") or online
(https://cran.r-project.org/package=camtrapR/vignettes/camtrapr3.pdf)).
Phil Harvey's ExifTool https://exiftool.org/
if (FALSE) # the examples take too long to pass CRAN tests
# set directory with camera trap images in station directories
wd_images_ID_species <- system.file("pictures/sample_images_species_dir",
package = "camtrapR")
if (Sys.which("exiftool") != ""){ # only run these examples if ExifTool is available
rec_table1 <- recordTable(inDir = wd_images_ID_species,
IDfrom = "directory",
minDeltaTime = 60,
deltaTimeComparedTo = "lastRecord",
writecsv = FALSE,
additionalMetadataTags = c("EXIF:Model", "EXIF:Make")
)
# note argument additionalMetadataTags: it contains tag names as returned by function exifTagNames
rec_table2 <- recordTable(inDir = wd_images_ID_species,
IDfrom = "directory",
minDeltaTime = 60,
deltaTimeComparedTo = "lastRecord",
exclude = "UNID",
writecsv = FALSE,
timeZone = "Asia/Kuala_Lumpur",
additionalMetadataTags = c("EXIF:Model", "EXIF:Make", "NonExistingTag"),
eventSummaryColumn = "EXIF:Make",
eventSummaryFunction = "unique"
)
# note the warning that the last tag in "additionalMetadataTags" ("NonExistingTag") was not found
any(rec_table1$Species == "UNID") # TRUE
any(rec_table2$Species == "UNID") # FALSE
# here's how the removeDuplicateRecords argument works
rec_table3a <- recordTable(inDir = wd_images_ID_species,
IDfrom = "directory",
minDeltaTime = 0,
exclude = "UNID",
timeZone = "Asia/Kuala_Lumpur",
removeDuplicateRecords = FALSE
)
rec_table3b <- recordTable(inDir = wd_images_ID_species,
IDfrom = "directory",
minDeltaTime = 0,
exclude = "UNID",
timeZone = "Asia/Kuala_Lumpur",
removeDuplicateRecords = TRUE
)
anyDuplicated(rec_table3a[, c("Station", "Species", "DateTimeOriginal")]) # got duplicates
anyDuplicated(rec_table3b[, c("Station", "Species", "DateTimeOriginal")]) # no duplicates
# after removing duplicates, both are identical:
whichAreDuplicated <- which(duplicated(rec_table3a[,c("Station", "Species", "DateTimeOriginal")]))
all(rec_table3a[-whichAreDuplicated,] == rec_table3b)
### extracting species IDs from metadata
wd_images_ID_species_tagged <- system.file("pictures/sample_images_species_tag",
package = "camtrapR")
rec_table4 <- recordTable(inDir = wd_images_ID_species_tagged,
IDfrom = "metadata",
metadataSpeciesTag = "Species",
exclude = "unidentified")
### Including videos
# sample videos are not included in package
# with videos, IDfrom = "directory", not extracting digiKam metadata
rec_table4 <- recordTable(inDir = wd_images_ID_species,
IDfrom = "directory",
video = list(file_formats = c("jpg", "mp4"),
dateTimeTag = "QuickTime:CreateDate")
)
# with videos, IDfrom = "metadata", extracting digiKam metadata
rec_table5 <- recordTable(inDir = wd_images_ID_species,
IDfrom = "metadata",
metadataSpeciesTag = "Species",
video = list(file_formats = c("jpg", "mp4", "avi", "mov"),
dateTimeTag = "QuickTime:CreateDate",
db_directory = "C:/Users/YourName/Pictures",
db_filename = "digikam4.db")
)
} else {
# show function output if ExifTool is not available
message("ExifTool is not available. Cannot test function. Loading recordTableSample instead")
data(recordTableSample)
}
#> timeZone is not specified. Assuming UTC
#> Error in eval(expr, envir, enclos): object 'wd_images_ID_species' not found