The two main data types used in pems.utils are pems.elements, typically data-series with assigned units, and pems, typically datasets of simultaneously logged pems.elements.
This document provides an overview of methods for making these and getting data in and out of the R package pems.utils.
For a quick and more general introduction to pems.utils, see [>pems.utils introduction]
If you have any suggestions how to make either pems.utils or this document better or you have any problems using either, please let me know [>email me].
Unless you have setup R to automatically load pems.utils, you will need to load it at the start of each R session e.g. using
library(pems.utils).
You can make pems.elements from most common types of vectors (numerics, characters, logicals, characters, etc) using pems.element(), for example:
my.speed.element <- pems.element(1:10, units="m/s")
my.speed.element## pems.element [n=10]
## [1] 1 2 3 4 5 6 7 8 9 10
## ... <integer> [m/s]If, as above, declared, units are assigned to the pem.elements.
See, for example, ?convertUnits in R documentation for more on pems.utils unit handling.
You can make pems from a data.frame in a similar fashion, for example:
my.pems <- pems(data.frame(speed=1:10, emissions=11:20),
units=c("m/s", "g/s"), source="i.made.this")
my.pems## pems (10x2)
## speed emissions
## [m/s] [g/s]
## 1 1 11
## 2 2 12
## 3 3 13
## 4 4 14
## 5 5 15
## 6 6 16
## ... not showing: 4 rowsWith pems, units are assigned to the columns (or pems.elements) of the pems dataset on a column-by-column basis, while other arguments not formally declared, e.g. source in the above example, are treated as meta-data and associated with the data generally.
With pems, pems[…] accesses data subsets and pems[[…]] accesses meta information, for example:
my.pems[1,1:2]## pems (1x2)
## speed emissions
## [m/s] [g/s]
## 1 1 11my.pems[["source"]]## [1] "i.made.this"This structure allows you to include any sampling session information that you would like to keep with the dataset, for example vehicle, fuel, route or emission measurement system identifiers.
See [>pems.utils operators] and [>pems.utils generics] or R help documentation for more about pems data handling.
Although these functions allow you to easily make pems.elements and pems, you are more likely to want to import PEMS data as a pems directly from a file. A family of import2PEMS() functions are included in pems.utils for such work.
These are typically used in the form:
my.pems <- import2PEMS()If, as above, the path-to-file is not supplied as the first function argument, these use file.choose() to open a browser that can be used to manually select the file to be imported.
By default, import2PEMS() assumes files are tab delimited text and have column headers in first row and data from the second row onwards.
Extra arguments can be used to modify the import operation, for example:
names: dataset column names, either a character vector of the names to use or the number of the file row where the names are recorded. By default this is set to 1 and assumes first row.
units: dataset units, either the units to use supplied as character vector, the file row where the units are recorded or a special operation. No default is applied, so units are not extracted unless declared in the import command. The special operation "get.from.names" is for cases where names are logged names(units).
data.from: The first row of data in the supplied file, by default assumed to be the first row if names are supplied separately or the row after names if a row number is supplied for names.
The argument file.reader can also be used to change the function used to read the supplied file. The default is read.delim, so files are assumed to be tab delimited unless another read function is supplied.
import2PEMS() passes all arguments to the file.reader function, so it should also handle all arguments this function accepts.
import2PEMS() functions:
| pems.utils Command | Imports | Notes |
|---|---|---|
| Common File Types | ||
| import2PEMS | - | Assumes file tab delimited; needs units assigned |
| importTAB2PEMS | tab delimited files | Needs units assigned |
| importCSV2PEMS | comma delimited files | Needs units assigned |
| PEMS Output Files | ||
| importOBS2PEMS | Horiba OBS1300 files | |
| importSEMTECH2PEMS | Sensors SEMTECH files | |
| importParSYNC2PEMS | 3DATX parSYNC files |
See ?import2PEMS in R documentation for further details.
If there is not a dedicated import2PEMS() function for your file type, you can often import PEMS data from files with more commonplace file structures using a modified import call, for example:
The SIM_FILE_001.AAA file is in the refs subdirectory on the project website [>here].
If you download it and open it using a text editor or read it into R using readLines(), you will see that it is the pems.1 dataset saved as in a block layout, with a header (the file name) in the first row, and then names, units and data in rows 3 and 5 and from 7 onwards, respectively:
[01] SIM_FILE_001.AAA
[02] <names>
[03] time.stamp,local.time,conc.co,conc.co2,conc.hc,conc.nox,afr,exh.flow.r ...
[04] <units>
[05] Y-M-D H:M:S GMT,s,vol%,vol%,ppmC6,ppm,,L/min,degC,kPa,degC,kPa,%,km/h, ...
[06] <data>
[07] 2005-09-08 11:46:07,0,0,0,0,20.447,199.85,-8.0626,51.315,99.993,20.905 ...
[08] 2005-09-08 11:46:08,1,0,0,0,21.973,199.89,-13.419,50.812,100.04,21.075 ...
[09] 2005-09-08 11:46:09,2,0,0,0,20.752,199.91,-7.4939,48.538,100.06,21.146 ...
[10] 2005-09-08 11:46:10,3,0,0,0,22.583,199.88,-8.0824,50.058,99.988,21.201 ...
[[not showing lines 11 onwards]]Things to note when importing this file:
.csv reader to import this data. So, either:
import2PEMS(..., file.reader = read.csv) orimportCSV2PEMS(...)names, data.from and units. So, for this file:
import2PEMS(..., names=3, units=5, data.from=7)import2PEMS(..., time.format="%Y-%m-%d %H:%M:%OS", tz="GMT")So, to import from source:
path.to.file <- "http://pems.r-forge.r-project.org/refs/SIM_FILE_001.AAA"
pems.2 <- import2PEMS(path.to.file, file.reader=read.csv,
names=3, units=5, data.from=7,
time.format="%Y-%m-%d %H:%M:%OS", tz="GMT")
pems.2## pems (1000x25)
## time.stamp local.time conc.co conc.co2 conc.hc conc.nox
## [Y-M-D H:M:S GMT] [s] [vol%] [vol%] [ppmC6] [ppm]
## 1 2005-09-08 11:46:07 0 0 0 0 20.447
## 2 2005-09-08 11:46:08 1 0 0 0 21.973
## 3 2005-09-08 11:46:09 2 0 0 0 20.752
## 4 2005-09-08 11:46:10 3 0 0 0 22.583
## 5 2005-09-08 11:46:11 4 0 0 0 20.142
## 6 2005-09-08 11:46:12 5 0 0 0 20.142
## ... not showing: 994 rows; 19 cols (elements)
## ... other cols: afr; exh.flow.rate[L/min]; exh.temp[degC]; exh.press[kPa];
## amb.temp[degC]; amb.press[kPa]; amb.humidity[%]; velocity[km/h];
## revolution[rpm]; option.1[V]; option2[V]; option.3[V];
## latitude[d.degLat]; longitude[d.degLon]; altitude[m];
## gps.velocity[km/h]; satellite; n.s; w.eMore complex file structures may require more work, but assuming the data can be read into R, it should always be possible to build your own import function by reading in the different data parts, tidying them and passing these to pems().
If you have any problems importing data or you would like to discuss dedicted import functions for other file types, please let me know [>email me].
You can save a pems dataset for later use in R using any standard R method, for example either individually or as part of worksheet using save(). But, saveRDS() is maybe worth considering if you are likely to work with multiple individual pems datasets:
#to save pems.1 in working directory
saveRDS(pems.1, "pems.1.rds")
#then to later re-load it into R but under different name
my.pems <- readRDS("pems.1.rds")This combination is particularly useful if you want to batch process multiple previously saved pems datasets using common code.
You may also want to export pems data from R to other software packages. For this, three options are currently provided, exportPEMS(), exportPEMS2CSV() and exportPEMS2TAB.
These are typically used in the form function(pems, "file.name") and work like write() functions in R, for example:
exportPEMS(pems.1, "pems.1.data")
exportPEMS(pems.1, "pems.1.data.csv", sep=",")All three export the pems data.frame.
exportPEMS() makes a relatively crude ‘as.is’ export and is a useful building-block if you want to develop different export functions.
exportPEMS2CSV() and exportPEMS2TAB() make comma-delimited and tab-delimited files, respectively, but also modify the files to make them easier to work with outside R. So, these are these probably the best start-point for those wishing to quickly export pems.utils data from R.
pems units can be added to the header row of the exported file in the form name(units) using the extra argument units = "add.to.names".
The recommended workflow for a user intending to export pems.utils data from R, process it elsewhere and then return it to R would be something like:
exportPEMS2CSV(pems.1, "pems.1.data.csv", units="add.to.names")
# read export into other software
# process it there
# then export it (e.g. again as .csv) and import the modified data into R
pems.1b <- importCSV2PEMS("pems.1.data.csv", units="get.from.names")If you have any suggestions how to make either pems.utils or this document better or you have any problems using either, please let me know. [>email me].