375Prerequisites for ePROM Development and Their Assessments
in Routine Care
Before broad introduction of eP ROMs into clinical routine, careful comparison of
data obtained by paper–pencil and computerized versions of the assessments was
and is crucial, because equivalence of data obtained by the two acquisition meth-
ods cannot be taken for granted [ 18 , 19 , 36 , 50 , 51 ]. Detectable (test–retest) cor-
relations between the modes of administration need to meet methodological
requirements for demonstrating reliability and validity [ 50 , 51 ]. Assessments on
electronic devices need to be able to detect changes over time [ 17 ]. Coons et al.
published a general framework for the transfer of paper-based PROs to electronic
devices [ 52 ].
Meanwhile many traditional scales have been evaluated [ 5 ]. Gwaltney et al.
showed in their meta-analysis that computer and paper measures produce equivalent
scores. Subjects’ computer experience and age did not infl uence this result when
“small mean differences” were not regarded as clinically relevant [ 50 ]. Similarly,
Campbell et al. published a review that summarizes 55 studies investigating 79
instruments. It provides a good overview for rheumatology and other disciplines [ 47 ].
The authors stated that paper-based and electronic formats are usually rated to be
equivalent, and that study participants prefer electronic assessments [ 47 ]. However,
they recommend further validations of electronic versions, taking into account that
data assessed electronically should produce fi gures that are equivalent or superior to
those retrieved via paper-based versions [ 47 , 52 ].
ePROMs Application Systems
Electronic PROMs are available not only in EHRs and rheumatology-specifi c
patient documentation systems, but also in (online) registries and other partly “ reg-
istered user-restricted ” online applications developed in a number of countries
worldwide (e.g., http:// http://www.medal.org and http://www.nihpromis.org/?AspxAuto
DetectCookieSupport=1#2 ; [ 12 , 32 , 33 , 44 ]). In addition, (non)profi t organizations
have implemented systems that allow ePROMs assessments (e.g., http://c-path.org/
programs/epro/ and https://www.parexel.com/solutions/informatics/clinical-
outcome- assessments/epro/ ). Studies showed that use of health information tech-
nology may be associated with better outcomes (e.g., reductions in mortality,
complications, and costs) [ 53 ]. Developed systems have become more affordable
and feasible to implement [ 5 ]. Patients’ and others stakeholders’ integration into the
development and implementation process is necessary and will lead to better accep-
tance of the systems [ 54 ].
The heterogeneity of systems allowing to document ePROMs becomes obvious
from a cancer research review published in 2013. This paper identifi ed at least
33 unique systems [ 41 ]. However, each of the systems differed in features and
15 Electronic Patient-Reported Outcome Measures (ePROMs) in Rheumatology