Posted by Jane Carter on Fri, Dec 16, 2011 @ 11:30 AM
While providing site training at an Investigator Meeting recently, I came across some interesting comments from sites that were not familiar with using eDiaries or tablets to capture PRO or ClinRO data. One study coordinator was concerned about using eDiaries with certain populations, and was wondering if we were going to provide paper checklists or backup questionnaires on paper for subjects who were not comfortable using eDiaries. This comment brought me back to the realization that while I have been working with ePRO for over six years, there are some people who still are not familiar with the technology or the risks inherent in mixing modalities of PRO collection. I thought I would speak to some of these items, in hopes of providing a basic understanding of ePRO and some items to consider.
When choosing to collect Patient Reported Outcomes for a clinical trial, particularly for a primary or secondary endpoint, it is important to ensure the quality of the data being collected. Most of us have heard of or experienced firsthand the “parking lot syndrome”, where patients fill in dosing diary cards or other PRO questionnaires in the parking lot immediately before going in for an office visit. This is one of the main reasons sponsors choose ePRO over paper – ePRO technology allows for entry “windows” to ensure the data is being captured in a timely manner, and is not subject to the failings of memory recall. I’m lucky I can remember what I had for breakfast yesterday, let alone how I felt or at what time I took medicine!
While some people are still concerned about technology and dealing with certain patient populations – in particular, the elderly – our statistics have shown that eDiary compliance by elderly subjects is extremely high (>95%). Technology can be scary to people, but I have found that there are six key factors that improve the likelihood of success:
Success Factor 1 – Simple eDiary Design. I follow the KISS model when designing ePRO collection tools – Keep It Short and Simple. If the eDiary is too complicated, with strict entry windows and hard stops built into the eDiary, then workarounds need to be built and processes changed if a subject deviates from what is expected. While I design to help the subject follow the protocol, I try not to make the design so rigid that the programming increases frustration and reduces compliance.
Success Factor 2 - Logical Edit Checks and Information Messages. Inclusion of logical edit checks and information messages improves the quality of the data collected and provides guidance for the subject. For example, if a subject is filling in a multi-question PRO and one response prompts for additional questions, then I build in the branching logic to ask the appropriate questions on a “Yes” answer, and to skip those questions on a “No” answer. Additionally, I build in popup messages that remind the subject if he/she answers a question that is outside an expected range: for example, if a subject enters an amount of medication that is outside the expected range for the protocol, then the eDiary would remind the subject to call the doctor.
Success Factor 3 - Reminder Alarms. Reminder alarms can improve compliance and can provide “peace of mind” for subjects. If a subject is filling in multiple questionnaires daily and a separate weekly questionnaire at a different time of day, I would build in an alarm to remind the subject to fill in the daily questionnaires, and I would build in a separate alarm that would sound to remind the subject to fill in the weekly questionnaire.
Success Factor 4 – Training Material. The fourth success factor encompasses a wide area of items under the umbrella of Training. Providing detailed, hands-on training to the monitors and site personnel using a “Train the Trainer” approach helps facilitate the training subjects will receive at site. Additional support in the form of a detailed Site Manual helps reinforce the hands-on training and provides extensive troubleshooting and FAQs to ease site personnel with both their use and subjects’ use of the eDiary. Having a Training Module built into the eDiary allows the subject to practice using the device and to become familiar with the device before leaving the office. Providing easy to read, step-by-step guides for the subject to read and follow when using the eDiary helps ensure a comfortable transition from on-site support with the study coordinator to self-use at home. While these guides are most used by older subjects who are less tech-savvy, they are very helpful for all subjects. The guides should be easy to follow, include pictures of both the device and the screens to walk the subjects through everything they need to know about the device, and have a list of steps that they need to follow at any given time point. Finally, access to helpdesk support for both sites and subjects is a key “safety net” to reinforce training and to get sites and subjects “back on track”.
Success Factor 5 – One Method of PRO Collection. Sites often want “back up” methods of collecting PRO, and want to have paper questionnaires on hand in case a subject is not comfortable using ePRO. I am now seeing protocols have an inclusion criterion that states that the subject must be willing to use an eDiary. I recommend this be included in all protocols. If a study provides alternatives like paper, then we not only deal with two points of collection and the multiple processes that need to be built to ensure consistency, but we also send a mixed message to the subjects and undermine their confidence in ePRO
Success Factor 6 – Site Personnel. Whenever I am at an investigator meeting and am training sites on how to use the ePRO devices, I always encourage site personnel to be “cheerleaders” or “champions” of ePRO. I remind them that If they act frustrated with a device or appear to treat the ePRO collection tool as just one more piece of equipment that they need to deal with, this attitude is transferred to the subjects. I have seen a higher incidence of helpdesk calls, a lower rate of compliance, and a larger number of issues at sites where the site personnel appear unwilling or unmotivated during that initial training.
Conversely, I have spoken with study coordinators who are high enrollers with high compliance rates, and I have discovered that universally those study coordinators make it a point of knowing everything they can about the devices, are engaged and ask lots of questions during the training, read all of the study materials including the guides given to subjects, and motivate their subjects by saying things like, “Your data is so important to this study, that the sponsor spent all this money getting these cool devices to collect your data. You (the subject) are key to the success of this study!”
In conclusion, while we can provide all of the technology, design and support for the study, we must never underestimate the human factor, and how we present to and train our subjects on the use of ePRO. It is a team effort! The sponsor and ePRO provider can design and deliver quality ePRO to the sites, but we need the help of the doctors, nurses, monitors and helpdesk to provide seamless support and to be champions of ePRO in order to ensure subjects are successful.
FUTURE TOPICS:
ePRO Basics – Partnering with an ePRO Provider – What to Consider
ePRO Basics – Design Considerations for ePRO
ePRO Basics – Optimal Training Methods When Using ePRO
ePRO Basics – Monitoring ePRO Data: Does ePRO Data Get “Cleaned”?
Best Regards,
Jane Carter
Program Manager, CRF Health
Posted by John Hutchin on Fri, Dec 02, 2011 @ 10:11 AM
Part II
Cloud Computing
This was a very interesting session. By now we’ve all heard the term “cloud” computing – even my wife, who is a dedicated anti-computer person and just recently got a mobile phone, asked me about it last week. Still, I find myself thinking "There’s no way I’m putting my personal information at risk by trusting it to something so nebulous that it’s actually named the 'cloud'". It turns out that eClinical’s main interest is really in private clouds these days – not public ones. Nick Neri of PharmaPros and Kevin Ahonen of Biogen-Idec did a great job 
presenting a case study of cloud technology put to good use. They very clearly described the concept of cloud computing and how it differs from simply hosted solutions. It was discussed that there is some information that is not really meant for the cloud – at least not yet. In the case study that was presented, only operational data is stored in and accessed from the cloud. What is operational data? It’s the data about the data. Metadata if you will; status values and metrics and information pertaining to how clinical data is being used. Having an external system out there that can securely access data from many distinct, “silo” locations within your company’s clinical infrastructure – in a controlled fashion and without adding burden to your existing IT system – is an idea that can meet a lot of needs. “Terminology standardization” was brought up; the need to reconcile different naming conventions across different systems. This is vital to cloud computing so that information is correctly aggregated and people know what it is they’re being presented with. As with a lot of things these days, the technology implementation is often simpler than getting people to accept new processes and open their minds to accept new ideas. This is especially true for those people who have had much success living in their very tall silo. “Don’t fix it if it’s not broken” is a valid argument unless what’s currently working needs to work within a much larger scope and cannot. Nick and Kevin made a very good argument for stepping back in order to be able to see the forest in addition to the trees. There was a lot of interest in this and a lot of interesting side conversations throughout the week.
Workshops
The week wasn’t just a series of presentations, with the attendees sitting passively and listening. The eCF meetings also have a number of workshops focused on issues relevant to eClinical and designed to formulate a group consensus that can foster change in the industry or recommend best practices.
Each of these workshops broke the attendees into groups of 8 or so people per table. Tables were assigned a particular task and reported back to the entire group at the end of the breakout session. The details of these workshops are beyond the scope of this blog, but I can say that they were well run, interesting and designed such that each group had a cross-section of people with a variety of backgrounds and expertise.
There were separate workshops pertaining to:
- Site qualification of sites that intend to use electronic health records (EHR) as source data for clinical trials. Regulations require that such an EHR system meet the same requirements as others uses for eSource in a clinical trial.
- A risk-based approach to SDV
- Leveraging Electronic Health Records for clinical research
- Serious Adverse Event integration
Attendees were assigned to groups such that each table contained a wide range of experience: CRO, ePRO, Data Management, Safety, etc. This helped to ensure a thoughtful exchange of ideas and kept the discussion lively.
If you’re not currently a member of the eClinical Forum, I can highly recommend it. If you are, I hope to see you at the next meeting!
Best regards,
John Hutchin
Senior Director, Technical Support
CRF Health
Posted by John Hutchin on Thu, Dec 01, 2011 @ 09:29 AM
Can it really be 7 years since I attended an eClinical Forum (eCF) meeting? Back in 2004, the meeting I attended was held in Cincinnati and hosted by Proctor & Gamble. At that time, I was filling in for our regular representative who had conflicting obligations. I thought then 
what a good organization this was. Beginning this year I’m fortunate enough to represent CRF Health at the bi-annual eCF meetings in the US. I love this organization. It brings together a diverse group of pharma, biotech and technology provider representatives to exchange ideas and help shape the future of the eClinical landscape. Ideas appeared, conversation flowed and it seemed I could literally feel progress being made. Everyone there was genuinely interested in advancing the field of eClincial and coming up with best practices and recommendations. What I really like is the feeling of comaraderie and the lack of commercialism. I sat with competitors and clients alike and discussed issues that can improve our ability to bring better healthcare products to market faster. What a refreshing experience.
Day 1 began with presentations and discussion related to integration. Case studies were presented by Allergan, PHT and CRF Health. The overall feeling I got from these case studies is that the time is ripe from a technology perspective for real-time, transactional integrations based on web-services APIs. I commented (to some laughter) that CRF Health has had web-service APIs for a number of years now, but it was like having one “walkie-talkie”… there was no one to talk to! I believe that the maturity of this technology, together with CDISC and HL7 standards, is finally resulting in adoption rates that will rapidly transform the eClinical landscape in the next 2-3 years. I don’t say that lightly. The clinical research industry evolves and changes direction about as fast as you can turn the Titanic with a canoe paddle. In fact, when I attended my last eCF meeting in 2004, there were representatives from Siemens and GE Health forecasting the integration of Electronic Health Record systems (EHR) and clinical systems within 5 years. At last week’s eCF meeting in Newport Beach, California, the same topic was discussed in detail – and I hadn’t really detected any meaningful progress. The same major problems seem to be blocking our path, privacy concerns, lack of standard nomenclature and an effective business model that will sustain such a system beyond an ideological pilot. Still, we need to start somewhere, and real-time integration is now possible with very little capital investment. Technology is such that we no longer need to worry about the “plumbing”. We can focus on determining the best information to send through the pipes and how to best present it to the end user. One step at a time and soon we’ll be off and running. It’s apparent from these case studies that learning to walk in this environment is easier than first anticipated.
Open Source
We then had a presentation and discussion of open source for eClinical software. I must admit that I remain a skeptic. To me, open source software seems best suited for foundation technologies that underlie specific needs, rather than meet those specific needs themselves. Open source operating systems seem more feasible to me than open source eClinical software. Maybe I’m just too old; working on something only to give it away for free goes against my idea of a good business model. In this regard, turning the Titanic with a canoe paddle seems to be further complicated by the need to convince the captain to even dip the paddle into the water. I’d love to hear other’s thoughts on this.
In the second part of this blog post, I will discuss Cloud Computing and the eClinical workshops that were held at the eCF Meeting.
Best regards,
John Hutchin
Senior Director, Technical Support
CRF Health
Posted by Kai Langel on Mon, Nov 28, 2011 @ 01:00 PM
Diane Wild is a colleague that I have the privilege of working with closely. She is at the leading edge of what’s new with ePRO in terms of best practices and current industry guidelines regarding instrument migration, usability testing and linguistic validation. She is one of the founders of Oxford Outcomes, a leading international health outcomes consultancy that was recently acquired by ICON. Not only is Diane a well respected professional, but she is also a wonderful person much liked by myself as well as many of her colleagues at Oxford Outcomes.
I had an opportunity to pick her brain and conduct an interview at the ISPOR Annual European Congress in Madrid, Spain.
Me: The FDA PRO guidance came out in 2009. How has it been received and adopted during the last two years?
Diane: There was a lot of uncertainty once the draft guidance came out 
and nobody knew quite how the final guidance would turn out. It was finally released in 2009 and since then, many companies have conducted webinars to help the industry understand the implications of the final guidance. Now after two years, there is a general feeling of acceptance of the guidance and deeper understanding of it amongst vendors as well as Sponsors. For example, in the beginning, nobody quite knew what the PRO evidence dossier should look like, but now many people know how to put them together. However, there are still some specific questions. For example, there is some ambiguity regarding the use of mixed modes of delivery, such as paper and electronic PROs in clinical studies as well as how to show evidence of content validity when doing linguistic validation.
Me: What about the EMA? How has the FDA guidance affected the way things are done in Europe?
Diane: There is a general acceptance that the FDA documentation will be acceptable for EMA submissions as well. We also have more experience now in submitting to the EMA and there is a greater emphasis looking for evidence of psychometric properties of measures rather than focusing so much on content validity. The EMA also has a more pragmatic approach, especially when considering the use of widely used measures.
Me: Doesn’t the EMA differentiate between ‘simple’ measures, such as a symptom scale and more complex QoL instruments?
Diane: Yes that is right. The EMA currently differentiate between PROs that are well established and measure symptoms such as pain or itch in psoriasis, and multidimensional measures of health-related quality of life. Different levels of evidence are required by the EMA as a result of this distinction.
Me: Having worked in this field for such a long time, what do you see as the top trends or hot topics in the PRO / ePRO space right now?
Diane: At least from my perspective, one of the most important areas regarding PRO and the FDA is the expansion of the PRO final guidance to cover other endpoints under SEALD’s consideration – observed outcomes (ObsRO) and clinician reported outcomes (ClinROs). This is going to have an impact on how we work in terms of needing to demonstrate content validity and measurement properties for these clinical outcome assessments moving beyond the current PRO focus.
From the ePRO perspective, it has now been 2 years since the ISPOR ePRO Task Force Report by Coons et. al. regarding paper to ePRO migration. Technology has moved fast in this area and this is now forcing us to think about how things have changed since the publishing of this paper. For example, we might now need to consider migrating from one ePRO platform to another and consider the differences between different ePRO modalities and not just paper vs. ePRO. There is now the ePRO Consortium organized by the C-Path institute and there are six ePRO vendors who are members and are now working together collaboratively to create standards and to provide updates and guidance in this area.
Me: So there is much more cross-vendor collaboration than in the past? Do we have a chance to come up with meaningful standards?
Diane: Absolutely. I think you can compare ePRO migration to the world of translations and linguistic validation. We are not constantly re-inventing the wheel as translations are re-usable between Sponsors and vendors. This is where we want to get to with ePRO migration as well, so that we are not constantly re-evaluating the same instruments and that they are effectively shared.
Me: My next question is more on the personal level. You founded Oxford Outcomes 14 years ago and it has now been acquired by ICON. Your colleagues have said very nice things about working with you and you are clearly still inspired and motivated to work in this space. What is your secret?
Diane: Well, we have always been about organic growth and over the period of 14 years, we have grown from just two of us to about 80 people at the time we were acquired. The key really has been about recruiting and working with the right people as well as being passionate about the work.
Me: I think it’s a job really well done. There is a very positive atmosphere around Oxford Outcomes and I have certainly enjoyed working with all of you. Thank you very much for the interview.
Best Regards,
Kai Langel
Senior Director, Technical Support, CRF Health
Posted by Chris Clancy on Tue, Nov 15, 2011 @ 09:50 AM
Clinovo hosted a truly informative panel discussion on the topic of adaptive clinical trials in Palo Alto,CA on November 2. It was a great forum to hear some very experienced clinical trial professionals share their experiences with a standing-room-only audience. In addition, it was a friendly environment to network with Bay Area colleagues before and after the panel discussion. The event was impressive enough to make me want to embark on my first visit to the blogosphere. Here goes…
During the panel session, the first thing that struck me was when the panelists explained that while the phrase “adaptive clinical trial” is a somewhat new bit of bio-pharma jargon, we all have been running trials for decades that fit this moniker – we just didn’t know it. It is also clear that these trials are increasing in number, and are helping to save millions for the biopharmaceutical industry.
From the FDA’s Draft Guidance: Adaptive Design Clinical Trials for Drugs and Biologics – February 2010
(Page 6) Definition and Concept of an Adaptive Design Clinical Trial
For the purposes of this guidance, an adaptive design clinical study is defined as a study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypotheses based on analysis of data (usually interim data) from subjects in the study.
Analyses of the accumulating study data are performed at prospectively planned timepoints within the study, can be performed in a fully blinded manner or in an unblinded manner, and can occur with or without formal statistical hypothesis testing.
As a member of the eClinical community myself, I was also pleased to hear a common theme throughout the discussion, which was how great an impact technology was making on the ability to achieve the goals of adaptive trials. All of the panelists explained how EDC made it possible to have access to their data in a way that facilitated the type of decisions critical to adaptive studies. It was clear that ‘in the old days with paper’, it was very difficult, or impossible to make the type of mid-stream analysis and decisions which we now take for granted.
I was pleased to hear Oranee Daniels from Theravance mention that she could look at her ePRO data every morning if needed, and always know that she was looking at the most up-to-date information for making the crucial decisions associated with the adaptive design.
Don Kellerman from Map Pharmaceuticals also explained his experience with killing studies by determining futility through adaptive trial design. Don had a great sense of humor; he warned about giving him a compound to evaluate, since he has a proven track record for quickly finding futility.
All in all, I felt the event was a great success. I would certainly attend future sessions of the Silicon Valley BioTalk series. Kudos to Clinovo for putting it together and generating the interest for such a great turn out on a Wednesday evening. It attracted participants from all over the Bay Area, and beyond.
Best,
Chris Clancy
Director, Business Development, CRF Health
Posted by Kai Langel on Fri, Aug 19, 2011 @ 01:00 PM
Subject compliance is a hot topic in the industry right now. Some of the discussions out there can be a little misleading since ‘compliance’ means different things to different people / companies. Many eDiary vendors focus on a subject’s compliance with data entry, whereas Sponsors are more interested in the subject’s compliance with the study procedures, and clinicians are interested in the subject’s compliance with their medication / treatment.
I think it’s important that we define what we mean when we use the term because it can mean any of the above items. It is also important that we carefully define the reports and metrics collected in clinical studies. I see many studies attempt to come up with a single metric for ‘total compliance’, which includes a number of different things, like compliance with different medications, diary entry, etc. By doing this, they are actually hiding some important details. Individual metrics for individual items would most likely be much more useful. If a subject is having issues with their medication compliance, it should be addressed separately from their compliance with the entering of information into their eDiary.
CRF Health has been collecting eDiary compliance metrics for many years now and our compliance database is getting quite extensive and more useful the more data we collect. Every single CRF Health eDiary device automatically collects project-specific eDiary compliance for every subject. While it is sometimes difficult to define a reliable compliance metric for every study (OAB and other event-driven diaries are difficult, for example), there are some interesting trends that are starting to emerge:
Africa is the leading region with 94.5% compliance
Personally, I find this surprising. However, we only have limited amount of data for Africa – only 2% of our subjects are from Africa (n=1785). Africa is followed by Eastern Europe (92.8%) and Asia (92.1%).
Adult populations have the lowest compliance
This is another surprise, since most people would expect the ‘tech-savvy’ groups to be high on the list, but the statistics are almost the opposite. Infants (e.g. caregivers) lead with 93.9%, elderly subjects are close behind with 92.3%, and adults are well below the average with 88.5%.
Healthier populations perform better than sicker populations
This is not surprising at all. Our vaccine studies have 93.8% compliance with a standard deviation of 10.9, compared to 86.1% / 15.7 in Pain and 86.3% / 18.2 in COPD. What is interesting is that there are studies within some of the ‘difficult’ indications that have very good compliance and there are studies that are not doing so well. It proves that it is possible to get very good results even within the difficult indications. This actually raises some interesting questions – what are the key success factors in these studies and what can we learn from the studies that are not doing so well?
Compliance success factors
The space allotted for this blog post is not enough to discuss the details, but here’s my short list of what I personally consider to be the key points:
- Subject-focused eDiary design that will proactively guide the subject AND study sites through the study
- Efficient compliance management tools for sites and CRA’s – online reports, email notifications, etc
- Active study team from the Sponsor / CRO who will use the compliance monitoring tools and stay on top of everything
These are all generic concepts that can be adopted in virtually every study. There are obviously many protocol / disease specific best practices that can be utilized and study Sponsors should expect the eDiary vendor to be able to proactively guide them through the design process to ensure best results for each study. This is exactly what the CRF Health teams are used to doing – we actively use our existing experience and best practices and provide these recommendations to our Sponsors.
For those interested in the topic, subject compliance was discussed as the topic of our August 2011 webinar. Request access to the session by clicking this button:
Best Regards,
Kai Langel
Senior Director, Technical Support, CRF Health
Posted by Kai Langel on Wed, Jul 06, 2011 @ 04:00 PM
CRF Health has been very active recently by doing some usability research on our own. We have conducted workshops with patient focus groups as well as usability research and testing with different devices and indications. One of the most interesting and challenging groups we have worked with are patients with Parkinson’s Disease. We first did some PD studies a few years ago and got to understand some of the challenges these patients have to deal with as well as the research issues for clinical studies. The requirements for data collection in PD are the most demanding for the patients that I have seen in my 11 plus years of doing ePRO. This combined with the motor symptoms and the ON / OFF cycles the patients have to go through is not exactly an easy challenge for them. I took a bit of a personal interest to the matter after the FDA PRO guidance came out in 2009. I realized that paper data collection is no longer going to be a very reliable option for collecting this data and that efficient and easy-to-use solutions are needed. It wasn’t very clear in the beginning what those solutions might be and it took some Sponsor input, creative thinking and significant advances in device usability to come up with a new kind of user interface to collect this data.
The end result is quite interesting. Where previously it took several ‘clicks’ to enter symptom status data for a single 30-minute slot, the patients can now enter several time points at once, with just a single click each using a very visual and logical interface. There is minimal amount of text on the screen as the visual, iPhone-inspired icons are quite self-explanatory.
| Figure 1 - Improved and efficient interface for PD symptom status collection |
We have also benefitted greatly through collaboration with Dr. Robert Hauser, who is the creator of the original paper version for the ON / OFF diary that is so typical in PD research. Dr. Hauser has given us great insight into current PR research and medical issues and has been instrumental throughout the process. We have also tested the design with some other investigators in Europe. The feedback has been very positive and there have been great practical suggestions for improvement from the clinicians.
Most importantly, this design has been tested and well received by patients. Patients prefer the electronic version to paper, although some had initial reservations about their ability to use it. However, once they gave it a try, most found it easy to use. There were some patients who are simply in such an advanced stage of the disease that they commented they would not be able to use either method. Many patients, though, especially liked the fact that with the electronic version, they can easily change their answers and can avoid having to do any handwriting at all. In fact, the electronic version can be used with just the patient’s finger, which is an advantage to many of them.
It has been very interesting to work with the challenge of ePRO for Parkinson’s Disease patients and I’m quite pleased with the results so far. We hosted a webinar on the topic a couple of weeks ago with Dr. Hauser as a guest speaker. It’s all recorded in case you’re interested in learning more of the details of our work so far. Click this button to request access to the recorded webinar.
Best Regards,
Kai Langel
Senior Director, Technical Support, CRF Health
Posted by Rauha Tulkki-Wilke on Fri, Jun 10, 2011 @ 11:39 AM
“Computers and the Internet are as much an integral part of our lives as coffee makers. We would not want to live without them.” This is a comment that I heard recently when interviewing elderly cancer patients.
A surprising comment? Not really. Statistics show that most people in Europe, US, and many countries in Asia Pacific are mobile phone owners and Internet users. Those of you who are interested in numbers, check out the links for more information.
Another interesting patient encounter occurred while sitting next to an elderly patient who needed to have a personal assistant help in completing a traditional paper questionnaire but could easily use an electronic diary without support to report his outcomes himself. This made me think of all the times I’ve had to answer the most often asked question about ePRO: “You don’t seriously think elderly patients can use an eDiary?”
I was sitting next to the patient at an oncology association office in Helsinki with the goal of collecting feedback because we believe strongly that if patients are provided with an ePRO tool that they find easy to use in their daily lives, they are more likely to report data on a timely basis, and not think of it as a burden. This means increased compliance and fewer errors in data entry, both critical factors in the collection of high quality data.
This was part of a series of focus groups Diane Wild and Sarah Hearn from Oxford Outcomes, and I conducted in the UK, Finland and Poland with the goal of learning more about factors that impact patient preferences for different ePRO modalities. We learned a lot about how essential technology is in patients’ lives.
First of all, generally all patients regardless of age and functional status prefer to use an electronic diary rather than a paper diary because:
- It is easy to complete ePRO: it is an intuitive process with no need to try to figure out what to enter and when.
- It is easy to carry an electronic diary: in studies where patients answer questions several times a day, it is easier to put an eDiary in one’s pocket or purse than a pack of papers.
- ePRO is discreet and private: no one can see the questionnaires or answers to them. Especially patients who need to report data during the day at the office or school find that important.
- It is just the way of life in the 21st century. As basic services, such as banking and healthcare, are now used online, people consider paper diaries old fashioned.
We found that there were some differences between age groups and also based on the functional status of patients:
In general, older people without disabilities preferred an eDiary application that opens when the device is turned on and hides all the other functionality on the device – they found it simple and effortless to use. When it came to long assessments, the patients strongly preferred to fill them out on an electronic tablet or online, since it is faster when there are several questions on the same screen. Several commented that it is actually easier to answer questions when they are shown on the same screen with the other questions – an interesting comment from an instrument design and modality transfer point of view. For home use, they considered a handheld eDiary to be the best solution. It is easier to carry around.
The younger patients in our study (20 year old students in Poland), wanted to use an online diary for all data entry. They wanted to complete long assessments faster on a computer, and shorter, more frequent diaries on a smart phone using a mobile Internet browser.
In the UK we interviewed elderly patients with Parkinson’s Disease or Arthritis who had movement disorders and some issues when using their hands. They felt it was easier to use an eDiary than paper. They had a very strong preference for tablet eDiaries because they were sturdy and significantly easier to handle than handhelds or the web-based eDiary. However, they preferred all electronic methods over paper!
My role at CRF Health is in R &D. I work in product management and for me personally, the key take-away was that patients expect to use electronic data capture in clinical studies. Their preferences are based on how well the method fits into their life, which is mainly driven by their lifestyle and how often and how much they need to enter data into the eDiary.
And the answer to the question about ePRO I mentioned earlier – don’t worry whether patients can use it or not, focus on selecting the method that is best for them and your research!
Best regards,
Rauha Tulkki-Wilke
Director Product Management
Posted by Mika Nuutilainen on Tue, Jun 07, 2011 @ 10:00 AM
In the last blog article I discussed the definition of cloud and its advertised benefits. In regard to clinical trials data, the biggest problem with cloud services is that there is no easy way to ensure that data really will stay unaffected when stored in the cloud service. How could I be assured that no other application, person, or service will change my ePRO data? And how could I guarantee the FDA the same.
Major cloud computing service providers do have security certificates, but it is still uncertain for a customer how well the data is really taken care of. Cloud service providers have developed their processes to provide service within minutes and completely automatically. So IQ, OQ, and PQ (system validation steps) with objective evidence could be hard cases to provide for in all server installations. When your servers are provisioned and deployed as virtual machines in the cloud, validation of your installation in the way we are used to is not possible anymore.
In April 2011 part of the Amazon EC2 cloud service was down for several days. It was widely broadcast in the news, especially since many well-known social media services using EC2 were down through the whole break. These companies included Quora.com, Reddit.com, GroupMe.com, Scvngr.com, and foursquare.com. And when the service was finally restored, Amazon declared in the aftermath that they had lost 0.07% of data. That does not sound like a huge amount, but considering it has been estimated in 2010 that Amazon EC2 contains 1.8 million server instances, the amount of lost data cannot be ignored.
All top 5 cloud providers have suffered unexpected downtimes (Amazon EC2 in April 2011, Rackspace has had multiple power failures over the years, salesforce.com in 2009, Google Apps multiple times over the years, Microsoft Azure for 22 hours in 2009). Some have had serious power failures and some connectivity issues, but all biggest cloud providers have had disruptions, and are likely to have them in the future.
Cloud services provide a fast way for small companies to launch a new product and also allow them to focus on developing product features instead of fine-tuning the servers. Also cloud enables them to increase server capacity immediately when needed. With cloud there is no need to invest on big server farms, basing decisions on uncertain sales budgets. Or to suffer from poor service quality when a service becomes too popular and servers cannot handle all the requests. When analyzing if an already set-up and well working service should be integrated into a cloud, it is not so straightforward; there has already been a lot of money invested on your servers and other infrastructure.
It is clear that in the long run the cheapest way of buying server capacity is to buy instances from a cloud, so it is inevitable that clouds will be used in clinical studies too. But it most probably means that a specific clinical cloud will be implemented for that purpose. Clinical requirements are so tight that in order to fulfil them, no cloud provider would be interested in making their whole cloud compliant. In order for such a clinical cloud to be created, a lot of process harmonization and co-operation will be required between Sponsors, CROs and trusted 3rd party providers such as CRF Health, and some well trusted and established IT provider to make this happen.
For a well-established system, integration to a cloud server and storage system could be considered when a new step is taken in business or when the whole existing infrastructure is being upgraded. There are no cost savings easily obtained when replacing existing hosting centre and dedicated servers with a cloud.
Meanwhile, CRF Health continues to have well working dedicated TrialMax® servers and trained professionals to keep them running and providing good quality service 24/7, day after day. And when you ask who guarantees that your data is protected – we can honestly say that we guarantee it – and are really in a position to do so.
Best Regards,
Mika Nuutilainen
Director, Product Development, CRF Health
Posted by Mika Nuutilainen on Mon, Jun 06, 2011 @ 06:30 PM
If someone asked you recently what is the most hyped computing buzz word today, you probably would have answered: Cloud Computing. The clinical world has noticed this, and has begun asking if cloud computing should be used in clinical trials. That is a good question. But first I need to ask, what do you think cloud is? That must be clear before it can be discussed at all. Is it an IT service you can buy over the network? No, that is just a web service. For our convenience, the term cloud has been defined by the National Institute of Standards and Technology (NIST):
"Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction."
NIST says that applications and services are included in the definition, but let’s analyze what some of the biggest cloud providers are really offering.
Amazon EC2 provides servers, network connections, firewalls and other security services, operating systems, storage space, databases and application servers. The second largest provider, rackspace.com, provides cloud servers, storage, and load balancers. They also support some database implementations, but they are not included in the service itself. So it is safe to assume that cloud typically provides virtualized servers including operating system and optionally databases and application servers, with a cost effective pricing model.
The next question is, for what purpose would you need cloud? Clouds are a trend now because they provide reduced costs, scalable storage options, high automation level (e.g. automatic updates), flexibility, more mobility (remote access from everywhere), ease of implementation, and good performance and durability. Regulated software has not been known to be riding the first wave of the latest trends. How about this time? Are the benefits too appealing?
Could these all be applicable for clinical trials too? Reduced cost? Not sure, but at least everybody welcomes that. Scalable storage? Yes of course. Highly automated system? There we have our first problem, because we just don't like automatic updates in clinical studies. While our virus scanners can be automatically updated, how about our software that stores the audit trails and source data? We are not interested in that being done automatically – we want to validate our updates. Remote access? That provides nothing new. But the rest are the interesting parts. Flexibility, when considering the number of deployed servers at a given time would be a very useful. Easy implementation of new servers together with good performance and durability are market promises that I would like to be able to utilize.
A cloud implementation of TrialMax®would be a system where data is collected with our TrialMax® Touch or Slate devices, but instead of sending the data to our dedicated hosting centre and dedicated servers, it would be sent to the cloud, to an unknown location. Hopefully, the data would be there somewhere.
In the second part of this article I will address some of the concerns regarding clouds and clinical trial data.
Best regards,
Mika Nuutilainen
Director Product Development, CRF Health