Posted by John Harrison on Thu, May 03, 2012 @ 02:00 PM
March and April are traditionally two months that for me involve an unusual amount of business travel. This year has been no exception and in the past 6 weeks I have visited Istanbul, Paris, Vienna, Dallas, Washington, Philadelphia, Cape Town and at the time of writing I am in Tokyo with just Sydney and Melbourne to visit before returning home for Easter. The work has been varied and interesting, including advisory boards, user group presentations and a good number of investigator meetings. What is different about this year is that the indications under discussion have included depression, schizophrenia, ADHD, Parkinson’s disease, Alzheimer’s disease and a variety of other diseases, including hypertension, diabetes and fibromyalgia.
Of course the theme of these discussions and training sessions has been the measurement of cognitive change. What has been remarkable is that the same issues of, i) which cognitive domains (e.g. memory, attention, executive function, etc.) are impaired, ii) dealing with practice effects, iii) issues of rater training, etc. have occurred in the context of each indication. What is also remarkable is that the same or at least similar candidate tests have been proposed for use across all these indications. Sometimes the very same test has been proposed, so tests of executive function such as the Controlled Oral Word Association Test are near ubiquitously employed. Sometimes the specific test to be used varies. For example, episodic verbal memory (psychologists’ fancy name for remembering a list of words) is measured using the Hopkins Verbal Learning Test in patients with schizophrenia, the ADAS-cog Word Recall test in patients with Alzheimer’s disease and the Rey Auditory Verbal Learning Test in patients with depression. However, at root these tests are all measuring the same cognitive construct using a word list learning paradigm. Word list learning is also a component of many of the multi-domain composite tests such as the MMSE, MoCA and SCOPA-cog that get used to measure cognition in a variety of indications. Often the measures selected are traditional measures, so-called ‘paper-and pencil’ tests, borrowed from clinical psychology. More often these days these measures are augmented or replaced by computerized measures from commercial systems such as the CDR System, CogState, CANTAB, Cogtest and CNS Vital Signs. Experts typically have their own views on which is the best test of each cognitive domain, though are typically agreed on what the fundamental cognitive domains are. Characterising the domains of interest was helpfully provided by the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) initiative. The seven domains listed are typically impaired in patients suffering from Cognitive Impairment Associated with Schizophrenia (CIAS). A recent review from Milan et al has described how many of these domains are compromised in a number of psychiatric indications. Impaired performance in these same domains is also often seen in dementia and other neurological diseases.
The MATRICS group chose to borrow tests from clinical psychology to cover the specific domains of function. This approach whilst pragmatic as a legacy inherits some of the traditional challenges of measuring cognition. An alternative approach would have been to develop brand new tests, specifically designed to address the measurement of cognitive change and ideally computer-based. Computer assessment can, by delivering stimuli at the prescribed time and capturing data, lighten the burden of test administrators. Computer assessment allows also for the rapid transfer of trial data, which lends itself neatly to prompt inspection and audit, ideally automatically. These new measures could be designed to more obviously reflect the tasks actually encountered in everyday life. Back in the 1970s Ulric Neisser begged psychologists to develop measures of cognition that had what he called ecological validity i.e. tests that index cognition using paradigms encountered in real life. Developing a brand new set of tests could have addressed up front the issues faced when using measures in different linguistic and cultural settings. We could also employ sophisticated psychometric approaches such as Item Response Theory to develop tests that can be titrated to the abilities of the patients we test. These approaches don’t preclude the use of paper and pencil testing, but would benefit from computerised administration, as with computers only the items appropriate to the current patient would be presented. This would save time and, potentially, confusion. We could develop a selection of brief, easy to administer, sensitive ‘best of class’ measures. The challenge always is that there is precious little money to be made in developing new tests. Consequently we often borrow measures that were designed as simple, fairly rudimentary bedside measures of cognition and press them into service as efficacy measures in clinical drug trials.
Initiatives directed at improving our cognition assessment measures are often specific to individual indications, such as was the case with the MATRICS program. Having spent time in different places, with different groups, considering the same issues for different indications, I’m wondering whether we should pool our skills, experience and resources to develop a cognition assessment that would serve all our needs. A true, international consensus would lend credibility and authority to such an enterprise. The support of stakeholders such as academic and clinical experts, regulatory bodies, commercial test vendors and other interested groups would be important in helping to get this done.
Academic and clinical research groups are a good source of creative ideas and innovative solutions. However, the process of integrating computer based assessments into a format acceptable for clinical trial use is an expensive and exacting process. Perhaps one way to meet current need would be for partnerships between research groups and commercial entities, which could operate in collaboration to mutual benefit. An assessment system capable of being used in a variety of CNS indications could accommodate the needs of many stakeholders. Regulators could include reference to a system in notes for guidance, sponsors could employ standard versions in their trials and vendors could be licensed the rights to distribute and support the resulting measures. Ultimately patients, caregivers and society as a whole would benefit from knowing that the drugs for which marketing approval was being sought had been shown to be truly efficacious.
Best Regards,
John Harrison, PhD
Scientific Consultant, CRF Health
Posted by John Harrison on Wed, Feb 22, 2012 @ 01:30 PM
I managed to find time last November to attend the second CNS Summit Meeting. It seems difficult to find time for these events, but I am always pleased to have the opportunity to attend. The theme for this event was development and innovation in CNS drug development, a topic close to my heart. Plus the Boca Raton Resort & Club is no bad place to hang out. We started with an excellent ‘Call to arms’ from Patrick Kennedy, an inspirational speaker (and if you closed your eyes you could hear the same rich speaking tones of his uncle John). His topic was ‘The Importance of Collaboration and Innovation in Developing New CNS Treatments’. This is just one of the drives for innovation, a keenness to establish collaborative arrangements in a pre-competitive space. The theme is one of sharing, especially of data collected for validation purposes. I’m convinced this is an idea whose time has come. Working in psychiatric and neurological drug development is a considerable challenge. Luca Santarelli, Sr. Vice President at Roche, has a magnificent slide on which he uses a Dantean metaphor comparing different therapeutic areas – with CNS development mapping to ‘hell’.
I get the opportunity to work in a number of CNS indications, but Alzheimer’s disease (AD) still occupies more than 50% of my professional time. AD seems to me the quintessential example of the challenging CNS disease. We need desperately to find a cure, or even some symptomatic relief, as the number of cases in many countries is set to treble over the next few years. Not too long ago we had a number of compounds in late stage development, most of which were focused on amyloid plaques, one of the hallmark pathologies of this dreadful disease. Things are not so rosy now, with lots of high profile failures in the interim, though we still have Bapineuzumab and Solanezumab to read out.
Remedying CNS disease usually means finding a compound for diseases for which the cause, or causes are seldom known. We have become adept at screening for disease and have often met with success in identifying the pathological markers and have had some success with developing biomarkers. We’ve also become very aspirational with respect to compound development, genetic screening, neuroimaging, etc., and have ploughed a good deal of cash into the enterprise and into the aforementioned collaborative initiatives. Yet there is still a good deal to do be done, especially with respect to the measurement instrumentation we employ.
For example , take cognition, a field of key interest for me. As is the case for many CNS indications, when we need a measure we tend to borrow from clinical psychology and end up employing tests that were rarely developed for repeated assessment of the kind we need to conduct to detect drug effects. A consequence of this approach is that we have employed measures such as the Mini-Mental States Examination (MMSE) for recruiting patients and the ADAS-cog for measuring efficacy. However neither test was designed for the purposes to which we have put them.
The inadequacies of the MMSE were recently very publicly highlighted in a BBC drama ‘Five Days’, which began with a caregiver tutoring her AD suffering mother on the content of the scale! The inadequacies of the ADAS-cog have also been evident for some time. This is not the place to rehearse the various issues attached to its use, but we might profitably consider a key issue, the presence of ‘ceiling’ effects. Here I am referring to tests on which study participants tend to perform perfectly and which are consequently incapable of showing improvement. Previous studies have reported that performance on nine of the 12 most popularly used subtests reaches ceiling in about 80% of all patients. The three tests that are not prone to this effect all assess memory. A popular comment in defence of the continued use of the ADAS-cog is that a genuinely efficacious drug would be capable of demonstrating improvement. This may or may not be true, but it still seems troubling that such an innovative, creative and progressive group of professionals such as CNS drug developers would be willing to make do with the status quo. Incidentally, the ADAS-cog can be a confusing instrument to administer and consequently recording scores can be a source of error. We could also bring a little profitable innovation by parsing early returns from site for any conspicuous errors. This could be an automatic process based on known data characteristics, such as the correlations between different subtests.
I began considering the content of this blog a couple of weeks ago when it was still appropriate to wish email correspondents a ‘Happy New Year’. New Year is a time to be resolute and it seems to me that this might be the time to start work on improving the quality of our measurement instruments so that when we do finally explain the causes and develop cures, we can characterize the effects of our drugs using first rate, best of class, validated tools.
Best Regards,
John Harrison, PhD
Scientific Consultant, CRF Health
Posted by Pekka Keskiivari on Fri, Feb 17, 2012 @ 01:00 PM
I note that the discussion regarding the future of 2G networks has been initiated by a few machine-to-machine providers who seek to use claims of an impending phase-out against their competitors deploying GSM-only equipment. There are currently no published plans or announcements from any wireless carrier regarding phase-out of their 2G networks, and this lack of any published plans creates an opportunity to present such claims.
There is naturally an on-going evolution of networks, and eventually any technology will become out-dated. To analyze the future of 2G, we should try to understand the drivers for wireless carriers in the development of their networks. Essentially, their decisions are based on the relevant regulative framework and business case. Wireless radio spectrum is typically licensed for a fixed term and for specific use, determining which wireless technology can be used to operate on the licensed frequency band. While it is difficult to predict how the license landscape will change over time in each country, the periods tend to be fairly long, and it is not reasonable for the regulator to make abrupt changes that would severely affect how the carriers can utilize their massive investments in network infrastructure. Carriers will not phase out a technology that is generating them a profit if they don't have to, and if it does not make business sense to them.
Let's look at why and when a network operator would want to phase out a given network technology. There are technical concerns regarding the use of different wireless technologies on the same frequency bands, and traditionally large chunks of the wireless radio spectrum have been dedicated to a given technology (like GSM, WCDMA etc). Also, as can easily be seen from the specifications of handsets, support for each technology/frequency band set needs to be built into the radio design on the phone. Consequently, there have been strict and established standards for which technology to use on which frequency band.
With the development of wireless technology it has recently become more viable to be flexible in the assignment of technologies. A concept of "refarming" frequencies has been introduced and is also increasingly endorsed by regulators. In refarming, frequencies previously allocated to one technology, like 2G GSM, are reused using a new technology, usually with the goal of providing more capacity to high speed mobile broadband services. Overall, the rationale for this process is highly dependent on both the frequency bands held by the carrier, the customer needs and the business case. As parts of the old frequency band are being refarmed, this first results in a drop in capacity (how many simultaneous users are supported), and only impacts coverage (how good and comprehensive the service is) once the process goes so far that complete bands are overtaken by the new technology. The carriers will obviously control the process in such a way that their existing subscriber base is not adversely affected.
When assessing the possible effects of 2G refarming, it should be noted that the new technologies like LTE are really not supported by many handsets yet. Each operator has a unique selection of licenses, frequencies and technologies that they need to leverage. They need to support their existing customers, so switching to a new technology that their customers' phones cannot use is not a viable option. For example, a carrier may have licenses for 850 and 1900 Mhz used by GSM, and a WCDMA license for 1700 MHz, but even the newest phones may not support WCDMA on this band, and will so rely on 2G to operate in this carrier's network. The carriers need to plan their possible refarming upgrades carefully to match the evolution of technology standards and handsets in order to minimize the negative impact on their existing subscriber base, at the same time maximizing the total return on their investments. In some countries like Finland where operator subsidies have not been commonplace, there is even less carrier control over the handsets used by the subscribers, and a significant portion of users are still using old 2G-only phones. To conclude, the machine-to-machine market that was mentioned in the beginning is largely static with slow turnover of technology. This all means there is a large existing base of devices in the current networks requiring continuing support of the traditional 2G service.
Mobile broadband services will continue to grow in popularity and carriers will need more radio spectrum in order to support the growth of these services. However, refarming of existing frequency bands is not the only alternative since new frequency bands are also being allocated. For instance, part of the radio spectrum previously used by analogue TV is going to be reallocated to wireless broadband (so called ‘digital dividend’). It can be estimated that refarming will probably have an effect on availability of 2G services in the next ten years. Nevertheless, any wireless carrier that will engage in this process will provide a graceful migration to the new technologies and is likely to make announcements in plenty of time before they see it will affect the existing GSM subscriber base. Until these announcements, partial refarming of the 2G frequencies may cause some decrease in the capacity of GSM-based packet data services, but this will have minimal or no effect on ePRO data collection due to the relatively low data bandwidth needed for collection of data from the eDiaries.
Best Regards,
Pekka Keskiivari
CTO, CRF Health
Posted by Jane Carter on Fri, Dec 16, 2011 @ 11:30 AM
While providing site training at an Investigator Meeting recently, I came across some interesting comments from sites that were not familiar with using eDiaries or tablets to capture PRO or ClinRO data. One study coordinator was concerned about using eDiaries with certain populations, and was wondering if we were going to provide paper checklists or backup questionnaires on paper for subjects who were not comfortable using eDiaries. This comment brought me back to the realization that while I have been working with ePRO for over six years, there are some people who still are not familiar with the technology or the risks inherent in mixing modalities of PRO collection. I thought I would speak to some of these items, in hopes of providing a basic understanding of ePRO and some items to consider.
When choosing to collect Patient Reported Outcomes for a clinical trial, particularly for a primary or secondary endpoint, it is important to ensure the quality of the data being collected. Most of us have heard of or experienced firsthand the “parking lot syndrome”, where patients fill in dosing diary cards or other PRO questionnaires in the parking lot immediately before going in for an office visit. This is one of the main reasons sponsors choose ePRO over paper – ePRO technology allows for entry “windows” to ensure the data is being captured in a timely manner, and is not subject to the failings of memory recall. I’m lucky I can remember what I had for breakfast yesterday, let alone how I felt or at what time I took medicine!
While some people are still concerned about technology and dealing with certain patient populations – in particular, the elderly – our statistics have shown that eDiary compliance by elderly subjects is extremely high (>95%). Technology can be scary to people, but I have found that there are six key factors that improve the likelihood of success:
Success Factor 1 – Simple eDiary Design. I follow the KISS model when designing ePRO collection tools – Keep It Short and Simple. If the eDiary is too complicated, with strict entry windows and hard stops built into the eDiary, then workarounds need to be built and processes changed if a subject deviates from what is expected. While I design to help the subject follow the protocol, I try not to make the design so rigid that the programming increases frustration and reduces compliance.
Success Factor 2 - Logical Edit Checks and Information Messages. Inclusion of logical edit checks and information messages improves the quality of the data collected and provides guidance for the subject. For example, if a subject is filling in a multi-question PRO and one response prompts for additional questions, then I build in the branching logic to ask the appropriate questions on a “Yes” answer, and to skip those questions on a “No” answer. Additionally, I build in popup messages that remind the subject if he/she answers a question that is outside an expected range: for example, if a subject enters an amount of medication that is outside the expected range for the protocol, then the eDiary would remind the subject to call the doctor.
Success Factor 3 - Reminder Alarms. Reminder alarms can improve compliance and can provide “peace of mind” for subjects. If a subject is filling in multiple questionnaires daily and a separate weekly questionnaire at a different time of day, I would build in an alarm to remind the subject to fill in the daily questionnaires, and I would build in a separate alarm that would sound to remind the subject to fill in the weekly questionnaire.
Success Factor 4 – Training Material. The fourth success factor encompasses a wide area of items under the umbrella of Training. Providing detailed, hands-on training to the monitors and site personnel using a “Train the Trainer” approach helps facilitate the training subjects will receive at site. Additional support in the form of a detailed Site Manual helps reinforce the hands-on training and provides extensive troubleshooting and FAQs to ease site personnel with both their use and subjects’ use of the eDiary. Having a Training Module built into the eDiary allows the subject to practice using the device and to become familiar with the device before leaving the office. Providing easy to read, step-by-step guides for the subject to read and follow when using the eDiary helps ensure a comfortable transition from on-site support with the study coordinator to self-use at home. While these guides are most used by older subjects who are less tech-savvy, they are very helpful for all subjects. The guides should be easy to follow, include pictures of both the device and the screens to walk the subjects through everything they need to know about the device, and have a list of steps that they need to follow at any given time point. Finally, access to helpdesk support for both sites and subjects is a key “safety net” to reinforce training and to get sites and subjects “back on track”.
Success Factor 5 – One Method of PRO Collection. Sites often want “back up” methods of collecting PRO, and want to have paper questionnaires on hand in case a subject is not comfortable using ePRO. I am now seeing protocols have an inclusion criterion that states that the subject must be willing to use an eDiary. I recommend this be included in all protocols. If a study provides alternatives like paper, then we not only deal with two points of collection and the multiple processes that need to be built to ensure consistency, but we also send a mixed message to the subjects and undermine their confidence in ePRO
Success Factor 6 – Site Personnel. Whenever I am at an investigator meeting and am training sites on how to use the ePRO devices, I always encourage site personnel to be “cheerleaders” or “champions” of ePRO. I remind them that If they act frustrated with a device or appear to treat the ePRO collection tool as just one more piece of equipment that they need to deal with, this attitude is transferred to the subjects. I have seen a higher incidence of helpdesk calls, a lower rate of compliance, and a larger number of issues at sites where the site personnel appear unwilling or unmotivated during that initial training.
Conversely, I have spoken with study coordinators who are high enrollers with high compliance rates, and I have discovered that universally those study coordinators make it a point of knowing everything they can about the devices, are engaged and ask lots of questions during the training, read all of the study materials including the guides given to subjects, and motivate their subjects by saying things like, “Your data is so important to this study, that the sponsor spent all this money getting these cool devices to collect your data. You (the subject) are key to the success of this study!”
In conclusion, while we can provide all of the technology, design and support for the study, we must never underestimate the human factor, and how we present to and train our subjects on the use of ePRO. It is a team effort! The sponsor and ePRO provider can design and deliver quality ePRO to the sites, but we need the help of the doctors, nurses, monitors and helpdesk to provide seamless support and to be champions of ePRO in order to ensure subjects are successful.
FUTURE TOPICS:
ePRO Basics – Partnering with an ePRO Provider – What to Consider
ePRO Basics – Design Considerations for ePRO
ePRO Basics – Optimal Training Methods When Using ePRO
ePRO Basics – Monitoring ePRO Data: Does ePRO Data Get “Cleaned”?
Best Regards,
Jane Carter
Program Manager, CRF Health
Posted by John Hutchin on Fri, Dec 02, 2011 @ 10:11 AM
Part II
Cloud Computing
This was a very interesting session. By now we’ve all heard the term “cloud” computing – even my wife, who is a dedicated anti-computer person and just recently got a mobile phone, asked me about it last week. Still, I find myself thinking "There’s no way I’m putting my personal information at risk by trusting it to something so nebulous that it’s actually named the 'cloud'". It turns out that eClinical’s main interest is really in private clouds these days – not public ones. Nick Neri of PharmaPros and Kevin Ahonen of Biogen-Idec did a great job 
presenting a case study of cloud technology put to good use. They very clearly described the concept of cloud computing and how it differs from simply hosted solutions. It was discussed that there is some information that is not really meant for the cloud – at least not yet. In the case study that was presented, only operational data is stored in and accessed from the cloud. What is operational data? It’s the data about the data. Metadata if you will; status values and metrics and information pertaining to how clinical data is being used. Having an external system out there that can securely access data from many distinct, “silo” locations within your company’s clinical infrastructure – in a controlled fashion and without adding burden to your existing IT system – is an idea that can meet a lot of needs. “Terminology standardization” was brought up; the need to reconcile different naming conventions across different systems. This is vital to cloud computing so that information is correctly aggregated and people know what it is they’re being presented with. As with a lot of things these days, the technology implementation is often simpler than getting people to accept new processes and open their minds to accept new ideas. This is especially true for those people who have had much success living in their very tall silo. “Don’t fix it if it’s not broken” is a valid argument unless what’s currently working needs to work within a much larger scope and cannot. Nick and Kevin made a very good argument for stepping back in order to be able to see the forest in addition to the trees. There was a lot of interest in this and a lot of interesting side conversations throughout the week.
Workshops
The week wasn’t just a series of presentations, with the attendees sitting passively and listening. The eCF meetings also have a number of workshops focused on issues relevant to eClinical and designed to formulate a group consensus that can foster change in the industry or recommend best practices.
Each of these workshops broke the attendees into groups of 8 or so people per table. Tables were assigned a particular task and reported back to the entire group at the end of the breakout session. The details of these workshops are beyond the scope of this blog, but I can say that they were well run, interesting and designed such that each group had a cross-section of people with a variety of backgrounds and expertise.
There were separate workshops pertaining to:
- Site qualification of sites that intend to use electronic health records (EHR) as source data for clinical trials. Regulations require that such an EHR system meet the same requirements as others uses for eSource in a clinical trial.
- A risk-based approach to SDV
- Leveraging Electronic Health Records for clinical research
- Serious Adverse Event integration
Attendees were assigned to groups such that each table contained a wide range of experience: CRO, ePRO, Data Management, Safety, etc. This helped to ensure a thoughtful exchange of ideas and kept the discussion lively.
If you’re not currently a member of the eClinical Forum, I can highly recommend it. If you are, I hope to see you at the next meeting!
Best regards,
John Hutchin
Senior Director, Technical Support
CRF Health
Posted by John Hutchin on Thu, Dec 01, 2011 @ 09:29 AM
Can it really be 7 years since I attended an eClinical Forum (eCF) meeting? Back in 2004, the meeting I attended was held in Cincinnati and hosted by Proctor & Gamble. At that time, I was filling in for our regular representative who had conflicting obligations. I thought then 
what a good organization this was. Beginning this year I’m fortunate enough to represent CRF Health at the bi-annual eCF meetings in the US. I love this organization. It brings together a diverse group of pharma, biotech and technology provider representatives to exchange ideas and help shape the future of the eClinical landscape. Ideas appeared, conversation flowed and it seemed I could literally feel progress being made. Everyone there was genuinely interested in advancing the field of eClincial and coming up with best practices and recommendations. What I really like is the feeling of comaraderie and the lack of commercialism. I sat with competitors and clients alike and discussed issues that can improve our ability to bring better healthcare products to market faster. What a refreshing experience.
Day 1 began with presentations and discussion related to integration. Case studies were presented by Allergan, PHT and CRF Health. The overall feeling I got from these case studies is that the time is ripe from a technology perspective for real-time, transactional integrations based on web-services APIs. I commented (to some laughter) that CRF Health has had web-service APIs for a number of years now, but it was like having one “walkie-talkie”… there was no one to talk to! I believe that the maturity of this technology, together with CDISC and HL7 standards, is finally resulting in adoption rates that will rapidly transform the eClinical landscape in the next 2-3 years. I don’t say that lightly. The clinical research industry evolves and changes direction about as fast as you can turn the Titanic with a canoe paddle. In fact, when I attended my last eCF meeting in 2004, there were representatives from Siemens and GE Health forecasting the integration of Electronic Health Record systems (EHR) and clinical systems within 5 years. At last week’s eCF meeting in Newport Beach, California, the same topic was discussed in detail – and I hadn’t really detected any meaningful progress. The same major problems seem to be blocking our path, privacy concerns, lack of standard nomenclature and an effective business model that will sustain such a system beyond an ideological pilot. Still, we need to start somewhere, and real-time integration is now possible with very little capital investment. Technology is such that we no longer need to worry about the “plumbing”. We can focus on determining the best information to send through the pipes and how to best present it to the end user. One step at a time and soon we’ll be off and running. It’s apparent from these case studies that learning to walk in this environment is easier than first anticipated.
Open Source
We then had a presentation and discussion of open source for eClinical software. I must admit that I remain a skeptic. To me, open source software seems best suited for foundation technologies that underlie specific needs, rather than meet those specific needs themselves. Open source operating systems seem more feasible to me than open source eClinical software. Maybe I’m just too old; working on something only to give it away for free goes against my idea of a good business model. In this regard, turning the Titanic with a canoe paddle seems to be further complicated by the need to convince the captain to even dip the paddle into the water. I’d love to hear other’s thoughts on this.
In the second part of this blog post, I will discuss Cloud Computing and the eClinical workshops that were held at the eCF Meeting.
Best regards,
John Hutchin
Senior Director, Technical Support
CRF Health
Posted by Kai Langel on Mon, Nov 28, 2011 @ 01:00 PM
Diane Wild is a colleague that I have the privilege of working with closely. She is at the leading edge of what’s new with ePRO in terms of best practices and current industry guidelines regarding instrument migration, usability testing and linguistic validation. She is one of the founders of Oxford Outcomes, a leading international health outcomes consultancy that was recently acquired by ICON. Not only is Diane a well respected professional, but she is also a wonderful person much liked by myself as well as many of her colleagues at Oxford Outcomes.
I had an opportunity to pick her brain and conduct an interview at the ISPOR Annual European Congress in Madrid, Spain.
Me: The FDA PRO guidance came out in 2009. How has it been received and adopted during the last two years?
Diane: There was a lot of uncertainty once the draft guidance came out 
and nobody knew quite how the final guidance would turn out. It was finally released in 2009 and since then, many companies have conducted webinars to help the industry understand the implications of the final guidance. Now after two years, there is a general feeling of acceptance of the guidance and deeper understanding of it amongst vendors as well as Sponsors. For example, in the beginning, nobody quite knew what the PRO evidence dossier should look like, but now many people know how to put them together. However, there are still some specific questions. For example, there is some ambiguity regarding the use of mixed modes of delivery, such as paper and electronic PROs in clinical studies as well as how to show evidence of content validity when doing linguistic validation.
Me: What about the EMA? How has the FDA guidance affected the way things are done in Europe?
Diane: There is a general acceptance that the FDA documentation will be acceptable for EMA submissions as well. We also have more experience now in submitting to the EMA and there is a greater emphasis looking for evidence of psychometric properties of measures rather than focusing so much on content validity. The EMA also has a more pragmatic approach, especially when considering the use of widely used measures.
Me: Doesn’t the EMA differentiate between ‘simple’ measures, such as a symptom scale and more complex QoL instruments?
Diane: Yes that is right. The EMA currently differentiate between PROs that are well established and measure symptoms such as pain or itch in psoriasis, and multidimensional measures of health-related quality of life. Different levels of evidence are required by the EMA as a result of this distinction.
Me: Having worked in this field for such a long time, what do you see as the top trends or hot topics in the PRO / ePRO space right now?
Diane: At least from my perspective, one of the most important areas regarding PRO and the FDA is the expansion of the PRO final guidance to cover other endpoints under SEALD’s consideration – observed outcomes (ObsRO) and clinician reported outcomes (ClinROs). This is going to have an impact on how we work in terms of needing to demonstrate content validity and measurement properties for these clinical outcome assessments moving beyond the current PRO focus.
From the ePRO perspective, it has now been 2 years since the ISPOR ePRO Task Force Report by Coons et. al. regarding paper to ePRO migration. Technology has moved fast in this area and this is now forcing us to think about how things have changed since the publishing of this paper. For example, we might now need to consider migrating from one ePRO platform to another and consider the differences between different ePRO modalities and not just paper vs. ePRO. There is now the ePRO Consortium organized by the C-Path institute and there are six ePRO vendors who are members and are now working together collaboratively to create standards and to provide updates and guidance in this area.
Me: So there is much more cross-vendor collaboration than in the past? Do we have a chance to come up with meaningful standards?
Diane: Absolutely. I think you can compare ePRO migration to the world of translations and linguistic validation. We are not constantly re-inventing the wheel as translations are re-usable between Sponsors and vendors. This is where we want to get to with ePRO migration as well, so that we are not constantly re-evaluating the same instruments and that they are effectively shared.
Me: My next question is more on the personal level. You founded Oxford Outcomes 14 years ago and it has now been acquired by ICON. Your colleagues have said very nice things about working with you and you are clearly still inspired and motivated to work in this space. What is your secret?
Diane: Well, we have always been about organic growth and over the period of 14 years, we have grown from just two of us to about 80 people at the time we were acquired. The key really has been about recruiting and working with the right people as well as being passionate about the work.
Me: I think it’s a job really well done. There is a very positive atmosphere around Oxford Outcomes and I have certainly enjoyed working with all of you. Thank you very much for the interview.
Best Regards,
Kai Langel
Senior Director, Technical Support, CRF Health
Posted by Chris Clancy on Tue, Nov 15, 2011 @ 09:50 AM
Clinovo hosted a truly informative panel discussion on the topic of adaptive clinical trials in Palo Alto,CA on November 2. It was a great forum to hear some very experienced clinical trial professionals share their experiences with a standing-room-only audience. In addition, it was a friendly environment to network with Bay Area colleagues before and after the panel discussion. The event was impressive enough to make me want to embark on my first visit to the blogosphere. Here goes…
During the panel session, the first thing that struck me was when the panelists explained that while the phrase “adaptive clinical trial” is a somewhat new bit of bio-pharma jargon, we all have been running trials for decades that fit this moniker – we just didn’t know it. It is also clear that these trials are increasing in number, and are helping to save millions for the biopharmaceutical industry.
From the FDA’s Draft Guidance: Adaptive Design Clinical Trials for Drugs and Biologics – February 2010
(Page 6) Definition and Concept of an Adaptive Design Clinical Trial
For the purposes of this guidance, an adaptive design clinical study is defined as a study that includes a prospectively planned opportunity for modification of one or more specified aspects of the study design and hypotheses based on analysis of data (usually interim data) from subjects in the study.
Analyses of the accumulating study data are performed at prospectively planned timepoints within the study, can be performed in a fully blinded manner or in an unblinded manner, and can occur with or without formal statistical hypothesis testing.
As a member of the eClinical community myself, I was also pleased to hear a common theme throughout the discussion, which was how great an impact technology was making on the ability to achieve the goals of adaptive trials. All of the panelists explained how EDC made it possible to have access to their data in a way that facilitated the type of decisions critical to adaptive studies. It was clear that ‘in the old days with paper’, it was very difficult, or impossible to make the type of mid-stream analysis and decisions which we now take for granted.
I was pleased to hear Oranee Daniels from Theravance mention that she could look at her ePRO data every morning if needed, and always know that she was looking at the most up-to-date information for making the crucial decisions associated with the adaptive design.
Don Kellerman from Map Pharmaceuticals also explained his experience with killing studies by determining futility through adaptive trial design. Don had a great sense of humor; he warned about giving him a compound to evaluate, since he has a proven track record for quickly finding futility.
All in all, I felt the event was a great success. I would certainly attend future sessions of the Silicon Valley BioTalk series. Kudos to Clinovo for putting it together and generating the interest for such a great turn out on a Wednesday evening. It attracted participants from all over the Bay Area, and beyond.
Best,
Chris Clancy
Director, Business Development, CRF Health
Posted by Kai Langel on Fri, Aug 19, 2011 @ 01:00 PM
Subject compliance is a hot topic in the industry right now. Some of the discussions out there can be a little misleading since ‘compliance’ means different things to different people / companies. Many eDiary vendors focus on a subject’s compliance with data entry, whereas Sponsors are more interested in the subject’s compliance with the study procedures, and clinicians are interested in the subject’s compliance with their medication / treatment.
I think it’s important that we define what we mean when we use the term because it can mean any of the above items. It is also important that we carefully define the reports and metrics collected in clinical studies. I see many studies attempt to come up with a single metric for ‘total compliance’, which includes a number of different things, like compliance with different medications, diary entry, etc. By doing this, they are actually hiding some important details. Individual metrics for individual items would most likely be much more useful. If a subject is having issues with their medication compliance, it should be addressed separately from their compliance with the entering of information into their eDiary.
CRF Health has been collecting eDiary compliance metrics for many years now and our compliance database is getting quite extensive and more useful the more data we collect. Every single CRF Health eDiary device automatically collects project-specific eDiary compliance for every subject. While it is sometimes difficult to define a reliable compliance metric for every study (OAB and other event-driven diaries are difficult, for example), there are some interesting trends that are starting to emerge:
Africa is the leading region with 94.5% compliance
Personally, I find this surprising. However, we only have limited amount of data for Africa – only 2% of our subjects are from Africa (n=1785). Africa is followed by Eastern Europe (92.8%) and Asia (92.1%).
Adult populations have the lowest compliance
This is another surprise, since most people would expect the ‘tech-savvy’ groups to be high on the list, but the statistics are almost the opposite. Infants (e.g. caregivers) lead with 93.9%, elderly subjects are close behind with 92.3%, and adults are well below the average with 88.5%.
Healthier populations perform better than sicker populations
This is not surprising at all. Our vaccine studies have 93.8% compliance with a standard deviation of 10.9, compared to 86.1% / 15.7 in Pain and 86.3% / 18.2 in COPD. What is interesting is that there are studies within some of the ‘difficult’ indications that have very good compliance and there are studies that are not doing so well. It proves that it is possible to get very good results even within the difficult indications. This actually raises some interesting questions – what are the key success factors in these studies and what can we learn from the studies that are not doing so well?
Compliance success factors
The space allotted for this blog post is not enough to discuss the details, but here’s my short list of what I personally consider to be the key points:
- Subject-focused eDiary design that will proactively guide the subject AND study sites through the study
- Efficient compliance management tools for sites and CRA’s – online reports, email notifications, etc
- Active study team from the Sponsor / CRO who will use the compliance monitoring tools and stay on top of everything
These are all generic concepts that can be adopted in virtually every study. There are obviously many protocol / disease specific best practices that can be utilized and study Sponsors should expect the eDiary vendor to be able to proactively guide them through the design process to ensure best results for each study. This is exactly what the CRF Health teams are used to doing – we actively use our existing experience and best practices and provide these recommendations to our Sponsors.
For those interested in the topic, subject compliance was discussed as the topic of our August 2011 webinar. Request access to the session by clicking this button:
Best Regards,
Kai Langel
Senior Director, Technical Support, CRF Health
Posted by Kai Langel on Wed, Jul 06, 2011 @ 04:00 PM
CRF Health has been very active recently by doing some usability research on our own. We have conducted workshops with patient focus groups as well as usability research and testing with different devices and indications. One of the most interesting and challenging groups we have worked with are patients with Parkinson’s Disease. We first did some PD studies a few years ago and got to understand some of the challenges these patients have to deal with as well as the research issues for clinical studies. The requirements for data collection in PD are the most demanding for the patients that I have seen in my 11 plus years of doing ePRO. This combined with the motor symptoms and the ON / OFF cycles the patients have to go through is not exactly an easy challenge for them. I took a bit of a personal interest to the matter after the FDA PRO guidance came out in 2009. I realized that paper data collection is no longer going to be a very reliable option for collecting this data and that efficient and easy-to-use solutions are needed. It wasn’t very clear in the beginning what those solutions might be and it took some Sponsor input, creative thinking and significant advances in device usability to come up with a new kind of user interface to collect this data.
The end result is quite interesting. Where previously it took several ‘clicks’ to enter symptom status data for a single 30-minute slot, the patients can now enter several time points at once, with just a single click each using a very visual and logical interface. There is minimal amount of text on the screen as the visual, iPhone-inspired icons are quite self-explanatory.
| Figure 1 - Improved and efficient interface for PD symptom status collection |
We have also benefitted greatly through collaboration with Dr. Robert Hauser, who is the creator of the original paper version for the ON / OFF diary that is so typical in PD research. Dr. Hauser has given us great insight into current PR research and medical issues and has been instrumental throughout the process. We have also tested the design with some other investigators in Europe. The feedback has been very positive and there have been great practical suggestions for improvement from the clinicians.
Most importantly, this design has been tested and well received by patients. Patients prefer the electronic version to paper, although some had initial reservations about their ability to use it. However, once they gave it a try, most found it easy to use. There were some patients who are simply in such an advanced stage of the disease that they commented they would not be able to use either method. Many patients, though, especially liked the fact that with the electronic version, they can easily change their answers and can avoid having to do any handwriting at all. In fact, the electronic version can be used with just the patient’s finger, which is an advantage to many of them.
It has been very interesting to work with the challenge of ePRO for Parkinson’s Disease patients and I’m quite pleased with the results so far. We hosted a webinar on the topic a couple of weeks ago with Dr. Hauser as a guest speaker. It’s all recorded in case you’re interested in learning more of the details of our work so far. Click this button to request access to the recorded webinar.
Best Regards,
Kai Langel
Senior Director, Technical Support, CRF Health