By Eirini Saloniki, Research Associate at Centre for Health Services Studies & Personal Social Services Research Unit, University of Kent,
Health preferences research is now a substantial field in itself, known not least for its contribution to assessing different health and care services by measuring the outcomes that people experience in using these services. Well-known and comprehensive indicators are used to measure these outcomes whereby people place different values on the different ways that services (can) impact their quality of life.

Preference studies have traditionally used face-to-face (paper-and-pencil or computer-assisted) interviews to gather such data. Despite their good completion rates and reliability, face-to-face interviews are costly and time-consuming. A shift towards internet surveys has been gaining popularity, allowing for accurate recording of response time and for targeting groups of respondents quickly at lower cost. Nevertheless, internet surveys have also been criticised, with poor data quality and difficulty in achieving sample representativeness being the main concerns. Can the advantages and disadvantages of the different methods of data collection be a source of variation in the results? Yes, not only as a result of how someone responds, but also because of who responds to each survey.

Several studies in environmental and health economics have compared preferences elicited from face-to-face interviews and internet surveys, but the evidence is mixed. The discrete choice experiment (DCE) technique was commonly used amongst the different studies to elicit preferences while the sampling process varied considerably. Another technique – best-worst scaling (BWS) – known for its lower cognitive burden compared to the DCE, has not been used to compare preferences elicited from different data collection methods. Nor have preferences been compared across the different methods in the context of long-term care.

In this paper, for the first time, we compare preferences elicited from face-to-face and internet surveys for the best-worst scaling (BWS) task using the Adult Social Care Outcomes Toolkit (ASCOT) service user measure. ASCOT measures social care-related quality of life (SCRQoL) across eight domains (accommodation cleanliness and comfort, safety, food and drink, personal care, control over daily life, social participation and involvement, dignity, occupation and employment), and has been recommended by NICE for use in the economic evaluation of social care services. This paper is part of a larger study (EXCELC) that aimed to establish relative preferences for care-related outcomes for people using long-term care.

The BWS experiment involving the ASCOT measure was included in a face-to-face (n=500) and an online (n=1001) survey and was completed by a sample targeted to be representative of the general population in England. Each respondent in both surveys was presented with a set of eight hypothetical scenarios. Each scenario contained eight attributes reflecting the eight ASCOT domains. In each scenario, the respondent was firstly asked to select the best (or most preferred) choice, with the selected choice being greyed out. The same process was repeated for the worst (or least preferred) choice, the second best, and second worst choices. Thus, each respondent made in total 32 choices.

We used a multinomial logit framework to analyse the data, ensuring that it is appropriate to pool the two datasets (internet and face-to-face). The initial pooled model assumed that there are no differences in observable characteristics between the two samples (i.e. no taste heterogeneity exists), but it included a parameter to control for scale differences between the two datasets. An additional (pooled) model controlled for differences in the sample composition of the two datasets as well as scale heterogeneity between different groups of respondents across the two datasets. All models took into account the repeated nature of the data.

The results from our first pooled model revealed several but small differences in preferences for SCRQoL across the two methods of data collection, with half of the coefficient differences compared (15 out of 30) found to be statistically significant at the 5% level. The number of significant differences compared reduced substantially to five when further controlling for observable and unobservable characteristics between the two samples. The limited (and small in value) significant differences were mostly at attribute levels indicating higher needs. A number of significant scale effects were also identified – for instance, participants were less certain when making their worst choices than when making their best choices which relates to framing effects reported in the preferences literature.

Overall, we find fairly similar preferences for SCRQoL between the two methods of data collection, suggesting that we can be confident enough in the internet results from a practical point of view. There is scope for future studies to not only explore a different sampling frame, but also consider providing sufficient clarifications to internet respondents in an attempt to minimise the level of uncertainty in the choice process for this group.

For a copy of the full paper please see: http://dx.doi.org/10.1007/s11136-019-02172-2

Authors: Assma Hajji, Judith Kieninger, Ruth Fulterer, Judith Litschauer and Birgit Trukeschitz (WU Vienna University of Economics and Business, Research Institute for Economics of Aging)

Over 400 interviews with Austrian home care service users were originally planned as part of the EXCELC project in 2016/2017. More than 600 have been carried out successfully. Data collection was supported by the Austrian Federal Ministry of Labour, Social Affairs, Health and Consumer Protection, by social care organisations and local authorities.

More than 60 trained interviewers took on the task of visiting people in need of care in their own homes to conduct interviews. About half of them gave us detailed written feedback describing their experiences and their impressions. This blog is based on unstructured accounts of their experiences. Quantitative analyses of the data collected through the standardised personal interviews can be expected in September 2018.

“Sharing a part of their lives”

Respondents freely voiced their hopes and fears relating to their situations. Our interviewers were able to gain insight into the respondents’ lives and their care situations both through the interviews and the conversations that often followed. One interviewer happily noted: “the ladies were very vivid in their descriptions, it felt like they were sharing a part of their lives with me”.

The term “in need of long-term care“, and the multitude of situations to which it can refer

Some long-term care service users are quite affected by cognitive decay, memory problems, or are depressed or afflicted by other mental conditions. Others, however, have an active mind but are trapped in a frail body. They, for example, follow technological innovations and keep up with digital means of communication – such as Mr. S, who likes to download the newest apps onto his smartphone, or Mrs. M, who regularly uses Skype to make phone calls to her grandchildren living abroad. Digital innovations aside, they also get creative in other areas when it comes to dealing with the physical constraints they are facing. Mrs. L, for instance, makes use of a cord and basket to bring groceries and mail up to her first-floor apartment.

The challenge of accepting one’s own need of care

The first important step – and often a challenge in itself – consists of coming to terms with one’s own need for care and in accepting help and assistance. Sometimes, this leads to a reversal of roles – as in Mrs. W’s case: she has spent most of her life caring for her physically disabled daughter but now finds herself in need of assistance because of cognitive impairments. While some people take a long time to come to terms with restrictions they face, others adapt to their new situation rather quickly. In the course of the interviews respondents discussed the process of adapting and adjusting to new circumstances. In many cases, these mechanisms are offered as an explanation as to why respondents are happy and doing well even in difficult situations. Our interviewers were often impressed with the respondent’s positive attitude and coping strategies.

Care at home – not enough assistance?

In many cases, our interviewers got the impression that the care services people received in their homes helped them a great deal – at least at first glance. Home care enables them to stay in their homes and help them overcome the obstacles and challenges of daily life. However, at second glance, it often became evident that the services provided only basic support and that some needs remained unmet.

As our interviewers have observed, it is the social needs of care recipients which are left unattended. Care service recipients feel clean, well-fed and safe, but seem to lack interpersonal relations and meaningful interaction. Visits from care workers are a welcome change but time is short and the required paperwork often ate up the little time left that could be used for some conversation.

The role of care workers and implications of recurring changes

Care workers are trusted and significant people in the lives of long-term care service users. This is especially true for persons living alone who don’t have many outside contacts – for them, visits from care workers can brighten up their entire day. On many occasions, interviewers have heard respondents talking about care workers using terms of endearment, speaking of their “fairy” or “wonderful Maria”.

However, recurring changes in personnel and uncertainties about the times of visit are often causes of discomfort or insecurity for the care service recipients. A great deal of flexibility and patience is needed on their part to adapt to new schedules and new caregivers, requiring them to explain over and over again what’s important and what to watch out for. This creates a lot of insecurity, especially for older people who might not be as able to adapt to changes and have a preference for clearly structured routines.

Care arrangements you can depend on – especially in the case of live-in carers

Some respondents with live-in caregivers have trusting and amicable relationships with them. Others, however, are less satisfied with their arrangements, voicing their despair and anger. Ideally, most care recipients would prefer a reliable, stable care setting. Finding a suitable live-in caregiver is difficult and full of obstacles, making the process difficult and tiresome. Moreover, even those who have found a live-in caregiver might still be faced with uncertainties, such as their care provider finding a more promising job and leaving them with no other option than to move into a nursing home.

Financial distress and its consequences

Financial strain, small pensions and a lack of means for taking up more care services are brought up and discussed. Thus, long-term care allowance (“Pflegegeld”) recipients became frustrated if applications for higher long-term care allowance had not been successful.

For some, lack of information is a legitimate problem…

Our interviewers noted that there tends to be a lack of information – many respondents don’t know about available long-term care services and ways of paying for them. Mr. P is one of those cases – he would like more social interaction in his daily life, but had never heard of visiting services or day care centres before the interview. Or Mrs. D, who was applying for care allowance for the first time and didn’t know who to turn to for assistance. There is substantial information available via the internet – but only a small share of those in need of care services have access to it or are web-savvy enough to use it for this purpose.

This has been a selection of interviewer impressions from interviews with long-term care recipients in Austria. Results from the statistical analyses will be available in September 2018.

By: Birgit Trukeschitz & Assma Hajji (WU Vienna University of Economics and Business, Research Institute for Economics of Aging)

The Austrian EXCELC team held two presentations at the 3^rd ATHEA (Austrian Health Economics Association) Conference in Vienna, both on preferences for long-term care outcomes in Austria:

• “Understanding best-worst experiments for calculating quality-of-life preferences in long-term care settings: How they work and what to consider” (“Hinter den Kulissen – Wie funktionieren Best-Worst Experimente für die Berechnung von erwarteten Präferenzen bei Pflegebedürftigkeit und welche Faktoren beeinflussen die Ergebnisse?“)Assma Hajji gave an overview of the method used, the experimental design and methodological issues. The data was collected using a best-worst experiment for calculating preferences for different social care-related quality of life domains using the ASCOT questionnaire.

• “Food over company or vice-versa? Results from a best-worst-analysis of expected quality-of-life preferences in long-term care settings for Austria” (“Gutes Essen oder nette Gesellschaft oder..? – Eine Best-Worst-Analyse zu den erwarteten Präferenzen bei Pflegebedürftigkeit in Österreich”)Birgit Trukeschitz presented preliminary results of the Austrian preference weights based on a general population sample, showing some clear differences in preference between various domains of social care-related quality of life.

Both presentations were well received and led to some insightful discussions on potential implications and next steps in the process.

by Lien Nguyen Researcher at the Centre for Health and Social Economics, National Insitute for Health and Welfare (THL)

Two EXCELC team members presented their works at the iHEA World Congress in Health Economics held in Boston on July 8–11, 2017. Eirini Saloniki presented early findings on differences in preferences elicited from internet and face-to-face surveys, based on data from a best-worst scaling experiment using the English Adult Social Care Outcomes Toolkit service user measure (ASCOT-S). Ismo Linnosmaa gave a talk on our preliminary work exploring whether general good health leads to better social care related quality of life. Both presentations were well received. We are using the feedback from the useful discussions to refine our work.

In addition to the iHEA Congress, the Finnish team submitted two written papers to the Nordic Health Economists’ Study Group Meeting (NHESG) held in Helsinki on August 23–25, 2017. Ismo Linnosmaa submitted the paper on the relationship between general good health and social care related quality of life and Lien Nguyen submitted a paper presenting Finnish population preferences for the ASCOT service user measure. The discussants for these written papers (Professor Jan Aben Olsen, University of Tromsø, and Associate Professor Line Bjørnskov Pedersen, University of Southern Denmark) provided detailed and useful feedback, which has helped to further refine our analysis. Many colleagues showed an interest in our work and are looking forward to us returning to a future study group meeting with our final results.

The iHEA (International Health Economic Association) World Congress in Health Economics is a biennial conference traditionally held in July. The NHESG (Nordic Health Economists’ Study Group) Meeting is an annual conference traditionally held in August, involving researchers mostly from Denmark, Finland, Iceland, Norway and Sweden. The number of researchers, including doctoral students that attend the Nordic conference each year is about 70–100. Written papers must be submitted to the conference organisers beforehand and 40 minutes are allotted to each paper for discussion, led by a discussant. The discussant will first summarise the paper and then give his/her comments on the paper, after which the paper’s authors can reply and the audience can join the discussion. 

By Eirini Saloniki, Research Associate at PSSRU & CHSS at the University of Kent.
It was nine years ago when I was given the opportunity to study for a year abroad at the University of Kent with the Erasmus programme. A fantastic and mind-opening experience – in fact, so fantastic that I returned to Kent not only to study for a PhD but now also working at the University. Who would have thought that I will live the Erasmus ‘dream’ again, this time as a staff member? Yes, the Erasmus programme is available for staff too…! I was eager to apply but I had to find a hosting institution first. It took a Skype meeting and “I suppose we talk about 2-4 weeks visit?” before I found myself applying to spend two weeks as a Visiting Researcher at the National Institute for Health and Welfare (THL) in Helsinki. The application process was fast and within a couple of weeks I was informed that I was successful. I put my excitement aside and focused on organising everything for the visit with the weather not being a concern as I was told that it will be over 10 degrees in May!

Nervous (as it was my first time in Helsinki) but at the same time ready to face the challenge, I spent my first day at the Institute trying to get used to the new routine and working environment, including having lunch at as early as 11:30am. Everyone was very welcome and it really felt like I was a member of the Centre for Health and Social Economics (within THL). The days to follow involved many meetings with health and social economists and late stays at the office (I must have lost track of time as it was so bright in the evening). I had the opportunity to discuss the projects we work on at the Centre for Health Services Studies and the Personal Social Services Research Unit with senior economists, and share best practice. I was also able to share my knowledge of how to control for taste and scale heterogeneity in preference elicitation for social care-related quality of life, and present some preliminary results from the work we are doing in Kent as part of the EXCELC project in an internal seminar. The seminar was well-attended with the audience showing real interest in the project. Importantly, I got to know all about the Finnish health and social care system, and its associated difficulties when it comes to data handling and interpretation.  This is an important consideration for the cross-national element of the project that I am involved in.
It was a great honour to be asked to co-write (with Professor Ismo Linnosmaa) a short piece about “Measuring effectiveness in social care: the present and future for researchers and policy-makers” which is to be featured in the THL newsletter, distributed across Finland.

Before I even realised, my time in Helsinki was over. This experience was invaluable and once again undoubtedly mind-opening. The interaction with senior economists working under a different research environment and the transfer of knowledge as well as discussions for future collaboration were beyond my expectations.

Would I do it again? ABSOLUTELY. Lots to learn in this visit but above all lots to gain. I wholeheartedly recommend the Erasmus programme to other researchers.

By Dr Laurie Batchelder, Research Officer at PSSRU Kent
Greetings from America! Just a quick update on how things went at ISPOR this week (International Society for Pharmacoeconomics and Outcomes Research – 22nd Annual International Meeting). I gave a talk on our preliminary work on the generation of preferences for the ASCOT-C in the English data, which was well received. I fielded a few questions about how we plan to compare these preferences for the ASCOT-C across countries, so I highlighted our plans for the cross-country analysis using a pooled model. Many people were also interested in when we plan to publish the ASCOT-C work, which is great news! In addition, my presentation had been selected as an ISPOR 22nd Annual International Meeting Research Presentation Award finalist!

I also presented our preliminary work on the comparison between face-to-face versus internet data for eliciting SCRQoL preferences using the ASCOT-S in the English dataset, which was also well-received.  Similarly many were very interested in this work and asked questions about the presentation of the Best Worst Scaling task & how this differed between the two modes. There was also a focus on looking at heterogeneity in a very detailed way (which we plan on doing). Again many people were very interested in ASCOT and when we plan to publish this work, which is great news!
In general, I found that there was a lot of discussion talking with others (economists, pharmacists, consultants) after the talk, at the poster, and even in some preference sessions that other methods, such as BWS is the “way forward” and the “next step” with eliciting preferences, given it’s less cognitively burdensome and that it provides more detailed information within a profile compared to TTO,  which I think is really promising.

Full poster and presentation references:

COMPARING INTERNET AND FACE-TO-FACE SURVEYS AS METHODS FOR ELICITING PREFERENCES FOR CARE-RELATED QUALITY OF LIFE: EVIDENCE FROM ENGLAND USING THE ASCOT SERVICE USER MEASURE
Saloniki E¹, Batchelder L¹, Malley J², Burge P³, Lu H³, Forder J^1
1University of Kent, Canterbury, UK, ²London School of Economics and Political Science, London, UK, ³RAND Europe, Cambridge, UK

CARER SOCIAL CARE-RELATED QUALITY OF LIFE OUTCOMES: ESTABLISHING PREFERENCE WEIGHTS FOR THE ADULT SOCIAL CARE OUTCOMES TOOLKIT FOR CARERS Batchelder L1, Saloniki E1, Malley J2, Burge P3, Lu H3, Linnosmaa I4, Trukeschitz B5, Forder J1 1University of Kent, Canterbury, UK, 2London School of Economics and Political Science, London, UK, 3RAND Europe, Cambridge, UK, 4National Institute for Health and Welfare (THL), Helsinki, Finland, 5Wu – Vienna University of Economics and Business, Vienna, Austria

CHESS are hosting a seminar on ‘Preference elicitation for social care-related quality of life: theory and practice’ on May 17 in the NCC buildings in Helsinki.
There will be presentations by members of the EXCELC team, including some early findings from the preference study.  The programme for the seminar can be downloaded here.