[Verzerrung durch Erinnerungen in der Methode des Ecological Momentary Assessments – eine Pilotstudie]

Patricia Fürstenberg ^1,2
Inga Holube ¹
Dina Lelic ³
 Nadja Schinkel-Bielefeld <sup>2

1</sup> Institut für Hörtechnik und Audiologie, Jade Hochschule, Oldenburg, Germany
² WS Audiology, Erlangen, Germany
³ WS Audiology, Lynge, Denmark

Zusammenfassung

Ecological Momentary Assessment (EMA) ermöglicht die Bewertung aktueller Hörsituationen in Echtzeit. Dies kann für die Hörsystemanpassung von Nutzen sein, indem möglichst viele alltägliche Situationen abgefragt werden und die Hörsysteme aufgrund der individuellen Bewertungen feiner eingestellt werden. Da jedoch nicht alle Situationen immer in dem Moment beurteilt werden können, weil die Sicherheit nicht gewährleistet ist oder es die Höflichkeit verbietet, sollten Situationen auch rückblickend bewertet werden können. Diese Studie befasst sich mit der Frage, inwieweit eine retrospektive Beurteilung von Ereignissen die zu beurteilende Situation valide wiedergibt. Zu diesem Zweck wurden zehn Teilnehmende mit Hörgeräten und einem EMA-System für zwei Wochen ausgestattet. Aufgabe war es, verschiedene Situationen zeitnah und nochmals verzögert nach 30 bzw. 60 Minuten zu bewerten. Zusammenfassend kann festgestellt werden, dass Situationen möglichst zeitnah bewertet werden sollten. Wenn dies nicht möglich ist, dann sollte der Zeitrahmen 30 Minuten nicht überschreiten, da seltene Situationen besonders von einer kurzen Verzögerung profitieren. Bei der Verkürzung der Verzögerung von 60 auf 30 Minuten reduzierte sich die Abweichung der Antworten um 8%-Punkte. Zudem könnten Stimmung und Stress einen größeren Zusammenhang zur Bewertung nach einer Verzögerung von 60 Minuten aufweisen. Die Selbsteinschätzung, wie gut das Ereignis abgerufen werden konnte, korreliert mit der Zuverlässigkeit der getätigten Antworten.

---

---

Introduction

Hearing loss (HL) can be treated with hearing aids (HAs). The sound and speech perceptions of people with HL vary due to different types of HL, cognitive abilities, and age [1].Therefore, different requirements for HAs are necessary to improve satisfaction. To achieve this goal, German health insurance companies recommend testing HAs in real life [2], [3]. Questionnaires, such as the Abbreviated Profile of Hearing Aid Benefit (APHAB [4]), are used as part of the fitting process to assess the daily life situations, but they have disadvantages as they are retrospective assessments and address situations that may not occur in the user’s life.

Ecological Momentary Assessment (EMA) is a method used to evaluate situations at different times, in the exact moment, to reduce recall bias [5]. An advantage of EMA in audiology is the ability to gather information about the everyday life of HA users, which can improve the HA fitting [6], [7]. The fitting is validated in real-life situations through subjective assessments, rather than standardized measurements in laboratory conditions. Previous EMA studies in audiology have shown that data loss can occur due to prompting in certain situations that are considered inappropriate [7], [8], [9]. As a result, inclusion of somewhat retrospective assessments was recommended. Responses have typically been accepted within up to 30 minutes after the prompt, but it is unclear whether this delay affects the accuracy of responses.

Retrospective assessments may be biased by memory capacity and cognitive abilities. Research has shown that negative aspects are often rated more negatively with delay, while positive aspects are reported less frequently [10]. Additionally, participants may describe retrospective situations based on their current state of mind rather than their actual thoughts or feelings. Memory tests, such as the Verbal Learning and Memory test (VLMT [11]), can be used to determine an individual’s ability to remember.

This study investigated the impact of retrospective assessments compared to momentary ratings. The influence of time was examined with delays of 30 and 60 minutes. In addition to the effect of time, this study examined whether memory tests, situational differences, and mood are related to the number of questions answered identically in momentary and delayed assessments.

Research questions

RQ1: Is there a memory bias in assessing a situation after 30 or 60 minutes?

RQ2: Is the memory bias of individual participants related to their ability to remember as measured by the VLMT?

RQ3: How is memory bias affected by type and occurrence of the situation?

RQ4: Does the memory bias depend on the current mood and stress level?

RQ5: Is the memory bias related to the self-reported difficulties to remember?

Methods

Ten German-speaking experienced bilateral HA users participated in this study (nine male, one female). The participants had an average age of 72.5 years (range 44–82 years). Their average hearing aid experience was 10.5 years (range 0.5–41 years). The average hearing threshold across both ears for the frequencies 0.5, 1, 2, and 4 kHz (referred to as PTA4) was 41.5 dB HL (with a standard deviation of 12 dB). The study was approved by the ethics committee of the Carl-von-Ossietzky University Oldenburg (Drs.EK/2022/065-01).

Each participant was fitted with Signia Pure 7AX HAs with closed coupling and an M-powered receiver. The HAs were fitted according to the NAL-NL2 rationale based on the in-situ audiogram provided by the Connex fitting software (Version 9.11.0.63). If needed, fine-tuning was performed.

The participants were given a Samsung Galaxy S20 FE 5G smartphone (Android 13) with an EMA application pre-installed, which was connected to the HAs via Bluetooth. This allowed both, the smartphone and the HAs, to prompt and collect objective HA-related data. The WS Audiology internal EMA application included an adaptive questionnaire designed for the study purpose. The user manual for this study describing the EMA application can be downloaded from https://osf.io/nq8jp/.

The initial appointment focused on determining the hearing level, memory capacity, and fitting the HAs, as well as introducing the EMA application. The memory test consisted of the VLMT. To avoid an influence of hearing loss on the VLMT results, a visual version [12] was used with a presentation of 15 words for two seconds each, which had to be recalled immediately after the presentation and after a delay of half an hour.

The participants were prompted four times per day to rate their current situation and then again after a delay of either 30 or 60 minutes, over a two-week period, in their daily environments. During the field trial, each week was completed with one delay using a cross-over design. The adaptive questionnaire included between 12 and 26 questions and asked about satisfaction, speech understanding, noise perception, mood, and stress. In addition to the field trial, the participants were asked to rate three controlled situations: listening to church music, having a conversation while walking outside in the woods, and talking at the coffee machine in the mostly crowded entrance hall (later shortened to Church, Woods, and Coffee). The church music was played in the ‘Wonderful Sound Lab’ at WS Audiology, a laboratory that can modify the acoustic environment by adjusting the reverberation time [13]. At the first appointment, participants were asked to rate these controlled situations in the moment and again with a delay of 60 minutes. This retrospective assessments could be completed at home. At the second appointment, there was a delay of 30 minutes between momentary and retrospective assessment, during which the participants completed a feedback questionnaire about the study.

To investigate the impact of time, the momentary and delayed assessments were compared, and the percentage of inconsistent answers was calculated to account for the different length of the questionnaires due to the adaptation process. The percentage was built on the number of different responses for the same questions, not on how strongly the responses differed.

For the controlled assessments, data were analyzed using Wilcoxon signed rank test to assess whether there was an effect of delay and situation on response consistency. The field data were analysed by conducting three repeated-measures analyses of variance (RM-ANOVA) with response consistency as the dependent variable. First, 1-way RM-ANOVA with delay (30-minute or 60-minute) as the independent variable was conducted. Then, 2-way RM-ANOVA with delay and situation occurrence as the independent variables and a 2-way RM-ANOVA with delay and type of situation as the independent variables were conducted. Post hoc tests, such as Wilcoxon signed rank test, were performed when necessary. Spearman’s correlation was conducted to assess relationships between the number of inconsistent responses and a) mood, b) stress level, and c) self-reported difficulty to remember. For both, the controlled assessments and the field trial, Pearson’s linear correlation analyses were done to assess whether VLMT results and response consistency were related.

Results

535 pairs of questionnaires in the field trial were used for the analyses, 294 for the 30-minutes delay and 241 for the 60-minute delay.

RQ1: The findings indicate that there were no significant differences between the two delays, only situational differences in the controlled environment (see Figure 1 [Fig. 1], Church-Coffee: p_30min=0.039, p_60min=0.012, Woods-Coffee: p_30min=0.031, p_60min=0.031). In the field trial, the median inconsistency was generally over 13%, suggesting an impact of time in retrospective assessments. The reduced delay did not result in less inconsistency (F(1, 515)=2.927, p=0.088).

RQ2: For the relation between memory bias and VLMT results, a negative correlation was expected: the higher the result of the VLMT, the fewer differences between momentary and retrospective assessments. However, no such correlation was found, neither in the controlled assessment (r_30min(17)=–0.005, p=0.980; r_60min(21)=–0.188, p=0.330) nor in the field trial (r_30min(14)=0.008, p=0.982; r_60min(14)=0.174, p=0.631).

RQ3: In the field trial, investigations of the relationship between situation and occurrence to remember the situation yielded mixed results. The type of situation itself had an impact on the inconsistency of answers (F(6,512)=12.69, p<0.001), but no significant relation was found when considering the interaction with the delay (F(6,512)=1.173, p=0.319). While it appears that the occurrence of situation may impact the ability to accurately recall it (see Figure 2 [Fig. 2]), the ANOVA analysis did not support this finding as the interaction between occurrence and delay was not significant (F(3,518)=2.379, p=0.070). For less frequent situations, the inconsistency of 20.0% for a delay of 60 minutes was reduced to 11.8% by shortening the delay to 30 minutes.

RQ4: In the field trial, the retrospective assessments were significantly correlated with mood and stress levels for a delay of 60 minutes (Mood: r(54)=0.160, p=0.012; Stress: r(53)=0.168, p=0.007). That is, if a person was experiencing negative emotions or elevated stress levels, the number of inconsistently answered questions increased.

RQ5: In the field trial, there was a correlation between the self-reported ability to recall past situations and response consistency after a delay of 30 minutes (r(45)=0.175, p=0.002) as well as a delay of 60 minutes (r(53)=0.277, p<0.001). The inconsistency of answers increased with a decrease in the ability to remember (see Figure 3 [Fig. 3]).

Discussion

Overall, the results show a trend towards an effect of memory bias on EMA responses. The overall effect on consistent responses is not significant between a delay of 30 and 60 minutes. However, the fact that in the field trial the median inconsistency was over 13% due to delay and considering that mood and stress may play a role in how consistent people are in their responses after 60-minute delay, indicates that a restriction of the delay for retrospective assessments is beneficial. Especially, less frequent situations benefit from a shorter delay, as the deviation in responses is lower by 8% in 30-minute delay assessments compared to 60-minute delay assessments. Nonetheless, there may be situations where it is more important to have an assessment than to adhere to the recommended delay. Responses for the elapsed time and the self-reported ability to remember a past situation could provide sufficient information on the reliability of the assessment.

This study design required repetition of previously assessed situations for comparison. As a result, participants may remember less of the past situation and more of their previous responses. In past EMA studies, some participants spontaneously reported to mentally answer the questionnaire in the situation and then type in the responses afterwards. While it is unknown how many participants employ this strategy, it would resemble the design used in this study.

During the controlled assessment, it was observed that two participants had the correct situation in mind, i.e. did not confuse the three controlled situations, as evidenced by their description of the situation in their own words while filling out the questionnaire. However, they still gave different answers when asked about the listening situation. For example, for the situation ‘listening to church music’, the momentary situation was described as either ‘listening live’ or ‘listening via media’, while the retrospective situation was described as ‘having a conversation’. It is unclear why this inconsistency occurred. These aspects require further investigation.

Study considerations

The study was conducted with only ten participants, all experienced HA users mostly satisfied with their fittings. In addition, only two delays without cross-over design of the controlled assessments were used. No training with the questionnaires was conducted before the assessment of the controlled situations. Future studies with more participants of varying HA experiences and shorter delays are warranted to learn the optimal delay that should be allowed in EMA studies.

Conclusion

In conclusion, it is recommended to keep the delay for valid retrospective assessments as short as possible. If this is not possible, the delay should not exceed 60 minutes, as emotions are more likely to influence the results beyond this point. Shorter delays are particularly beneficial for less frequent situations and reduce the bias by around 8%. The self-reported memory ability is an indicator for reliable retrospective assessments. Therefore, HA users participating in EMA studies should be given the opportunity to decide whether they feel confident in assessing a situation retrospectively or whether they are unsure of their ability to correctly recall the situation and choose not to answer.

Notes

Conference presentation

This contribution was presented at the 26^th Annual Conference of the German Society of Audiology and published as an abstract [14].

Competing interests

Dina Lelic and Nadja Schinkel-Bielefeld are employees of WS Audiology. Patricia Fürstenberg was an intern and bachelor student at WS Audiology when performing this study. This work is based on a bachelor thesis for Jade University of Applied Science in Oldenburg.

---

References

[1] Souza P. Speech Perception and Hearing Aids. In: Popelka GR, Moore BCJ, Fay RR, Popper AN, editors. Hearing Aids. Cham: Springer International Publishing; 2016. pp. 151-80. (Springer Handbook of Auditory Research; 56). DOI: 10.1007/978-3-319-33036-5_6
[2] Verband der Ersatzkassen e.V. (vdek). Vertrag zur Versorgung der Ersatzkassenversicherten mit Hörsystemen zum 01.01.2023 - Rahmenvertrag zur Versorgung mit Hörsystemen zwischen biha und vdek, Fassung nach Protokollnotiz vom 15.06.2023. Verband der Ersatzkassen e.V.; 2023 [cited 2024 Jan 21]. Available from: https://www.vdek.com/vertragspartner/hilfsmittel/hoerhilfen.html
[3] BKK Landesverband Mitte. Vertrag zur bundesweiten Hörsystemeversorgung BKK-Versicherter durch vertragsgebundene Hörgeräteakustiker zwischen den BKK LVn und der biha vom 25.04.2023 – Vertragsfassung gültig ab 01.06.2023. Betriebskrankenkasse Landesverbände; 2023 [cited 2024 Jan 22]. Available from: https://www.bkkmitte.de/krankenversicherung-und-pflege/hilfsmittelversorgung/hoergeraeteversorgung.html
[4] Cox RM, Alexander GC. The abbreviated profile of hearing aid benefit. Ear Hear. 1995 Apr;16(2):176-86. DOI: 10.1097/00003446-199504000-00005
[5] Shiffman S, Stone AA, Hufford MR. Ecological momentary assessment. Annu Rev Clin Psychol. 2008;4:1-32. DOI: 10.1146/annurev.clinpsy.3.022806.091415
[6] Holube I, von Gablenz P, Bitzer J. Ecological Momentary Assessment in Hearing Research: Current State, Challenges, and Future Directions. Ear Hear. 2020 Nov/Dec;41(Suppl 1):79S-90S. DOI: 10.1097/AUD.0000000000000934
[7] Timmer BHB, Hickson L, Launer S. Ecological Momentary Assessment: Feasibility, Construct Validity, and Future Applications. Am J Audiol. 2017 Oct 12;26(3S):436-42. DOI: 10.1044/2017_AJA-16-0126
[8] Schinkel-Bielefeld N, Kunz P, Zutz A, Buder B. Evaluation of Hearing Aids in Everyday Life Using Ecological Momentary Assessment: What Situations Are We Missing? Am J Audiol. 2020 Sep 18;29(3S):591-609. DOI: 10.1044/2020_AJA-19-00075
[9] Galvez G, Turbin MB, Thielman EJ, Istvan JA, Andrews JA, Henry JA. Feasibility of ecological momentary assessment of hearing difficulties encountered by hearing aid users. Ear Hear. 2012 Jul-Aug;33(4):497-507. DOI: 10.1097/AUD.0b013e3182498c41
[10] Eisele G, Vachon H, Myin-Germeys I, Viechtbauer W. Reported Affect Changes as a Function of Response Delay: Findings From a Pooled Dataset of Nine Experience Sampling Studies. Front Psychol. 2021 Feb 26;12:580684. DOI: 10.3389/fpsyg.2021.580684
[11] Helmstaedter C, Durwen HF. VLMT: Verbaler Lern- und Merkfähigkeitstest. Ein praktikables und differenziertes Instrumentarium zur Prüfung der verbalen Gedächtnisleistungen [The Verbal Learning and Retention Test. A useful and differentiated tool in evaluating verbal memory performance]. Schweiz Arch Neurol Psychiatr (1985). 1990;141(1):21-30.
[12] Meister H, Schreitmüller S, Ortmann M, Rählmann S, Walger M. Effects of Hearing Loss and Cognitive Load on Speech Recognition with Competing Talkers. Front Psychol. 2016 Mar 4;7:301. DOI: 10.3389/fpsyg.2016.00301
[13] Holland R, Barthel A, Berwick N, Bilert S. “The Wonderful Sound Lab” – An immersive communication laboratory for hearing aid development. In: Deutsche Gesellschaft für Akustik e.V. (DEGA), editor. Fortschritte der Akustik - DAGA 2024. 50. Jahrestagung für Akustik. Hannover, 18.- 21.03.2024. Berlin: DEGA e.V.; 2024. p. 210-3. Available from: https://pub.dega-akustik.de/DAGA_2024/
[14] Fürstenberg P, Holube I, Lelic D, Schinkel-Bielefeld N. Memory bias in Ecological Momentary Assessment. In: Deutsche Gesellschaft für Audiologie e.V., editor. 26. Jahrestagung der Deutschen Gesellschaft für Audiologie. Aalen, 06.-08.03.2024. Düsseldorf: German Medical Science GMS Publishing House; 2024. Doc190. DOI: 10.3205/24dga190