Taewook KIM

Taewook KIM

Curriculum Vitae

Contact: tw.kim at connect.ust.hk

© 2020

Love in Lyrics: An Exploration of Supporting Textual Manifestation of Affection in Social Messaging

jungsoo
Jung Soo Lee
Korea University

Abstract

Affectionate communication, the conveyance of closeness, care, and fondness for another, plays a key role in romantic relationships. While the pervasive use of digital technology for communication limits affectionate interaction through nonverbal cues – a major channel of expression in face-to-face settings, there have been few approaches which scaffold couples’ romantic text conversations. To bridge this gap, we propose a novel interactive system Lily which gives users inspirations to enrich their romantic expressions in text messaging. It first listens to users’ original input and then recommends romantic lyrics holding the closest meaning in real-time during chats with partners. After a three-day empirical study, participants who are real-life couples reported that they not only received useful cues from Lily in terms of how to polish their affectionate expressions, but also learnt to enrich the conversation with topics enlightened by its recommendations. Based on our findings, we finally provide several design considerations for actual deployment of such an application.


hkust


UI & Framework

lily

(A) is the initial user interface of our system. It shows where the recommendations would appear by displaying three blank oval shapes. (B) shows users’ typing input and three recommended lines of romantic song lyrics. (C) shows the randomized results of our system. Even if users type the same words, it will not always return the exact same lines.


framework

The overview of our system framework. (a) Once a user receives a message(M1) from his/her partner, (b) the user types some phrases(r1) to reply in the input line. (c) Then the system reads the phrases(r1). (d) The system then returns three corresponding lines(l1, l2, l3) of lyrics. (e) The lines(l1, l2, l3) are presented on the chat UI so that (f) the user can refer to them(l1, l2, l3) to refine their response(R1) before sending it.


Demo usage scenario

example

P3 and P4 are in a relationship. On the second day of the experiment, P4 asked P3 a question during the chat, ‘Which one do you prefer? short hair or long hair?’. In the context of typing a response to the this question, P3 was inspired by the recommendations provided. The figure illustrates the whole process which P3 followed to enrich his response.


Design Considerations

User engagement management

We need to consider how to engage users in the long run. First, the system can enrich and constantly update its example pool by incorporating a wide variety of language resources (e.g., movie lines), to reduce repetition in its suggestions. Second, the backend candidate ranking algorithm should include diversity – a “beyond accuracy” objective – into the optimization process while maintaining the semantic relevance of the output. Third, the system can expand its coverage of scenarios to assist verbal expressions in instrumental communication, since affectionate exchange only takes up a small portion of daily communication. Fourth, the system can provide topic recommendation in addition to expression suggestions, which is inspired by the positive side effect of Lily. Last but not least, the system can provide customized services tailored for different user needs.

Ethical implications

One participant (P8) prefers to hear genuine words from the partner because of the fear that people no longer mean what they say. It is thus critical to ensure that the suggested expressions would not alter users’ original intention. The system is meant to bring inspiration, not deception. In addition, users may sometimes fail to realize that the recommendation they take is an inappropriate way to express what they mean in a certain context, especially when unfamiliar words or phrases are involved. Another thing to note is that, “autocompletion” (or “autocorrection”) may not be a plausible feature to adopt in systems like Lily because it reduces meaningful user effort on communication which is considered important investment in relationship building.

Privacy concerns

The system Lily has to listen to users’ input to return semantically similar but more affectionate expressions. In other words, the system monitors what users are typing consistently, which may lead to privacy concerns. It feels like having an unknown third party listening to their conversation; but to our surprise, participants all took the experience quite positively. However, still, designers need to take effective measures to protect user privacy if launching such a system in reality. Note that these privacy measures may entail usability problems such as space consumption and response latency. System designers are supposed to consider the different trade-offs.


BibTex

@article{Kim:2019,
 author = {Kim, Taewook and Lee, Jungsoo and Peng, Zhenhui and Ma, Xiaojuan},
 title = {Love in Lyrics: An Exploration of Supporting Textual Manifestation of Affection in Social Messaging},
 journal = {Proc. ACM Hum.-Comput. Interact.},
 issue_date = {November 2019},
 volume = {3},
 number = {CSCW},
 month = nov,
 year = {2019},
 issn = {2573-0142},
 pages = {},
 articleno = {79},
 numpages = {27},
 url = {https://doi.org/10.1145/3359181},
 doi = {10.1145/3359181},
 acmid = {3359181},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {Affectionate communication, Expression, Interpersonal communication, Lyrics, Recommendation, Text messaging},
} 



We thank Ziming Wu and Huan Wei for valuable input.
This work is supported by the Research Grants Council of the Hong Kong Special Administrative Region, China under Grant No. C6030-18G.