Evaluating User Perceptions and Workflow Preferences in AI-Assisted Handwriting-to-Digital Note Transitions

Beiqiao Hu & Alicia Morgan

Carleton University, Ottawa, Ontario, Canada

BeiqiaoHu@cmail.carleton.ca • AliciaMorgan@cmail.carleton.ca

Abstract

As analog and digital workflows increasingly intersect, artificial intelligence (AI) presents new opportunities to enhance the note-taking experience. This study explores how users perceive the role of AI in facilitating seamless transitions between handwritten and digital note-taking.

Participants interacted with a mock interface simulating AI features like structured transcription, contextual tagging, and workflow integration.
We examined user expectations, trust, preferences, and perceived value of these AI features.
Findings aim to inform the design of intuitive, context-aware systems for efficient knowledge management.

CCS Concepts

Human-centered computing

User studies; HCI theory

Information systems

Contextual search

Computing methodologies

Artificial intelligence

Keywords

Context-aware systems Human-AI interaction Note-taking Hybrid workflows Handwriting recognition

1. Introduction

The Modern Note-Taking Dilemma

Valuable handwritten insights often remain siloed or require laborious manual transfer, leading to friction and inefficiency.

Handwriting Strengths

  • Brainstorming & learning
  • Focused engagement
  • Flexibility & cognitive benefits

Digital Platform Strengths

  • Efficiency & searchability
  • Organization & collaboration
  • Indispensable in modern workflows

Bridging the Gap: Current Shortcomings

Manual re-typing, photo scans, or basic Optical Character Recognition (OCR) often fail to preserve structural richness or capture context.

AI: A Potential Bridge

AI offers capabilities beyond simple transcription:

  • Analyze structure & summarize key points
  • Apply relevant tags & link notes to context
  • Transform static notes into dynamic knowledge assets

Research Question

How do potential users perceive the integration of context-aware AI in transforming handwritten notes into digital content, and what key factors influence their preferred workflows?

2. Related Work

Cognitive Benefits of Handwriting

  • Improves conceptual understanding [10].
  • Better memory retention compared to typing.
  • Activates broader neural networks (EEG studies) [19].

Context-Aware Systems

These systems use contextual info (time, location, activity) to adapt behavior. Early work by Schilit et al. [14] introduced mobile systems adapting to user location/activity.

Our study explores AI interpreting context to improve analog-to-digital transitions.

Technical Advances & Emerging Tools

  • InkSight [9]: Converts offline handwriting to structured digital notes, retaining layout.
  • Inkeraction [15]: Real-time AI interactions with ink.
  • GazeNoter [18]: Gaze-based selection of AI suggestions in AR note-taking.

Research Gap

Most systems emphasize performance, with less attention to user perceptions, preferences, and trust in AI-augmented workflows. This study aims to contribute a user-centered perspective.

3. Methodology

Approach

Mixed methods qualitative approach: surveys, semi-structured interviews, and interactive Wizard of Oz (WoZ) prototype sessions.

WoZ simulates system functionality via researcher intervention for realistic AI capability representation without full development.

3.1 Participants (N=14 Survey, N=10 Interview/Prototype)

We recruited 14 participants via university networks and social media. All participants were university-educated and regular users of both handwritten and digital notes. Of these 14 participants, 10 also participated in interviews and prototype testing sessions.

Participant Demographics (N=14)

Age Groups:

18–24: 21.4% (3)
25–34: 35.7% (5)
35–44: 42.9% (6)

Gender:

Man: 42.9% (6)
Woman: 57.1% (8)

Education:

Bachelor: 42.9% (6)
Graduate: 57.1% (8)

3.2 Prototype

Low-fidelity WoZ prototype simulating AI-driven note-taking. Output included OCR text and context-aware summaries (structured formatting, highlighting, context integration).

Figure 1: Comparison of pre-made and instantly generated outputs
Figure 1: Comparison of pre-made (left) and instantly generated (right) outputs.

Prototype evolved from pre-made to live-generated outputs (using Claude 3.7 Sonnet, Gemini 2.5 Pro) after P3 for better personalization and richer feedback.

Figure 2: Overview of prototype interaction
Figure 2: Overview of prototype interaction (Note-taking task with Dynamicland video).

3.3 Procedure (approx. 60 mins)

  1. Consent & Pre-study Questionnaire (Demographics, practices, attitudes).
  2. Initial Interview (Typical methods, challenges, AI expectations).
  3. Note-Taking Task (Watch 6-min YouTube video, take handwritten notes).
  4. Prototype Interaction with Think-Aloud (Review 3 digitized versions):
    • Photo Scan
    • OCR Text
    • Context-Aware Output (AI-enhanced)
    Figure 3: Three digitized versions of notes
    Figure 3: Participants reviewed three digitized versions: Photo scan, OCR text, Context-aware output.
  5. Prototype Explanation (Standardized AI-enhanced examples shown).
    Figure 4: Prototype explanations
    Figure 4: Participants reviewed prototype explanations to compare with personalized outputs.
  6. Concluding Interview & Wrap-up (Overall impressions, concerns, desired features).

3.4 Data Analysis

  • Survey Data (N=14): Descriptive statistics, qualitative analysis of open responses.
  • Interview Recordings (N=10): Transcribed verbatim, thematic analysis [1]. Two researchers independently coded.

4. Results

Integrating survey findings (N=14) and qualitative thematic analysis (N=10 interviews).

4.1 Handwriting versus digital: Complementary tools

The Value of Both Handwriting and Digital Tools

Key Finding

The participants clearly valued both handwriting and digital note taking, typically integrating both into a hybrid workflow. Handwriting remained essential due to its cognitive engagement and retention benefits, while participants universally relied on digital tools for their efficiency, searchability, and organizational advantages.

Why Participants Value Each Modality

Handwriting

Cognitive Engagement & Creative Flexibility

Primary Benefits:

Cognitive processing (5/10):

Enhanced thinking and retention

Layout flexibility (4/10):

Non-linear structures, diagrams, sketches

Tactile engagement (3/10):

Physical connection to content

Social appropriateness (2/10):

Context-dependent politeness

Participant Perspectives:

"I would think about it... reforming it." - P1

"translating knowledge into the brain." - P11

Digital Tools

Efficiency & Organization (Universal Adoption: 10/10)

Core Advantages:

Speed & efficiency (4/10):

Fast content creation and editing

Searchability (4/10):

Easy retrieval and organization

Accessibility (3/10):

Cross-platform access and sharing

Rich integration (2/10):

Multimedia and hyperlinks

Current Usage Patterns:

Survey results show extensive digital use (e.g., Word/Pages 10/14, Apple Notes 8/14). Participants universally relied on digital tools for their efficiency, searchability, and organizational advantages (10/10) (see Figure 5).

Current Digital Tool Usage Patterns

Figure 5: Digital tools used for note-taking

Figure 5: Survey results showing diverse digital tool usage among participants (N=14), highlighting the fragmented nature of current workflows.

4.2 Challenges in Hybrid Workflows

Significant friction reported during handwritten to digital transition

Time-intensive manual transcription

"Effortful" (P1), "frustrating" (P6)

(8/10 interviews, 14/14 survey)

Legibility/OCR issues

Accuracy & usability problems

(5/10 interviews, 10/14 survey)

Loss of spatial structure/formatting

Linear transcription lost "structure of idea building" (P5)

(4/10 interviews, 9/14 survey)

Tool fragmentation

Notes scattered across apps

(3/10 interviews, 6/14 survey)

Challenges

Figure 6: Challenges in note transitions

Figure 6: Challenges in hybrid note transitions (N=14)

Methods

Figure 7: Methods of transitioning notes

Figure 7: Methods used for note transitions (N=14)

Satisfaction

Figure 8: Participant satisfaction

Figure 8: Satisfaction with current process (12/14 neutral or dissatisfied)

4.3 AI-Enhanced Notes: Opportunities & Expectations

AI prototype received positive feedback (5/10 preferred AI version)

Participant Feedback

"really good" - P6

"much clearer than my notes" - P11

Key Expectations for AI

High precision handwriting recognition

(6/10 participants)

Preservation of layout, diagrams, hierarchy

(6/10 participants)

Contextual improvements (tags, links, summaries)

(7/10 participants)

Seamless integration into existing workflows

(4/10 participants)

4.4 User Control & Augmentation

AI should augment, not replace human cognition

Easy editing & verification of AI content

(3/10 participants)

Clear differentiation of original vs. AI

Mentioned by P3

Preserve user's original emphasis

(3/10 participants)

4.5 Adoption Barriers & Concerns

Key obstacles to AI adoption in note-taking workflows

High Priority Concerns

Privacy & Data Security

Concerns about data usage, security, transparency (P3, P9)

8/10 participants
Accuracy & Trust

Fear of misinterpretations needing constant verification

6/10 participants

Secondary Concerns

Cognitive Impacts

Reduced engagement, degraded learning from over-reliance

5/10 participants
Practical Barriers

Cost, usability, habit change, integration difficulties

3/10 participants each

Critical Adoption Insight

While participants see potential in AI-assisted note-taking, adoption success hinges on addressing fundamental concerns about privacy, accuracy, and cognitive impact. Trust-building through transparency and user control will be essential for meaningful adoption.

Summary of Key Interview Findings

(N=10 participants)

8/10

Value of handwriting

10/10

Preference for digital efficiency

8/10

Workflow friction

6/10

Accuracy concerns (AI/OCR)

7/10

Contextual enhancement desire

5/10

User control over AI

8/10

Privacy & data concerns

5/10

Cognitive impact concerns

3-4/10

Practical barriers

5. Discussion

Key Findings at a Glance

Cautious Optimism

Privacy Concerns

Structure Value

User Control

Cautious Optimism: AI can reduce friction, but privacy, accuracy, and cognitive impact are key concerns.

Research Alignment: Findings align with research on challenges in handwriting/digital bridging (loss of structure, tedious transcription).

Structure Preservation: Strong value placed on retaining spatial/semantic structure of handwritten notes, which traditional OCR fails to capture.

Privacy & Control: Privacy, trust, and user control are critical for acceptance. Users wary of cognitive impacts (reduced learning).

Methodological Notes: WoZ effective but has limitations. Modest sample size (N=10 interviews), mainly graduate students.

Key Design Implications 6 principles

Robust Systems

Prioritize structure recognition (text, layout, diagrams) and accuracy.

Seamless Integration

Embed AI features in popular note-taking platforms.

User Control

Intuitive options for review, editing, customization of AI output.

Transparency

Clear data handling practices, opt-in/out for data usage.

Augmentation, Not Replacement

Enhance user cognitive processes, retain user voice.

Contextual Linking

Capture and link relevant contextual information automatically.

Future Research Directions

Technical Capabilities

Advance recognition systems for complex note elements including diagrams, non-linear structures, and multi-modal content integration.

High Priority

AI Features

Explore nuanced AI capabilities like contextual summarization, intelligent tagging, and multi-modal integration methods that preserve note context.

Exploration Phase

Longitudinal Studies

Conduct long-term research on user adaptation patterns, learning outcomes, and cognitive impacts of AI-assisted note-taking over extended periods.

Long-term

Diverse User Groups

Expand research beyond academic settings to include diverse professional groups with varying note-taking practices, needs, and workflow patterns.

Population Study

Ethical Frameworks & Privacy

Develop comprehensive ethical frameworks and privacy-preserving methodologies for sensitive user data handling. Establish trust-building mechanisms and transparent data practices in AI note systems.

Cross-cutting Concern

6. Conclusion

Study underscores lasting value of handwriting and efficiency of digital tools, revealing demand for seamless integration.

Context-aware AI offers a compelling bridge via structure-preserving transcription and contextual enrichment.

Meaningful adoption depends on aligning with user expectations: accuracy, control, workflow compatibility, data privacy.

AI should be a supportive augmentation, enhancing user agency and preserving handwritten note intentionality.

Future tools must prioritize transparency, customization, and fluid integration for a user-centered hybrid note-taking ecosystem.

Acknowledgments

We acknowledge the use of large language models (Gemini 2.5 Pro, Claude 3.7 Sonnet) for assistance with processing interview transcriptions and providing initial suggestions for draft improvement and thematic analysis.

How to Cite This Work:

Hu, B., & Morgan, A. (2025). Evaluating User Perceptions and Workflow Preferences in AI-Assisted Handwriting-to-Digital Note Transitions. Carleton University, Ottawa, Ontario, Canada.

References

  • [1] Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
  • [2] Nils Dahlbäck, Arne Jönsson, and Lars Ahrenberg. 1993. Wizard of Oz studies: why and how. In Proceedings of the 1st international conference on Intelligent user interfaces. 193–200.
  • [3] Anind K Dey. 2001. Understanding and using context. Personal and ubiquitous computing 5 (2001), 4–7.
  • [4] @dynamicland1784. [n. d.]. Dynamicland. https://youtu.be/5Q9r-AEzRMA?si=WRAj3zZ2I-CHnlLQ
  • [5] Xianzhe Fan, Zihan Wu, Chun Yu, Fenggui Rao, Weinan Shi, and Teng Tu. 2024. ContextCam: Bridging Context Awareness with Creative Human-AI Image Co-Creation. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–17.
  • [6] Nuwan Janaka, Runze Cai, Shengdong Zhao, and David Hsu. 2024. Demonstrating PANDALens: Enhancing Daily Activity Documentation with AI-assisted In-Context Writing on OHMD. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–7.
  • [7] Karina Korovai, Dmytro Zhelezniakov, Oleg Yakovchuk, Olga Radyvonenko, Nataliya Sakhnenko, and Ivan Deriuga. 2024. Handwriting Enhancement: Recognition-Based and Recognition-Independent Approaches for On-device Online Handwritten Text Alignment. IEEE Access (2024).
  • [8] David Maulsby, Saul Greenberg, and Richard Mander. 1993. Prototyping an intelligent agent through Wizard of Oz. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. 277–284.
  • [9] Blagoj Mitrevski, Arina Rak, Julian Schnitzler, Chengkun Li, Andrii Maksai, Jesse Berent, and Claudiu Musat. 2024. InkSight: Offline-to-online handwriting conversion by learning to read and write. arXiv preprint arXiv:2402.05804 (2024).
  • [10] Pam A Mueller and Daniel M Oppenheimer. 2014. The pen is mightier than the keyboard: Advantages of longhand over laptop note taking. Psychological science 25, 6 (2014), 1159–1168.
  • [11] Subigya Nepal, Arvind Pillai, William Campbell, Talie Massachi, Eunsol Soul Choi, Xuhai Xu, Joanna Kuc, Jeremy F Huckins, Jason Holden, Colin Depp, et al. 2024. Contextual ai journaling: Integrating llm and time series behavioral sensing technology to promote self-reflection and well-being using the mindscape app. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems. 1–8.
  • [12] Lorelli S Nowell, Jill M Norris, Deborah E White, and Nancy J Moules. 2017. Thematic analysis: Striving to meet the trustworthiness criteria. International journal of qualitative methods 16, 1 (2017), 1609406917733847.
  • [13] Sharon Oviatt. 2006. Human-centered design meets cognitive load theory: designing interfaces that help people think. In Proceedings of the 14th ACM international conference on Multimedia. 871–880.
  • [14] Bill Schilit, Norman Adams, and Roy Want. 1994. Context-aware computing applications. In 1994 first workshop on mobile computing systems and applications. IEEE, 85–90.
  • [15] Lei Shi, Rachel Campbell, Peggy Chi, Maria Cirimele, Mike Cleron, Kirsten Climer, Chelsey Q Fleming, Ashwin Ganti, Philippe Gervais, Pedro Gonnet, et al. 2024. Inkeraction: An Interaction Modality Powered by Ink Recognition and Synthesis. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. 1–26.
  • [16] Timothy J Smoker, Carrie E Murphy, and Alison K Rockwell. 2009. Comparing memory for handwriting versus typing. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 53. SAGE Publications Sage CA: Los Angeles, CA, 1744–1747.
  • [17] Yee Fan Tan, Tee Connie, Michael Kah Ong Goh, and Andrew Beng Jin Teoh. 2022. A pipeline approach to context-aware handwritten text recognition. Applied Sciences 12, 4 (2022), 1870.
  • [18] Hsin-Ruey Tsai, Shih-Kang Chiu, and Bryan Wang. 2024. GazeNoter: Co-Piloted AR Note-Taking via Gaze Selection of LLM Suggestions to Match Users' Intentions. arXiv preprint arXiv:2407.01161 (2024).
  • [19] Audrey LH van der Meer and FR Van der Weel. 2017. Only three fingers write, but the whole brain works: a high-density EEG study showing advantages of drawing over typing for learning. Frontiers in psychology 8 (2017), 706.