Towards Machine-Generated Code for the

Resolution of User Intentions

arXiv:2504.17531v1 [[Link]] 24 Apr 2025

Justus FLERLAGE a , Ilja BEHNKE a and Odej KAO a

a Distributed and Operating Systems Group

Technische Universität Berlin

Berlin, Germany

Abstract. The growing capabilities of Artificial Intelligence (AI), particularly

Large Language Models (LLMs), prompt a reassessment of the interaction mecha-
nisms between users and their devices. Currently, users are required to use a set of
high-level applications to achieve their desired results. However, the advent of AI
may signal a shift in this regard, as its capabilities have generated novel prospects
for user-provided intent resolution through the deployment of model-generated
code, which is tantamount to the generation of workflows comprising a multitude
of interdependent steps. This development represents a significant progression in
the realm of hybrid workflows, where human and artificial intelligence collaborate
to address user intentions, with the former responsible for defining these intentions
and the latter for implementing the solutions to address them. In this paper, we in-
vestigate the feasibility of generating and executing workflows through code gen-
eration that results from prompting an LLM with a concrete user intention, such
as Please send my car title to my insurance company, and a simplified application
programming interface for a GUI-less operating system. We provide in-depth anal-
ysis and comparison of various user intentions, the resulting code, and its execu-
tion. The findings demonstrate a general feasibility of our approach and that the
employed LLM, GPT-4o-mini, exhibits remarkable proficiency in the generation of
code-oriented workflows in accordance with provided user intentions.
Keywords. User-Machine Interaction, Large Language Models, Artificial Intelligence,
Code Generation, GUI-less Operating Systems

1. Introduction

The current user interfaces on smartphones and other devices implement an action-
oriented paradigm, where the user transforms their own intention into a workflow of
commands and executes those by clicking on the corresponding entities, mostly apps,
web services or other activity representations. For example, the request ”reschedule my
appointment for tonight” appearing to be a relatively straightforward task, actually ne-
cessitates a plethora of sophisticated activities. The initial action for the user is to launch
the calendar application in order to ascertain which appointment with whom requires
rescheduling. Subsequently, the user is required to access the contacts application to as-
certain the telephone number of the appointment participant, open a channel of com-
munication and commence negotiations regarding an alternative appointment. This sim-
ple example illustrates the burdensome obligation for the user to ensure a consistency
between the individual steps, gather, adapt, and transfer the data between the workflow
stages, find ad-hoc alternatives in case of failures, and repeat the procedure until a sat-
isfactory result is achieved. Thus, the intelligence for the workflow design as well as
the orchestration of the execution are located at the user. Moreover, the efficiency of the
process and the user experience heavily depend on the user experience and skills using
electronic devices.
With the advent of LLMs, an opportunity arises to gain assistance with the design
and execution of such everyday workflows. Currently available LLMs are able to under-
stand natural language, identify the intention in the submitted phrase, and provide an-
swers or perform actions such as translation or summary of given documents [1]. How-
ever, LLMs offer a further opportunity to decompose the intent into actionable steps and
thus design a workflow in a similar fashion as human users. This capacity facilitates a
transition from an action-based user interface to an intention-based user interface. In this
new paradigm, an LLM assumes the role of an input processor for intents articulated in
natural language, a semantic analyst and an intention interpreter, a workflow designer,
and an execution orchestrator. This transition reduces the effort required from the device
user, ultimately resulting in a hybrid interaction and collaboration between the user and
the artificial intelligence, where the user articulates intentions and the artificial intelli-
gence actualises these.
The key to the effective utilisation of such LLMs is to embed them into the local
device. Currently, due to the substantial resource requirements of the inference step, it is
necessary to perform this process on remote infrastructure. Target devices for this type
of intent resolution can be future smartphones in combination with wearables such as
the currently introduced camera-equipped glasses. Removing large smartphone screens
creates significant challenges, but also allows a major reduction in size and leads to lower
energy consumption by 80 % [2]. Users may formulate their intentions in natural lan-
guage and receive acknowledgments and a resulting summary once the task is processed.
A detailed and urgent view on the incoming information can be projected on nearby
screens or on the glasses. Replacing the screen with camera-glasses will allow a two-way
visual communication: camera input along with the position data, surrounding noise, and
other sensor information will provide valuable multimodal information to be processed
by the LLM and played back enriched with augmented information.
In this paper we present the first step on the path to such a GUI-less operating sys-
tem. We investigate, whether existing LLMs can be used for the recognition and trans-
formation of intentions into workflows of application activities to fulfill them. We em-
ploy a LLM to process typical examples of intentions, subsequently extracting code with
sequences of operations as the output, which functions as an abstraction of finite state
machines. The employed LLM for this is OpenAI’s GPT-4o-mini [3] model accessed via
a paid plan for programmatic access. We present a proof of concept that demonstrates
the feasibility of using LLMs to generate and execute code that addresses a simplified
application programming interface (API) for resolving user intentions. Additionally, we
evaluate the quality and performance of the generated results.
In Section 2 of this paper, the related work in the domain of LLMs is presented.
This is followed by the introduction of preliminary considerations in Section 3. Section 4
introduces a system architecture for the user intention realisation through the generation
of code. The results employing the aforementioned architecture as well as a discussion
is presented in Section 5. Finally, the conclusion is presented in Section 6.
 The implemented prototype as well as the generated codes and execution traces of
different experiments can be found in our repository 1 .

2. Related Work

The advent of LLMs and the public accessibility of ChatGPT [4] have precipitated a
surge in prospective applications, including those in the public health and medicine sec-
tor [5][6][7], the education and didactic sector [8][9], and natural language processing
in general [10][11] with a growing necessity to categorize artificial intelligence [12].
The transformer model [13] is the foundational concept behind LLMs [14], which con-
sequently, has resulted in the design and development of applications and frameworks
such as AIOS [15][16], for machine-oriented user-intention resolution using LLMs. This
domain originates from voice assistants, such as Siri, Cortana or Alexa [17]. Recent re-
search around the idea of incorporating AI for resolving user intentions utilizes models
which are trained for operating GUI applications [18].
The generation of code with AI is a large subject of study. Evaluations show the
feasibility of solving certain problems by generating code with GPT language mod-
els [19][20]. Tools like GitHub Copilot already support engineers in every day tasks
[21][22]. Issues, such as hallucinations and erroneous code generation are addressed by
fuzzing and static analysis [23], novel approaches utilizing grammar augmentation [24]
as well as the redesign of transformer decoding algorithms [25].

3. Preliminary Considerations

The process of rescheduling a meeting with a person, for example, involves a series of
steps, including the retrieval of meeting information, the lookup of contact data, the initi-
ation of communication with the person, the proposal of alternative dates, the negotiation
with the person, and the consideration of proposed alternative dates by the person. This
illustrates the complexity of generating such workflows. However, the effective gener-
ation of workflows is of paramount importance in the resolution of user intentions. In
order to address intentions such as the aforementioned one, a semantic workflow model
is required, which is capable of dealing with the complexity of such intentions.
From a macro perspective, a single workflow can be conceptualized as a finite state
machine, constituted of multiple steps, which must be executed in a predetermined and
ordered fashion for the resolution of a user intention. The interconnection of these steps
allows for control flow with loops as well as case differentiations. Ideally, individual steps
of these workflows are reusable throughout different workflows for reducing the overall
complexity. Figure 1 illustrates an exemplary schematic of such a workflow generated
from an intention by an AI from a semantic point of view.
The elements of workflows and their depiction as finite state machines, in conjunc-
tion with the employment of control flow with loops and case differentiations, directly
result in the consideration and utilisation of programming languages in the context of
workflows and their execution. LLMs can support this execution model by their ability
to generate code.
1 [Link]
 Intention

AI

Workflow

Step 1 Step 2 Step 3

Step 4 Step 5 Step 6

External Event

Figure 1. Exemplary schematic showing a workflow with multiple asynchronous and eventually recurring
steps.

4. Method

We simulate the target environment of mobile devices by designing an architecture, as

shown in Figure 2, consisting of the following components:
• Operating system
• LLM Service for generating code from a prompt
• Voice-To-Text Service
• Controller as intermediary component, including a Prompt Formatter, the
Function Table and the Executor
In light of the inherent constraints of commodity hardware and the considerable
resource demands of LLM inference, we propose the introduction of an LLM Service
component, which is responsible for executing the LLM inference phase. The long-term
objective of reducing the size of the LLM to a level that can be accommodated within the
execution constraints of a typical mobile device represents a significant research chal-
lenge. In order to achieve this, it is essential to have a detailed understanding of the spe-
cific characteristics of the model in question. The LLM Service represents an abstraction
for utilising the LLM, and it is designed to operate on an external infrastructure that is
distinct from the user device.
The Controller serves as the central component responsible for addressing and en-
capsulating the LLM Service. It also oversees the serialisation and deserialisation of mes-
sages exchanged and manages the scheduling and execution of tasks. It is accessed by a
text prompt, or by voice using a Voice-To-Text Service for transformation.
Representing one of the most crucial components of this system, the Function Ta-
ble exploits the ability of LLMs to generate imperative code consisting of functions as
atomic, modular, and composable building blocks. This is analogous to a workflow com-
prising disparate steps. The Function Table serves two distinct purposes. Primarily, it is
 Intention

Text Controller
Voice

Prompt Formatter Function 0

Function 1

Function Table
Function 2

Function ...

Function n

Executor

Voice-To-Text
LLM Service
Service

Operating System

Figure 2. System architecture

employed by the Prompt Formatter to generate a documentation of accessible func-

tions, which is subsequently provided with the prompt to the LLM. Secondly, it contains
the actual executable code of the specific function, which is utilised by the Executor
for code execution following the generation step. These callbacks employ a range of
operating system interfaces to facilitate a high-level application programming interface.
In order to elucidate the concrete relationship between the components presented in
our system architecture, we present an exemplar data flow from intention to the execution
of the generated workflow.
At the outset of the intention realisation process, the user inputs the intention. In
the event that the intention is provided as voice input, the Controller is initially tasked
with transforming the voice into text. Subsequently, the text is conveyed to the prompt
formatter, which generates a prompt utilising a Prompt Template. This prompt format
incorporates both the intention expressed in text and a documentation of the function
signatures of the Function Table, thereby facilitating the LLM’s comprehension of its
capabilities. Subsequently, the prompt is conveyed to the LLM Service, which generates
code based on the provided prompt. The Controller then reads the output and initiates
the execution of the generated code.
We utilize the system architecture to demonstrate the feasibility of employing LLMs
for the generation and execution of code based on user intentions for exploiting the gen-
eration of workflows. In these experiments, the user intentions are provided as text input
to the Controller and are formatted with the prompt template illustrated in Figure 3. The
role of the LLM is set to You are a Python 3 code generator. The Prompt Template is
generated from the Function Table, which contains the signatures and implementation
callbacks of the pertinent functions. The resulting output is then read and executed sub-
You have the following application programming interface :

function find_file_id ( expression : String ) : Integer | null

function f in d _c on ta c t_ id ( expression : String ) : Integer | null
function f i n d _ c o n t a c t _ e m a i l ( contact_id : Integer ) : String | null
function play_voice ( text : String ) : void
function ask_question ( question : String ) : String
function p la y _a ud io _ fi le ( file : File ) : void
function send_email ( email : String , subject : String , text : String ,
attachments : Collection < Integer >) : void
function print_screen ( text : String ) : void
function shell ( command : String ) : String

Write Python 3 code only , which uses the application programming

interface for the instruction
"{ intention }"

Figure 3. Prompt template, which is generated from a function table.

Intention Successes Average Response Time (s) Average Time to First Token (ms)
1 5 ≈ 3.31 ≈ 466.0
2 5 ≈ 1.31 ≈ 460.5
3 5 ≈ 2.03 ≈ 491.4
4 3 ≈ 1.47 ≈ 461.0
Table 1. Overview of the intentions and their average response time as well as their average time to first token
for executing the particular generated code.

sequently, utilizing Python’s exec function to execute the generated code with the desig-
nated functions in the global execution namespace. This configuration facilitates a con-
strained execution, whereby only the functions specified by the Function Table can be
executed. In order to demonstrate the actual execution of the provided functions, the im-
plementation callbacks utilize stub functions that print the name of the calling function
and the arguments that have been passed. The following four intentions are considered:
1. Please send my car title to my insurance company
2. Please tell me the current temperature
3. Please play the song beat it by michael jackson
4. Please tell me all files in my home directory
Each intention is fed to the system for a total of five times. An overview of the col-
lected metrics, including the response time and the time to first token, is presented in
Table 1. For the first three intentions, the generated code successfully executes during the
five trials. However, for intention 4, two trials result in failure. The underlying causes of
these two failures are attributed to two primary factors. Firstly, there was unauthorized
access to packages that were not available in the isolated execution environment. Sec-
ondly, there was a violation of the rules governing scoping in programming languages.
The intention with the most involved steps, intention 1, is the intention for which the best
resulting generated code is provided in Figure Figure 5 and its execution trace is shown
in Figure 5. The output for the remaining intentions can be located in the previously cited
repository.
# Step 1: Define the important variables
c o n t a c t _ e m a i l _ e x p r e s s i o n = " insurance company "
email_subject = " Car Title Submission "
email_text = " Please find attached the car title for my vehicle . "
attachments = []

# Step 2: Find the contact ID for the insurance company

contact_id = f i nd _c on t ac t_ id ( c o n t a c t _ e m a i l _ e x p r e s s i o n )

if contact_id is not None :

# Step 3: Get the email address of the contact
email_address = f i n d _ c o n t a c t _ e m a i l ( contact_id )

if email_address is not None :

# Step 4: Find the car title file ID
f il e_ ex p re ss io n = " car title "
file_id = find_file_id ( f il e_ e xp re s si on )

if file_id is not None :

attachments . append ( file_id )

# Step 5: Send the email with car title attachment

send_email ( email_address , email_subject , email_text ,
attachments )
else :
print_screen ( " Could not find the email address for the
insurance company . " )
else :
print_screen ( " Could not find the contact ID for the insurance
company . " )

Figure 4. Generated code sample for the intention Please send my car title to my insurance company.

Execute " fi n d_ co nt a ct _i d " and arguments " insurance company "

Execute " f i n d _ c o n t a c t _ e m a i l " and arguments "1"
Execute " find_file_id " and arguments " car title "
Execute " send_email " and arguments " john . doe@example . com " , " Car Title
Submission " , " Please find attached the car title for my vehicle
." , "[1]"

Figure 5. Execution trace sample for the intention Please send my car title to my insurance company.

5. Results and Discussion

The experiments presented illustrate the viability of using LLM technology for code gen-
eration and execution tasks, contingent upon the input of user intentions. GPT-4o-mini
demonstrates remarkable proficiency in this domain. It is capable of comprehending both
the previously unknown application programming interface and the specific user inten-
tion, subsequently generating accurate and well-behaved code that is mostly executable
without any interference or issue. The generated code as well the execution traces reveal
that the model is capable of comprehending not only the function signatures associated
with the specified application programming interface, but also the intended purpose of
each function solely based on its signature. Moreover, it is capable of accurately compos-
ing these functions, performing the requisite case differentiation, and providing a com-
prehensive explanation. For all provided examples, the LLM responds with the generated
code within seconds. This remains within an acceptable range.
The primary objective for the generation of code is correctness. Our failed attempts
in intention 4 underline this. The subsequent consideration is the duration of the gen-
eration. It raises the question of how specialized LLMs perform in this regard, which
may involve further optimization through quantization, distillation, as well as pruning
[26], and may ultimately enable the execution of LLMs on local devices, since in the
experiments the LLM ran on external infrastructure and not locally. In general, build-
ing on well-established and well-understood operating system concepts with generic ab-
stractions around multiuser and batch-oriented computing, as well as battle-tested tech-
nologies, appears to be a reasonable approach at first glance. Nevertheless, the domain
of machine-oriented user intent realization is highly specialized and may not necessi-
tate such comprehensive mechanisms. This prompts the question of how LLMs and their
code generation capabilities might be employed in the future, as well as the extent to
which the LLM-accessed interfaces will be either generic or specific.
The optimal use of the abilities of LLMs to generate code remains undetermined.
Given its status as one of the most widely used programming languages in the current era
and its extensive array of supporting tools and resources, as well as the fact that LLMs
are primarily trained on publicly available data, we decided to use Python as our chosen
programming language for code generation. Moreover, the provision of the exec function
facilitates the execution of code based on given functions within a restricted environment.
Solutions based on serialisable message passing using exchange formats such as JSON
were considered but ultimately rejected due to their lack of support for features such as
control flow and function composition.
The automatic generation and execution of unverified code introduces a potential
security hazard, which requires further investigation. From a technical vantage point,
the isolated execution of unverified code necessitates counter-measures and hardening
to prevent environment breakouts. It is imperative that the execution environment reside
within a designated sandbox, thereby constraining its functionality to the parameters de-
fined within the prompt. While soliciting user consent for the execution of generated
code appears to be a viable solution, it ultimately necessitates the user’s capacity for
comprehending code or a graphical visualization of the generated code. However, this
approach does not consider the potential for malicious uses such as code injections. A
more viable approach would be to reduce the capabilities of functions that can be exe-
cuted by the LLM. In the previously mentioned examples, this approach would entail the
removal of the shell function, as it provides a very powerful interface to the underlying
operating system. Another viable mechanism involves the introduction of access layers
that, upon access by the LLM, must be explicitly acknowledged by the user. Ensuring
reproducibility and comprehensibility necessitates the implementation of a system capa-
ble of tracing the actions executed. The present system architecture is conducive to this
objective by virtue of its provision of function wrappers.
The transition from manual operation of a GUI for user-intention resolution to the
expression of user intention solely in natural language signifies a paradigm shift. Conse-
quently, the transition toward automated resolution of user intentions gives rise to con-
cerns regarding the reliance and dependability of users on AI and the impact of tech-
nology on users’ understanding of the processes involved when resolving user-provided
intentions manually. This issue is addressed in greater detail in [27], which examines the
technology gap that contemporary students face.

6. Conclusions

This paper presents requisite actions for the resolution of user intentions. Moreover, the
presented system architecture is employed to address workflows and their semantic as-
pects in experiments investigating the potential of machine-generated commands for the
resolution of user intentions. The viability of employing LLM for the generation and
execution of code is demonstrated. This is achieved by providing the LLM with prompts
that include a user intention as well as a specification for an application programming
interface. GPT-4o-mini displays a notable degree of precision in meeting this specific ob-
jective. The system is capable of comprehending the user intention and the proposed, yet
unknown, application programming interface, effectively generating Python 3 code that
is executed for the purpose of resolving exemplary user intentions. These findings sug-
gest a promising future for the use of LLMs in this particular application domain. There
is considerable potential for optimisation, particularly in the specific case of generating
code from provided user intentions.

References

[1] Jin H, Zhang Y, Meng D, Wang J, Tan J. A comprehensive survey on process-oriented automatic text
summarization with exploration of llm-based methods. arXiv preprint arXiv:240302901. 2024.
[2] Carroll A. Understanding and reducing smartphone energy consumption. University of New South
Wales; 2017.
[3] Achiam J, Adler S, Agarwal S, Ahmad L, Akkaya I, Aleman FL, et al. Gpt-4 technical report. arXiv
preprint arXiv:230308774. 2023.
[4] Wu T, He S, Liu J, Sun S, Liu K, Han QL, et al. A Brief Overview of ChatGPT: The History, Status Quo
and Potential Future Development. IEEE/CAA Journal of Automatica Sinica. 2023;10(5):1122-36.
[5] Biswas SS. Role of chat gpt in public health. Annals of biomedical engineering. 2023;51(5):868-9.
[6] Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models
in medicine. Nature medicine. 2023;29(8):1930-40.
[7] Nazi ZA, Peng W. Large language models in healthcare and medical domain: A review. In: Informatics.
vol. 11. MDPI; 2024. p. 57.
[8] Firat M. How chat GPT can transform autodidactic experiences and open education? 2023.
[9] Kasneci E, Seßler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good?
On opportunities and challenges of large language models for education. Learning and individual dif-
ferences. 2023;103:102274.
[10] Kalla D, Smith N, Samaah F, Kuraku S. Study and analysis of chat GPT and its impact on different
fields of study. International journal of innovative science and research technology. 2023;8(3).
[11] Minaee S, Mikolov T, Nikzad N, Chenaghlu M, Socher R, Amatriain X, et al. Large language models:
A survey. arXiv preprint arXiv:240206196. 2024.
[12] Morris MR, Sohl-Dickstein J, Fiedel N, Warkentin T, Dafoe A, Faust A, et al. Position: Levels of AGI
for Operationalizing Progress on the Path to AGI. In: Forty-first International Conference on Machine
Learning; 2024. Available from: 10.48550/arXiv.2311.02462.
[13] Vaswani A. Attention is all you need. Advances in Neural Information Processing Systems. 2017.
[14] Chang Y, Wang X, Wang J, Wu Y, Yang L, Zhu K, et al. A survey on evaluation of large language
models. ACM Transactions on Intelligent Systems and Technology. 2024;15(3):1-45.
[15] Mei K, Li Z, Xu S, Ye R, Ge Y, Zhang Y. AIOS: LLM agent operating system. arXiv e-prints, pp
arXiv–2403. 2024. Available from: 10.48550/arXiv.2403.16971.
[16] Shi Z, Mei K, Jin M, Su Y, Zuo C, Hua W, et al. From Commands to Prompts: LLM-based Semantic
File System for AIOS. arXiv preprint arXiv:241011843. 2024.
[17] Hoy MB. Alexa, Siri, Cortana, and more: an introduction to voice assistants. Medical reference services
quarterly. 2018;37(1):81-8.
[18] Liu X, Qin B, Liang D, Dong G, Lai H, Zhang H, et al.. AutoGLM: Autonomous Foundation Agents for
GUIs; 2024. Available from: [Link]
[19] Chen M, Tworek J, Jun H, Yuan Q, Pinto HPDO, Kaplan J, et al. Evaluating large language models
trained on code. arXiv preprint arXiv:210703374. 2021.
[20] Lin F, Kim DJ, Tse-Husn, Chen. SOEN-101: Code Generation by Emulating Software Process Models
Using Large Language Model Agents; 2024. Available from: [Link]
[21] Moradi Dakhel A, Majdinasab V, Nikanjam A, Khomh F, Desmarais MC, Jiang ZMJ. GitHub Copilot
AI pair programmer: Asset or Liability? Journal of Systems and Software. 2023;203:111734. Available
from: [Link]
[22] Wermelinger M. Using GitHub Copilot to Solve Simple Programming Problems. In: Proceedings of
the 54th ACM Technical Symposium on Computer Science Education V. 1. SIGCSE 2023. New York,
NY, USA: Association for Computing Machinery; 2023. p. 172–178. Available from: [Link]
1145/3545945.3569830.
[23] Ouyang S, Zhang JM, Harman M, Wang M. An Empirical Study of the Non-determinism of ChatGPT in
Code Generation. ACM Transactions on Software Engineering and Methodology. 2024 Sep. Available
from: [Link]
[24] Ugare S, Suresh T, Kang H, Misailovic S, Singh G. SynCode: LLM Generation with Grammar Aug-
mentation; 2024.
[25] Zhang S, Chen Z, Shen Y, Ding M, Tenenbaum JB, Gan C. Planning with Large Language Models for
Code Generation; 2023.
[26] Sreenivas ST, Muralidharan S, Joshi R, Chochowski M, Patwary M, Shoeybi M, et al. LLM Pruning
and Distillation in Practice: The Minitron Approach. arXiv preprint arXiv:240811796. 2024. Available
from: 10.48550/arXiv.2408.11796.
[27] Pfaltzgraf D, Insch GS. Digitally Native, Yet Technologically Illiterate: Methods to Prepare Business
Students to Create Versus Consume. Journal of Applied Business and Economics. 2021 May. Available
from: [Link]