Date of Award

5-14-2026

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Information Science

First Advisor

Xiaowei Xu

Abstract

Large Language Model research has made large strides in capabilities from sentiment analysis to writing code. These advancements have been realized thanks to research into specific capabilities such as prompting techniques. Language models today have demonstrated the ability to create content, transform, and classify. These capabilities are not limited to academic exploration but also found in commercial products that are positioning themselves from application augmentation to personal assistants. These commercial products tend to steer towards single actions such as “summarize this article” or “write a function that performs action...” In parallel research has continued to advance towards more advanced constructs such as agentic systems that are capable of more advanced activities such as “design and write a program that...” These agentic systems tend to rely on the language model itself as its policy driving actions. The ability for a language model to write code has been applied towards automation of training a model. In addition, language models have been used as part of data augmentation techniques to create synthetic data. Unfortunately, exploration of automating the Machine Learning workflow in concert with synthetic training data generation has been underrepresented in current research. This work aims to expand understanding of agentic architectures with a focus on synthetic data generation. To explore agency the task in focus is that of Entity Matching. Entity Matching lends itself to an agentic system. This task is an ideal problem due to the nature of Entity matching. Data privacy can be a challenge as well as the sparse nature of available training data provide two strong motivators for the need of a system capable of not only training but in the creation of training data. The target architecture therefore must not only automate the training of a downstream model, but create the training data needed with no direct access to ground-truth data. This work demonstrates through a novel agentic architecture in concert with a steering mechanism that relies on varying levels of agency a reliable and repeatable approach for taking as input only a human description of the entity and through all phases of of the process to result in a downstream iii trained model capable of reliably performing the task of Entity Matching. This is demonstrated primarily through quantitative methods relying on both exploratory and confirmatory experiments. The findings show that constraining agency provides benefit in overall performance. In addition, the work demonstrates a method for measuring and comparing differing text complexity measurements is a sufficient signal for steering and dataset planning for downstream model training. While general purpose LLMs are used directly or as components within an agent or agentic system there are trade-offs. System design is a series of compromises. For an Agent that must be capable of solving a diverse set of unknown problems at system construction then agency naturally must be higher. If the scope of the system is known at system design then by selecting lower levels of agency not only will improve downstream performance but provide a more computationally efficient system.

Download

Share

COinS