TY  - THES
AB  - Python’s dynamic type system offers flexibility but often leads to runtime errors and reduced maintainability in large-scale software systems. While optional type annotations (PEP 484) help mitigate these issues, they are inconsistently adopted across real-world codebases. To address this gap, recent studies have explored the use of Large Language Models (LLMs) for type inference, showing promising results on micro-benchmarks. However, their performance on real-world codebases remains underexplored.This thesis investigates the effectiveness of LLMs for Python type inference using a real-world dataset. We extend the TypeEvalPy framework by incorporating the ManyTypes4Py dataset, enabling a comprehensive evaluation of LLM performance across frequent, rare, and user-defined types. Two state-of-the-art LLMs, Codestral (22B) and Qwen2.5-Coder (7B), are evaluated using two prompting strategies on microbenchmark: mask-based prompting and question-and-answer (QnA) prompting. Furthermore, we apply Parameter-Efficient Fine-Tuning (PEFT) using LoRA to adapt these models to the type inference task.Our results show that QnA prompting significantly outperforms mask-based prompting on the TypeEvalPy micro-benchmark. Codestral achieves an overall exact match accuracy of 88.7% with QnA prompting, compared to 67.8% with mask-based prompting. Qwen2.5-Coder improves from 61.5% to 83.6% using the same strategy. Finetuning further boosts performance: Codestral improves from 86.4% to 96.9% , and Qwen2.5-Coder from 84.0% to 93.8%. Analysis of frequent and rare types shows that fine-tuning enhances structured type inference while occasionally misclassifying generictypes. These findings suggest that LLMs provide a robust solution for type inference in real-world scenarios, though improvements are needed for rare and user-defined types.
AU  - Rashida, Bharmal
CY  - Paderborn
DA  - 2025
DO  - 10.17619/UNIPB/1-2481
DP  - Universität Paderborn
LA  - eng
N1  - Tag der Abgabe: 28.03.2025
N1  - Universität Paderborn, Masterarbeit, 2025
PB  - Veröffentlichungen der Universität
PY  - 2026
SP  - 1 Online-Ressource (vi, 50 Seiten) : Illustrationen, Diagramme
T2  - Fakultät für Elektrotechnik, Informatik und Mathematik
TI  - Assessing Large Language Models for Type Inference in python on real-world dataset
UR  - https://nbn-resolving.org/urn:nbn:de:hbz:466:2-57058
Y2  - 2026-01-24T23:32:37
ER  -