Enhancements to Large Language Models: Introducing Dynamic Syntactic Insertion for Improved Model Robustness and Generalization DOI

Elena Tremaskina,

Santiago Deluca,

Christopher M. Thompson

et al.

Authorea (Authorea), Journal Year: 2024, Volume and Issue: unknown

Published: Oct. 14, 2024

The growing complexity and scale of modern deep learning models have improved the ability to generate understand human language, yet challenges persist in achieving robust generalization syntactic flexibility.Dynamic Syntactic Insertion (DSI) addresses these limitations through novel introduction random variations during finetuning phase, enhancing model's capacity process diverse linguistic structures.Through empirical experiments on GPT-NeoX architecture, significant performance improvements were observed across multiple metrics, including robustness, fluency, accuracy.The DSI-enhanced model consistently outperformed baseline, particularly handling syntactically complex perturbed datasets, demonstrating its adaptability a broader range inputs.Furthermore, incorporation variability led reductions perplexity increased tasks GLUE benchmark, highlighting method's effectiveness.The findings from this study suggest that augmentation techniques, such as DSI, provide promising pathway for improving resilience language environments.

Language: Английский

Enhancements to Large Language Models: Introducing Dynamic Syntactic Insertion for Improved Model Robustness and Generalization DOI

Elena Tremaskina,

Santiago Deluca,

Christopher M. Thompson

et al.

Authorea (Authorea), Journal Year: 2024, Volume and Issue: unknown

Published: Oct. 14, 2024

The growing complexity and scale of modern deep learning models have improved the ability to generate understand human language, yet challenges persist in achieving robust generalization syntactic flexibility.Dynamic Syntactic Insertion (DSI) addresses these limitations through novel introduction random variations during finetuning phase, enhancing model's capacity process diverse linguistic structures.Through empirical experiments on GPT-NeoX architecture, significant performance improvements were observed across multiple metrics, including robustness, fluency, accuracy.The DSI-enhanced model consistently outperformed baseline, particularly handling syntactically complex perturbed datasets, demonstrating its adaptability a broader range inputs.Furthermore, incorporation variability led reductions perplexity increased tasks GLUE benchmark, highlighting method's effectiveness.The findings from this study suggest that augmentation techniques, such as DSI, provide promising pathway for improving resilience language environments.

Language: Английский

Citations

0