0.25
0.5
0.75
1.25
1.5
1.75
2
Assessing the Evolution of LLM capabilities for Knowledge Graph Engineering in 2023
Published on Jun 17, 202432 Views
In this article, we evaluate the evolution of LLM capabilities w.r.t. the RDF Turtle and SPARQL language as foundational skills to assist with various KGE tasks. We measure the LLM response quality us
Related categories
Chapter list
Assessing the Evolution of LLM capabilities for Knowledge Graph Engineering in 202300:00
LLMs - KGs - 100:04
LLMs - KGs - 200:27
Motivation00:47
Research Questions02:03
Method: LLM-KG bench Framework [2]03:17
Benchmark Tasks: RDF skill dimension & levels [3,4]04:00
Benchmark Tasks: Setup05:33
T2 -Turtle Fix: Claude Evolution06:22
T2 –Turtle Fix: GPT Evolution07:33
Task T4 -Most-known Person: Input08:36
T4: Most-known Person -Special Case –6 Persons09:21
T4: GPT Evolution11:20
T4: Claude Evolution12:01
Summary & Conclusions12:49
Future Work15:35
Thank you16:49