README.md

Plots on the Results

Navigating Open LLM Semantic Web Technology Support with Capability Compass

Family	0.5B	1B	1.5B	3B/3.8B	7B/8B	MoE Active 6-14B	14B	32B/33B	70B/72B
Llama 3.0
-->					Meta-Llama-3-8B-Instruct				Meta-Llama-3-70B-Instruct
Llama 3.1
-->					Llama-3.1-8B-Instruct				Llama-3.1-70B-Instruct
Llama 3.2
-->		Llama-3.2-1B-Instruct		Llama-3.2-3B-Instruct
Llama 3.3
-->									Llama-3.3-70B-Instruct
Phi 3.0
-->				Phi-3-mini-128k-instruct	Phi-3-small-128k-instruct		Phi-3-medium-128k-instruct
Phi 3.5
-->				Phi-3.5-mini-instruct		Phi-3.5-MoE-instruct
Qwen2
-->	Qwen2-0.5B-Instruct		Qwen2-1.5B-Instruct		Qwen2-7B-Instruct	Qwen2-57B-A14B-Instruct			Qwen2-72B-Instruct
Qwen2.5
-->	Qwen2.5-0.5B-Instruct		Qwen2.5-1.5B-Instruct	Qwen2.5-3B-Instruct	Qwen2.5-7B-Instruct		Qwen2.5-14B-Instruct	Qwen2.5-32B-Instruct	Qwen2.5-72B-Instruct
Qwen2.5-Coder
-->								Qwen2.5-Coder-32B-Instruct
Infly-OpenCoder
-->					OpenCoder-8B-Instruct
Deepseek-coder
-->								deepseek-coder-33b-instruct

Task Plots

Rdf Connection Explain Tasks

plot	caption
	RdfConnectionExplainStatic, graphFormat=jsonld: listTrimF1 score
	RdfConnectionExplainStatic, graphFormat=nt: listTrimF1 score
	RdfConnectionExplainStatic, graphFormat=turtle: listTrimF1 score
	RdfConnectionExplainStatic, graphFormat=xml: listTrimF1 score

RDF Friend Count Tasks

plot	caption
	RdfFriendCount, graphFormat=jsonld, 1 additional link: F1 score
	RdfFriendCount, graphFormat=jsonld, 2 additional links: F1 score
	RdfFriendCount, graphFormat=nt, 1 additional link: F1 score
	RdfFriendCount, graphFormat=nt, 2 additional links: F1 score
	RdfFriendCount, graphFormat=turtle, 1 additional link: F1 score
	RdfFriendCount, graphFormat=turtle, 2 additional links: F1 score
	RdfFriendCount, graphFormat=xml, 1 additional link: F1 score
	RdfFriendCount, graphFormat=xml, 2 additional links: F1 score

RDF Syntax Fix Tasks

plot	caption
	RdfSyntaxFixList, graphFormat=jsonld: max(combined) score
	RdfSyntaxFixList, graphFormat=nt: max(combined) score
	RdfSyntaxFixList, graphFormat=turtle: max(combined) score

SPARQL Syntax Fix Task

plot	caption
	SparqlSyntaxFixingList, dataset=LcQuad: max(combined) score

SPARQL to Answer Tasks

plot	caption
	Sparql2Answer, dataset=Organisational, graphFormat=jsonld: combinedF1 score
	Sparql2Answer, dataset=Organisational, graphFormat=turtle: combinedF1 score

Text to Answer Tasks

plot	caption
	Text2Answer, dataset=Organisational, graphFormat=jsonld: combinedF1 score
	Text2Answer, dataset=Organisational, graphFormat=turtle: combinedF1 score

Text to SPARQL Tasks

plot	caption
	Text2Sparql, dataset=Organizational, graphInfo=turtle graph: max(combined) score
	Text2Sparql, dataset=Orga Numerical, graphInfo=turtle graph + ID-label-mapping: max(combined) score
	Text2Sparql, dataset=Coypu Mini, graphInfo=turtle graph: max(combined) score
	Text2Sparql, dataset=Beastiary, graphInfo=turtle schema: max(combined) score
	Text2Sparql, dataset=Beastiary, graphInfo=turtle subschema: max(combined) score
	Text2Sparql, dataset=Beastiary, graphInfo=turtle subgraph: max(combined) score