The Computational Assyriologist — a three-part series.
Stephen Wolfram observes that for every field of science X there eventually emerges a computational X. This series asks what a computational Assyriology looks like: can we read the roughly nineteen thousand cuneiform tablets that carry no published translation (and most likely never had one) – and then study them not one at a time, but as a single corpus? The train of thought runs in three movements – first build the instrument, then learn to ask questions with it, then turn it loose on the whole archive:
◼ Part I — Building the translator (you are here): fine-tune a small, laptop-runnable Akkadian–English model (QLoRA), and measure honestly how well it reads.
◼ Part II — A research instrument: turn the translator into a Wolfram-Language research instrument – an atlas, social networks, intertextuality, omen logic, the Ark, a 763-BC eclipse.
◼ Part III — Reading nineteen thousand dark tablets: let a frontier model loose on the ~19,700 untranslated tablets (most likely never translated) and mine the whole library at once for names, places, stories and discoveries.
Each part stands on its own; the earlier parts build the tools used later. All code, data and build scripts are in the project repository.
Stephen Wolfram observes that for every field of science X there eventually emerges a computational X. This series asks what a computational Assyriology looks like: can we read the roughly nineteen thousand cuneiform tablets that carry no published translation (and most likely never had one) – and then study them not one at a time, but as a single corpus? The train of thought runs in three movements – first build the instrument, then learn to ask questions with it, then turn it loose on the whole archive:
◼ Part I — Building the translator (you are here): fine-tune a small, laptop-runnable Akkadian–English model (QLoRA), and measure honestly how well it reads.
◼ Part II — A research instrument: turn the translator into a Wolfram-Language research instrument – an atlas, social networks, intertextuality, omen logic, the Ark, a 763-BC eclipse.
◼ Part III — Reading nineteen thousand dark tablets: let a frontier model loose on the ~19,700 untranslated tablets (most likely never translated) and mine the whole library at once for names, places, stories and discoveries.
Each part stands on its own; the earlier parts build the tools used later. All code, data and build scripts are in the project repository.
Run this notebook. Download the bundle assyriologist_part1.wl attached to this post into the same folder as this notebook, then evaluate the cell below once. It loads every function and dataset used here, so each command cell underneath runs and reproduces its figure.
SetDirectory[NotebookDirectory[]];Get["assyriologist_part1.wl"]
A single line of Neo-Assyrian royal cuneiform from the held-out test set (a campaign inscription of Ashurbanipal), rendered directly from its Unicode code points with the Noto Sans Cuneiform font. This notebook follows such a line all the way to English.
Akkadian — the language of Babylon and Assyria — was written in cuneiform on clay for roughly three millennia. Hundreds of thousands of tablets survive as scholarly transliterations, but only a fraction carry a published English translation. That gap is a natural target for machine translation.
This notebook tells the whole story end to end: what cuneiform and transliteration are; how a tablet becomes signs, then Latin-letter transliteration, then English; where the tablets in the training corpus were excavated; how a QLoRA fine-tune of Mistral-7B was trained on ~45,000 Akkadian–English sentence pairs to produce a model that runs offline on a laptop via Ollama; and — crucially — a leakage-aware benchmark with bootstrap confidence intervals that states honestly how good (and how fallible) the result is.
Audience: no knowledge of Akkadian is assumed. The aim is a self-contained, scientifically careful tour that a Wolfram Community reader can follow and reproduce.
1. Akkadian, cuneiform, and 3,000 years of clay
1. Akkadian, cuneiform, and 3,000 years of clay
Akkadian is the oldest attested Semitic language, a relative of Hebrew and Arabic, spoken in Mesopotamia (modern Iraq) from before 2300 BCE. Its two great dialects are Babylonian (south) and Assyrian (north). It was written in cuneiform — "wedge-shaped" — signs pressed into wet clay with a reed stylus, a script invented for Sumerian and adapted to Akkadian. Clay is nearly indestructible once baked or simply buried, so the written record is vast: royal inscriptions, letters, contracts, omens, accounts, science and literature including the Epic of Gilgamesh.
A single cuneiform sign can stand for a syllable (ša, mu) or a whole word (a logogram, conventionally transcribed in CAPITALS, e.g. LUGAL = "king"). The same sign has several possible readings depending on context, and sign shapes drift across centuries and regions — which is exactly what makes both human decipherment and automatic OCR hard.
Two famous Akkadian objects. Left: detail of the stele of the Laws of Hammurabi (Old Babylonian, ~1750 BCE; Louvre). Right: the Cyrus Cylinder (Neo-Babylonian, 539 BCE; British Museum). The dense columns of wedges are the raw input our pipeline ultimately has to read. Photos via Wikimedia Commons: Hammurabi stele © Mbzt, CC BY 3.0; Cyrus Cylinder © Prioryman, CC BY-SA 3.0.
1.5 How the language and its writing work
1.5 How the language and its writing work
A short primer before we start translating. The tablets are mostly Akkadian, the language of Assyria and Babylon – a Semitic language (the oldest attested branch, East Semitic), so a distant cousin of Hebrew and Arabic: Akkadian kalbu, Hebrew kelev, Arabic kalb all mean “dog.” It was written for roughly 3,000 years (c. 2350 BCE – 1st c. CE), one of the longest-lived written languages ever. Like its relatives it is built on a root-and-pattern morphology: a meaning lives in a skeleton of (usually three) consonants, and grammar is the vowels poured in – š-p-r “send/write” gives šapāru (to send), šipru (a message), šāpiru (an overseer). Word order is Subject–Object–Verb, and nouns take Latin-like case endings.
The writing. Cuneiform (“wedge-shaped”) began as pictures – a fish meant “fish,” a star meant “star / god / heaven.” Over centuries the pictures were rotated ~90°, simplified, and abstracted into bundles of wedges until the original picture vanished:
Figure 1. Six iconic signs from pictograph to wedge (god, water, head, fish, ox, barley). The pictographs are schematic; the “turned” stage is a literal rotation of the same strokes; the final column is the real Unicode sign. Generated by community/build_language_primer.wls.
Crucially the script is mixed – one sign can be a logogram (a whole word: 𒀭 = “god”), a syllabogram (a syllable: šar-ru for šarru), or a silent determinative (a classifier written but not read: {d} before gods, {ki} after places). And signs are polyvalent (one sign, many readings) and homophonous (many signs, one sound). Reading a tablet is therefore a constant act of disambiguation – which is exactly why machine help is useful. A starter vocabulary (cuneiform back-rendered by study/cuneiform.py):
wordTable
cuneiform | transliteration | meaning | pronunciation |
šarru | king | SHAR-roo | |
ilu | god | EE-loo | |
bītu | house | BEE-too | |
mātu | land, country | MAA-too | |
awīlu | man, person | a-WEE-loo | |
šamû | heaven, sky | sha-MOO | |
kalbu | dog | KAL-boo | |
libbu | heart, inside | LIB-boo |
And you can hear it. Akkadian has not been spoken for ~2,000 years, but its pronunciation is a careful scholarly reconstruction (pure continental vowels: a as in father, i as in machine, u as in boot). Below, every word in the table above is read aloud, voiced with ElevenLabs multilingual text-to-speech on a phonetic respelling:
And here is a longer passage – the famous opening of the Epic of Gilgamesh (Standard Babylonian prologue, Tablet I): ša naqba īmuru, išdī māti / ša kullati īdû kalāma … – “He who saw the Deep, the country's foundation; who knew all things, was wise in everything … who brought back a tale of before the Flood.”
Audio: reconstructed Akkadian, voiced with ElevenLabs multilingual TTS on a phonetic respelling (study/gen_audio_elevenlabs.py). A reconstruction of how the language likely sounded, not a recording – no one has heard Akkadian spoken for two millennia.
2. From wedges to Latin letters: transliteration
2. From wedges to Latin letters: transliteration
Assyriologists rarely work from raw signs directly; they first transliterate — write each sign with Latin letters following the ORACC / ATF convention. This is the representation our model reads. Three things to notice in the transliteration line below:
• determinatives in {curly braces} — silent classifiers, e.g. {URU} marks the following word as a city, {m} marks a male personal name, {d} a deity;
• Sumerograms in CAPITALS — LUGAL (king), KUR (land/mountain), URU (city);
• subscripts on signs (ti₂, šu₂) that disambiguate which sign with a given reading is meant.
• Sumerograms in CAPITALS — LUGAL (king), KUR (land/mountain), URU (city);
• subscripts on signs (ti₂, šu₂) that disambiguate which sign with a given reading is meant.
The same line as cuneiform signs and as ORACC transliteration. The transliteration is a faithful, reversible scholarly encoding — and a string a language model can be trained on.
3. The pipeline: tablet signs transliteration English
3. The pipeline: tablet signs transliteration English
Here is one real line from the held-out test set carried through every stage. Stage 1–2 (photograph to signs) is an OCR problem that is not solved in this project (see §7); we start from published transliteration. Stages 3–4 are what the model does.
Figure 2. One test line, end to end (stage 4a is generated live by the local model, so the exact wording may vary between runs). Note the honest detail: the model (4a) reproduces the formulaic conquest clause and the proper name correctly, but tends to mis-render the opening idiom ina metiq gerria (“in the course of my campaign”) as a fluent but invented specific — compare 4a with the human reference (4b). This confident hallucination on terse, formulaic openings is the characteristic failure mode of the model, and why its output needs an expert's eye.
3.1 The one step we don't automate: reading signs off the clay
3.1 The one step we don't automate: reading signs off the clay
That pipeline begins at transliteration – a scholar has already read the wedges off the tablet and written them in Latin letters. The step before that – photograph signs, optical character recognition for cuneiform – is the genuinely unsolved front end, and it is worth being clear about why. The forward direction is easy and we use it throughout (transliteration rendered signs, via the 14,241-entry Akkademia sign map – that is how every cuneiform glyph in this series is drawn, e.g. §12.7). The inverse – signs off a real clay photograph – is hard for reasons that compound: cuneiform is 3-D (wedges are impressions whose reading depends on raking light, not ink), the sign inventory is ~900 signs (many near-identical bundles of wedges), the forms drift over three millennia and by scribal hand, the clay is broken and abraded, and most signs are polyvalent (one shape, several readings). Research systems exist – sign-detection datasets and CNN/transformer detectors (CDLI / HeiCuBeDa, the Cuneiform Sign Detection corpus, DeepScribe) – but none is a turn-key reader, which is why scholarly editions are still hand-transliterated.
To show where it breaks – free-local, no model download – we built the last stage of an OCR system and stressed it. Render a sign as a clean glyph, then recognise it by nearest template (normalised cross-correlation) against a set of ~120 signs, while degrading the image with blur, noise, and occlusion (a stand-in for broken clay). A clean glyph classifies perfectly; accuracy then slides as the sign degrades – and this is the easy stage, with the glyph already segmented and undistorted:
Figure 2b. Why cuneiform OCR is hard, shown on its easiest stage. Top: twelve signs as clean Noto glyphs. Middle: one sign (LUGAL, “king”) under 050% occlusion – simulated breakage. Bottom: nearest-template accuracy over 120 signs collapses as the glyph blurs or breaks (noise the classifier shrugs off). And this is the last step – the real problem (raw photo segmented, de-skewed, lit glyphs) is harder still.
(* free-local: PIL renders signs from Noto Sans Cuneiform; numpy does the matching *)
python3 study/analysis/sign_ocr_demo.py (* -> community/sign_ocr.png + the accuracy curves *)
(* clean-glyph accuracy 100%; at 50% occlusion ~82%, at 6px blur ~85% – the EASY stage *)
python3 study/analysis/sign_ocr_demo.py (* -> community/sign_ocr.png + the accuracy curves *)
(* clean-glyph accuracy 100%; at 50% occlusion ~82%, at 6px blur ~85% – the EASY stage *)
The honest takeaway: the project automates everything after the signs are read, and renders signs forward perfectly, but the photographsigns reader is left to the specialist – the one place a human (or a future model) is still indispensable. Demo: study/analysis/sign_ocr_demo.py.
4. The corpus and where the tablets were found
4. The corpus and where the tablets were found
The training data comes from the ORACC project (Open Richly Annotated Cuneiform Corpus) via the Akkademia toolkit, combining five sub-corpora into ~45,000 aligned sentence pairs after de-duplication. Genre breadth matters: royal inscriptions are formulaic and easy; the letters, legal and administrative texts of the State Archives of Assyria are far more varied and make the model generalise.
Figure 3. Principal find-sites of the tablets in the corpus, on a relief map of Mesopotamia (the Tigris–Euphrates basin, modern Iraq/Syria). Colour = corpus group. The Assyrian heartland (Nineveh, Kalhu, Assur, Dur-Sharrukin) supplies the royal inscriptions and the State Archives; the Babylonian south (Babylon, Sippar, Uruk, Ur, Nippur) supplies Babylonian royal texts; Suhu sits on the middle Euphrates.
5. The model: a QLoRA fine-tune you can run on a laptop
5. The model: a QLoRA fine-tune you can run on a laptop
The model is Mistral-7B-Instruct adapted with QLoRA: the 7-billion-parameter base is frozen and quantised to 4-bit, and small low-rank adapter matrices (rank 16) are trained on top. The whole fine-tune fits in ~10 GB of GPU memory — a free Colab T4 suffices — and the trained adapter is only tens of megabytes. Merged back into the base and quantised to GGUF (q8_0, ~7.7 GB), it runs locally under Ollama on a 16 GB Mac.
Once the model is built (see the repository), translating from Wolfram is a single HTTP call to the local Ollama server — no API key, no cloud:
needs[] := Get["wolfram/Akkadian.wl"]; (* helper package in the repo *)
AkkadianTranslate["lu-u DI-mu ana PAB-ia"]
(* => "May my brother be well." (a standard Assyrian letter greeting) *)
(* under the hood it POSTs to the local server: *)
URLExecute[HTTPRequest["http://localhost:11434/api/generate",
<|"Method" -> "POST", "ContentType" -> "application/json",
"Body" -> ExportString[<|"model" -> "akkadian",
"prompt" -> "lu-u DI-mu ana PAB-ia", "stream" -> False|>,
"JSON"]|>], "RawJSON"]["response"]
AkkadianTranslate["lu-u DI-mu ana PAB-ia"]
(* => "May my brother be well." (a standard Assyrian letter greeting) *)
(* under the hood it POSTs to the local server: *)
URLExecute[HTTPRequest["http://localhost:11434/api/generate",
<|"Method" -> "POST", "ContentType" -> "application/json",
"Body" -> ExportString[<|"model" -> "akkadian",
"prompt" -> "lu-u DI-mu ana PAB-ia", "stream" -> False|>,
"JSON"]|>], "RawJSON"]["response"]
The notebook drives the exact same model you would run with `ollama run akkadian`. Wolfram becomes an interactive front end over the local pipeline.
5.0 What the model learned from – the training data, shown
5.0 What the model learned from – the training data, shown
Before the training, the data. A translation model is only as good as the pairs it sees, so it is worth looking at exactly what they are. Each example is a sentence pair – an Akkadian transliteration and its published English – from the Akkademia / ORACC gold corpus, wrapped in a three-part instruction template (a fixed system prompt, the Akkadian as the user turn, the English as the assistant turn the model learns to produce). Here is the template and three real pairs – a silver receipt, a record of named men, and a royal blessing:
Figure 3b. The actual training signal. The instruction template (with the verbatim system prompt) and three real AkkadianEnglish pairs from the 42509-pair training set. The pairs are short (median 6 words) and formulaic – receipts, letters, royal phrases – which is exactly what a small model can learn well.
How much data, and split how? The main set is 42509 training pairs (plus held-out 2166 validation / 2344 test). A de-leaked “clean” variant (36020 pairs) removes train/test overlap for the honest benchmark of §7; a genre-balanced set (50079 pairs, more literary text) is what the later Gemma fine-tune used so it could read poetry, not just ledgers (§9):
Figure 3c. The training data by the numbers. Left: the three splits (train / valid / test) for each dataset variant. Right: the Akkadian source-length distribution – most pairs are short single lines. Honest provenance: the pairs are published scholarly translations (Akkademia / ORACC), not machine output – the only model-distilled portion is a small (~5%) literary slice added to the genre-balanced set, audited in §9.
(* the pairs are built by make_finetuning_data.py from the Akkademia NMT corpus *)
{"messages": [{"role": "system", "content": "You are an expert Assyriologist. Translate the given Akkadian into clear, accurate English. Output only the English translation."},
{"role": "user", "content": "<Akkadian transliteration>"},
{"role": "assistant", "content": "<English translation>"}]}
(* -> finetuning_data/{train,valid,test}.jsonl, the input to train_lora.py *)
{"messages": [{"role": "system", "content": "You are an expert Assyriologist. Translate the given Akkadian into clear, accurate English. Output only the English translation."},
{"role": "user", "content": "<Akkadian transliteration>"},
{"role": "assistant", "content": "<English translation>"}]}
(* -> finetuning_data/{train,valid,test}.jsonl, the input to train_lora.py *)
Data tool: instruction-formatted parallel text – the substrate of every fine-tune. Built by make_finetuning_data.py; digest study/analysis/training_data.py.
5.1 How it was trained
5.1 How it was trained
Figure 3d. QLoRA in one picture. The frozen 4-bit weight W plus the trainable low-rank detour B A (only A, B learn). The table computes the adapter size over the seven target projections × 32 layers: ~42 M trainable parameters – about 0.6% of the 7.2 B base, which is why the saved adapter is only tens of megabytes and the fine-tune runs on a free Colab T4.
d = 4096; ff = 14336; kv = 1024; r = 16; (* Mistral-7B layer shapes *)
loraParams[din_, dout_] := r (din + dout); (* A is r×d (down), B is d×r (up) *)
perLayer = Total[loraParams @@@ {{d,d},{d,kv},{d,kv},{d,d},{d,ff},{d,ff},{ff,d}}];
32 perLayer (* => ~42 million trainable, ~0.6% of the 7.2-billion base *)
loraParams[din_, dout_] := r (din + dout); (* A is r×d (down), B is d×r (up) *)
perLayer = Total[loraParams @@@ {{d,d},{d,kv},{d,kv},{d,d},{d,ff},{d,ff},{ff,d}}];
32 perLayer (* => ~42 million trainable, ~0.6% of the 7.2-billion base *)
Wolfram tool: a Graphics schematic + a live parameter count – the arithmetic of why fine-tuning a 7 B model is cheap. Figure in community/analysis.wl (loraFigure).
Why so little training does so much. Fine-tuning only has to teach the mapping from transliteration to English — the base model already speaks fluent English and has seen some transliteration in pre-training — which is why a 7B model reaches near-frontier quality on a corpus of only ~10⁴ pairs, in one epoch. One epoch (not three) is deliberate: on a mostly formulaic corpus, more passes risk over-fitting. The exact settings:
Table 5. Training configuration (full detail and all the Colab gotchas are in the repository's notebook and README §3).
For full reproducibility, the exact training call – every argument – is kept in an importable script so it never drifts from the code. Both fine-tunes (Mistral-7B and Gemma-4-12B) share train_lora.py; only --base and --data differ. In the notebook, evaluating it renders the recipe in place:
Table 5b. The reproducible training recipe, imported from community/training_detail.wl (the two exact commands, the underlying train_and_eval call with all defaults, the QLoRA hyper-parameters, and the datatrainexportrun pipeline).
And the heart of it, copied straight from train_lora.py – the 4-bit load, the LoRA adapter, and the trl trainer that does the actual work:
model = AutoModelForCausalLM.from_pretrained( # Mistral-7B-Instruct-v0.2
base, quantization_config=BitsAndBytesConfig( # 4-bit NF4 (QLoRA)
load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=compute_dtype, bnb_4bit_use_double_quant=True),
device_map="auto")
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, bias="none",
task_type="CAUSAL_LM",
target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"])
cfg = SFTConfig(output_dir=outdir, num_train_epochs=1, learning_rate=2e-4,
per_device_train_batch_size=8, gradient_accumulation_steps=2, # effective batch 16
warmup_ratio=0.03, lr_scheduler_type="cosine", logging_steps=20,
bf16=bf16_ok, fp16=not bf16_ok, max_length=768,
gradient_checkpointing=True, report_to="none")
SFTTrainer(model=model, args=cfg, train_dataset=ds["train"],
eval_dataset=ds["validation"], peft_config=lora, processing_class=tok).train()
base, quantization_config=BitsAndBytesConfig( # 4-bit NF4 (QLoRA)
load_in_4bit=True, bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=compute_dtype, bnb_4bit_use_double_quant=True),
device_map="auto")
lora = LoraConfig(r=16, lora_alpha=32, lora_dropout=0.05, bias="none",
task_type="CAUSAL_LM",
target_modules=["q_proj","k_proj","v_proj","o_proj","gate_proj","up_proj","down_proj"])
cfg = SFTConfig(output_dir=outdir, num_train_epochs=1, learning_rate=2e-4,
per_device_train_batch_size=8, gradient_accumulation_steps=2, # effective batch 16
warmup_ratio=0.03, lr_scheduler_type="cosine", logging_steps=20,
bf16=bf16_ok, fp16=not bf16_ok, max_length=768,
gradient_checkpointing=True, report_to="none")
SFTTrainer(model=model, args=cfg, train_dataset=ds["train"],
eval_dataset=ds["validation"], peft_config=lora, processing_class=tok).train()
Reproducibility specifics (and one honest gap). Base: mistralai/Mistral-7B-Instruct-v0.2 (the Gemma-4 fine-tune just swaps --base). Stack: transformers + peft + trl + bitsandbytes + datasets + accelerate, installed at their current releases on the free Colab T4 image – the notebooks pin no versions, so a re-run tracks upstream. Data: ~45,000 Akkadian–English chat pairs after exact-match de-duplication, split train / validation / test. Seed: the HuggingFace default (train_lora.py sets none). One epoch (more risks over-fitting the formulaic corpus). The gap: with report_to="none" the loss is printed every 20 steps but not logged to a file, so no training-loss curve was saved – the one piece of evidence an ML reader would want that this project does not have.
5.2 The reproducible pipeline
5.2 The reproducible pipeline
Five stages take raw ORACC corpora to a model running on your laptop. Each is one script/command; the whole thing is orchestrated by the project's Colab notebook (GPU for the training stage only).
# 1. build the parallel corpus -> finetuning_data/{train,valid,test}.jsonl
python make_finetuning_data.py
# 2. QLoRA fine-tune (GPU: Colab/A100) -> adapter + merged fp16 model
python train_lora.py --base mistralai/Mistral-7B-Instruct-v0.2 \
--data finetuning_data --outdir akkadian-lora --epochs 1
# 3. convert merged model to GGUF q8_0 (llama.cpp)
python llama.cpp/convert_hf_to_gguf.py akkadian-lora-merged \
--outfile akkadian-q8_0.gguf --outtype q8_0
# 4. register with Ollama (Modelfile carries the Mistral [INST] template)
cd FinalModel && ollama create akkadian -f Modelfile
# 5. run it
ollama run akkadian "lu-u DI-mu ana PAB-ia"
python make_finetuning_data.py
# 2. QLoRA fine-tune (GPU: Colab/A100) -> adapter + merged fp16 model
python train_lora.py --base mistralai/Mistral-7B-Instruct-v0.2 \
--data finetuning_data --outdir akkadian-lora --epochs 1
# 3. convert merged model to GGUF q8_0 (llama.cpp)
python llama.cpp/convert_hf_to_gguf.py akkadian-lora-merged \
--outfile akkadian-q8_0.gguf --outtype q8_0
# 4. register with Ollama (Modelfile carries the Mistral [INST] template)
cd FinalModel && ollama create akkadian -f Modelfile
# 5. run it
ollama run akkadian "lu-u DI-mu ana PAB-ia"
Everything is in the repository; the notebook is resumable and writes all artifacts to Google Drive, so a timed-out runtime just resumes.
Repeat it yourself – on a free GPU. You do not need our hardware. The training runs on a free Google Colab T4: open one of the turn-key notebooks in the repository, set Runtime Change runtime type T4 GPU, and Run all. The notebooks wrap exactly the pipeline above (train_lora.py), write every artifact to your Google Drive, and resume after a timeout:
# turn-key Colab notebooks in the repo – open in Google Colab, T4 GPU, Run all:
akkadian_lora_colab.ipynb (* Mistral-7B QLoRA – the model used in this notebook *)
Finetune_Gemma_colab.ipynb (* the genre-balanced Gemma-4-12B fine-tune *)
Finetune_GPTOSS_colab.ipynb (* gpt-oss variant *)
# each runs the same chain, all scripts in the repo:
# make_finetuning_data.py -> train_lora.py -> GGUF (llama.cpp) -> Ollama Modelfile
# the training DATA regenerates from public ORACC/eBL corpora, so the whole chain reproduces.
akkadian_lora_colab.ipynb (* Mistral-7B QLoRA – the model used in this notebook *)
Finetune_Gemma_colab.ipynb (* the genre-balanced Gemma-4-12B fine-tune *)
Finetune_GPTOSS_colab.ipynb (* gpt-oss variant *)
# each runs the same chain, all scripts in the repo:
# make_finetuning_data.py -> train_lora.py -> GGUF (llama.cpp) -> Ollama Modelfile
# the training DATA regenerates from public ORACC/eBL corpora, so the whole chain reproduces.
Table 5c. The actual fine-tuning notebooks shipped in the repository. The entire chain – build the parallel data, QLoRA-train on a free T4, convert to GGUF, serve with Ollama – is reproducible from scratch; nothing here needs a paid runtime or our machines.
The last mile: the Modelfile, verbatim. Stage 4 hands the GGUF to Ollama via a Modelfile – the recipe that tells the server how to wrap the raw weights. It is worth showing in full, because it is where the model becomes usable: it pins the Mistral [INST] chat template the model was trained against, restates the system prompt, and – crucially for translation – sets temperature 0 so the output is deterministic (we want the one best reading, not a creative sample):
# FinalModel/Modelfile – register with: ollama create akkadian -f Modelfile
FROM ./akkadian-q8_0.gguf
TEMPLATE """[INST] {{ if .System }}{{ .System }}
{{ end }}{{ .Prompt }} [/INST] """
SYSTEM "You are an expert Assyriologist. Translate the given Akkadian transliteration into clear, accurate English. Output only the English translation."
PARAMETER temperature 0
PARAMETER stop "[/INST]"
FROM ./akkadian-q8_0.gguf
TEMPLATE """[INST] {{ if .System }}{{ .System }}
{{ end }}{{ .Prompt }} [/INST] """
SYSTEM "You are an expert Assyriologist. Translate the given Akkadian transliteration into clear, accurate English. Output only the English translation."
PARAMETER temperature 0
PARAMETER stop "[/INST]"
What it costs: nothing. The fine-tune is one epoch over the ~42.5 k pairs on a free Colab T4 (16 GB) – well under an hour of GPU, \$0. The merged model converts to a ~7.7 GB GGUF (q8_0) that runs on any 16 GB Mac under Ollama, also \$0 and offline. The only spend in the whole project was the frontier API drafting of §8 / §12 (~\$150) – the owned translator itself is free to build and free to run.
6. More example translations
6. More example translations
One example is an anecdote; here are several, all from the held-out test set (the model never saw these during training): the model's English (red) beside the published scholarly translation (blue). Read down the genres — royal deeds, treaties, administrative tallies — and judge for yourself where it is faithful and where it drifts.
Figure 4. Model output vs scholarly reference on held-out lines. The model is fluent and usually faithful on in-genre text; on terse or broken lines it can smooth over uncertainty. These are generated by the local model; exact wording may vary by run.
6.1 Try it yourself – the translator, live in the notebook
6.1 Try it yourself – the translator, live in the notebook
Reading about a translator is one thing; here is yours. The widget below is a self-contained DynamicModule holding 24 real held-out test lines – page through them or hit random and see, for each, the Akkadian transliteration, the local model's English, and the scholar's gold. The ✓ / badge flags whether the line is in the de-leaked clean set or sits near a training line – the honesty of §7, made clickable. (It stays live in the saved notebook; the same model answers any new line via the local Ollama server, as in §5.)
Figure 4b. A live reader over held-out test lines (24 of 200): Akkadian the owned model the scholar's translation, with a leakage badge. Outputs are the real akkadian model's; data study/analysis/translate_demo.py.
7. How good is it? Leakage-aware benchmarking
7. How good is it? Leakage-aware benchmarking
A translation model is only as trustworthy as its evaluation. We score with chrF (character-level F-score, robust to the editorial brackets and short fragments of scholarly translations) and BLEU, using sacreBLEU, on a held-out test split — and we take two extra steps for honesty.
(1) Confidence intervals. Each score is reported with a bootstrap 95% CI (resampling the held-out lines), so we can tell a real difference from noise.
(2) A leakage-aware “clean” subset. Because the split is line-level, near-duplicate formulaic lines can appear in both train and test and inflate the score. We flag every test line that is a near-duplicate of any training line (token-Jaccard ≥ 0.7) and report the metrics on the full set and on the clean subset. The gap between them is the “leakage premium.”
(2) A leakage-aware “clean” subset. Because the split is line-level, near-duplicate formulaic lines can appear in both train and test and inflate the score. We flag every test line that is a near-duplicate of any training line (token-Jaccard ≥ 0.7) and report the metrics on the full set and on the clean subset. The gap between them is the “leakage premium.”
Figure 5. Quality vs two reference points. The fine-tuned model scores well above the 2023 neural-MT BLEU baseline (~34 vs 15) – though on a harder, mixed-genre test set than the 2023 single-genre one, so this is a reference point, not a head-to-head – and approaches a frontier LLM used few-shot. But 33% of test lines are near-duplicates of training material; on the clean subset the honest figure is chrF ≈ 47 / BLEU ≈ 31 — still well above the 2023 baseline, but below the frontier model. Whiskers are bootstrap 95% CIs.
Table 7. The fine-tuned rows are read live from the project's benchmark and reproduce with analysis/benchmark.py. * Caveat: the ConvNMT and blind-LLM rows are from the 2023 study's 50-sentence RINAP royal-inscription set (BLEU 29.7 / chrF 55.5 for the blind LLM) — an easier, single-genre slice than our mixed-genre held-out set, so they are reference points, not a head-to-head on identical lines.
What chrF and BLEU actually measure – and why chrF here. Both compare the model's English to the scholar's. BLEU counts overlapping whole words (n-grams); chrF counts overlapping character n-grams. That difference matters for a heavily-inflected language. Take one held-out line where the model gets the meaning but not the exact endings – Wolfram scores it both ways:
Figure 5b. Why chrF is the fairer metric. A right-meaning, wrong-inflection line: word-level BLEU discards “restores” and “temples” wholesale for their endings, while character-level chrF keeps the shared stems. Akkadian inflects far more than English (šarrum / šarrim / šarram = “king” by case), so chrF – computed live here in Wolfram – is the less brittle score, and the one we headline. The leakage and bootstrap-CI safeguards above then keep even that score honest.
(* chrF = character n-gram F-score, averaged over n=1..3; BLEU = word n-gram precision *)
charNgrams[s_, n_] := StringJoin /@ Partition[Characters[StringDelete[s, " "]], n, 1];
metricsFigure[] (* the worked ref-vs-hypothesis example above *)
charNgrams[s_, n_] := StringJoin /@ Partition[Characters[StringDelete[s, " "]], n, 1];
metricsFigure[] (* the worked ref-vs-hypothesis example above *)
Wolfram tool: a from-scratch chrF / BLEU computation on one line – the metric made transparent. Figure metricsFigure in community/analysis.wl; the full benchmark is analysis/benchmark.py.
7.1 Fine-tuning vs frontier models, on identical lines
7.1 Fine-tuning vs frontier models, on identical lines
Is the fine-tune actually doing the work — and how does a small, local, fine-tuned model stack up against the best general models? We scored six systems on the exact same held-out lines: our two local fine-tunes (the original Mistral-7B and the new genre-balanced Gemma-4-12B), two off-the-shelf locals (gemma4 8B, gpt-oss:20b 21B, both few-shot via Ollama), plus two frontier APIs used vanilla few-shot (claude-opus-4-8, gpt-5.5). Same system prompt, same held-out lines, same metric — so the effect of fine-tuning is isolated: the same Gemma-4 family appears both off-the-shelf (few-shot) and fine-tuned, and the gap between them is what our training buys.
Figure 6. Six systems, the same held-out lines, with bootstrap 95% CIs. The result is more interesting than “bigger is better”: frontier APIs used vanilla (claude-opus-4-8, gpt-5.5) lead on chrF — no training at all — while our two local fine-tunes lead on BLEU (they reproduce the scholarly phrasing they were trained on). The decisive contrast is within the Gemma-4 family: off-the-shelf few-shot it is weak (chrF ~22), but the same model fine-tuned leaps to frontier-adjacent — right beside the Mistral fine-tune — which is the entire case for owning and training a local model. (gpt-oss, a reasoning model, over-thinks terse lines and returns empty/garbled output on ~1/3 of them — * unreliable, shown for honesty.)
The honest takeaways – is a worse-but-free model you own worth it?
• For a model you OWN — local, free, offline, private — fine-tuning is essential. It lifts a local model from unusable (~chrF 22) to frontier-adjacent (~chrF 45), and it matches the scholars' phrasing best (top BLEU). That is the whole value proposition of this project, and it is what §8–§9 then put to work.
• The honest flip side: for raw quality alone, a frontier model needs no fine-tuning — claude-opus-4-8 and gpt-5.5 are already excellent at Akkadian few-shot. The case for the owned model is not that it beats them on chrF; it is that you run it free, offline, and forever, on your own laptop.
• Reasoning models fit this task badly few-shot: terse one-line translation fights their think-first prior.
• For a model you OWN — local, free, offline, private — fine-tuning is essential. It lifts a local model from unusable (~chrF 22) to frontier-adjacent (~chrF 45), and it matches the scholars' phrasing best (top BLEU). That is the whole value proposition of this project, and it is what §8–§9 then put to work.
• The honest flip side: for raw quality alone, a frontier model needs no fine-tuning — claude-opus-4-8 and gpt-5.5 are already excellent at Akkadian few-shot. The case for the owned model is not that it beats them on chrF; it is that you run it free, offline, and forever, on your own laptop.
• Reasoning models fit this task badly few-shot: terse one-line translation fights their think-first prior.
So: fine-tune gemma or gpt-oss? Worth it precisely to get a better local model — not to beat claude, but to push the free/offline option from ~22 toward (or past) our Mistral's ~45. We did exactly this with Gemma-4 (see the literary result in §9): you cannot fine-tune the Ollama GGUF (it is inference-only); you fine-tune the HF base with the same recipe (train_lora.py --base unsloth/gemma-4-12b-it — the ungated Unsloth mirror, no HF token needed, or --base openai/gpt-oss-20b --lora-target all-linear with MXFP4 dequantisation), then merge GGUF Ollama. It needs a CUDA GPU (an Apple-Silicon Mac cannot run QLoRA), so it is a Colab job — turn-key notebooks Finetune_Gemma_colab.ipynb / Finetune_GPTOSS_colab.ipynb are provided.
8. Translating unknown texts at scale
8. Translating unknown texts at scale
The examples so far had a published translation to check against. The real prize is the hundreds of thousands of tablets that have only ever been transliterated but never published in translation. Because the model is local and free to run, it can be turned loose on them to produce draft translations for scholars to vet. The repository's concurrent, resumable batch runner does exactly this; below are real drafts for lines from an untranslated corpus (the ATAE archive), which have no reference translation at all — drafted by both local fine-tunes (the original Mistral and the new genre-balanced Gemma-4) so their readings can be compared at a glance.
At full scale. The complete untranslated ATAE corpus — all 21,592 lines — has now been drafted end to end by claude-opus-4-8 (see drafts/atae_claude.jsonl), turning a previously dark archive into a first-pass English draft an Assyriologist can review. The small table below is a sample; the run covers the whole thing.
Figure 7. Genuinely unknown text: Mistral (red) and the new Gemma-4 (orange) drafts for previously untranslated lines (tablet P-numbers shown). These are unvetted machine output — useful as a first pass for an Assyriologist, not citable translations. This is the intended production workflow, and the cost strategy: draft at scale with the free, local fine-tunes, and spend frontier-API budget (claude/gpt) only on the lines flagged as interesting or where the two locals disagree.
What it cannot (yet) do, stated plainly:
• Photographs. The hard, unsolved front end is image signs (OCR of impressed 3-D, often broken wedges). The pipeline here starts from transliteration.
• Hallucination. On short or broken lines it can invent fluent, confident, wrong specifics (as in Fig. 3). It does not flag its own uncertainty.
• Genre skew. Strongest on royal/administrative/legal text; weaker on literary and omen texts.
• Assistive, not autonomous. Output needs an Assyriologist's review, especially on damaged or out-of-genre text.
• Photographs. The hard, unsolved front end is image signs (OCR of impressed 3-D, often broken wedges). The pipeline here starts from transliteration.
• Hallucination. On short or broken lines it can invent fluent, confident, wrong specifics (as in Fig. 3). It does not flag its own uncertainty.
• Genre skew. Strongest on royal/administrative/legal text; weaker on literary and omen texts.
• Assistive, not autonomous. Output needs an Assyriologist's review, especially on damaged or out-of-genre text.
9. Beyond administrative text — a literary test (the Epic of Gilgamesh)
9. Beyond administrative text — a literary test (the Epic of Gilgamesh)
Everything so far has been royal and administrative prose — the genres the original local model was trained on. But the deeper prize of cuneiform is its literature, and the most famous of all is the Epic of Gilgamesh. As a stress test we took the opening hymn of the Standard Babylonian recension (Tablet I, lines 29–48) and asked four translators for English: our original Mistral-7B fine-tune (trained only on royal/administrative text), our new Gemma-4-12B fine-tune (trained on a genre-balanced corpus that adds literary text, then merged to GGUF and run locally under Ollama exactly like the Mistral model), claude-opus-4-8 used vanilla, and the published scholarly gold standard (A. R. George's edition, as carried by the electronic Babylonian Library, eBL).
Figure 8. The opening of the Epic of Gilgamesh, four ways — two local fine-tunes, one frontier API, and the scholarly gold. The original Mistral fine-tune (red) collapses: trained only on administrative tallies, it hallucinates ledger content onto poetry — “100,000 (men) his army”, “10 litres of oil”. The new Gemma-4 fine-tune (orange) fixes exactly this: trained on a genre-balanced corpus, it now reads the hymn as poetry — “The hero born in Uruk is a wild bull on the rampage”, “The wild bull of Lugalbanda (was) Gilgameš, a hero of strength” — closely tracking claude (blue) and George's published edition (green) on this passage, while still running free and offline. (One short, famous passage – an unvetted draft, not a verified collation.) This is the payoff of the genre-balanced corpus: literary competence that the admin-only model simply did not have, now in a model you own.
The failure, isolated. It is worth seeing the breakdown by line, because it is so systematic. Below are five consecutive lines of the hymn – the same model architecture and recipe, differing only in what genres the training data covered. The admin-only fine-tune does not produce bad poetry; it produces a ledger, reflexively reaching for the sheep, oil and silver of the archives it learned on:
Figure 8b. Why genre balance matters, line by line. The opening of Gilgamesh (SB I 29–33): George's edition, then the admin-only Mistral fine-tune (red – “100 sheep”, “10 litres of oil”, “100,000 men”), then the genre-balanced Gemma-4 (green – reading the verse the edition records). Same architecture, same hyper-parameters; only the training mix differs. Unvetted drafts, but only one is in the right register.
And it is not just one passage – the number says the same thing. On a held-out literary benchmark (Gilgamesh and other eBL-gold poetry, scored by chrF), the admin-only Mistral fine-tune manages only 13.6; the genre-balanced Gemma-4 fine-tune reaches 27.7 – a clean 2× – closing most of the gap to a frontier model (36.3) on the hardest register, for free and offline:
Figure 8c. The payoff, quantified. Literary-text chrF: the admin-only fine-tune (13.6) vs the genre-balanced one (27.7, ~2×) vs a frontier model (36.3). Genre balance roughly doubles the owned model on poetry – the §9 thesis as a measurement, not just an anecdote. (Gold benchmark vs George/eBL; literary is hard for every system, which is why even the frontier sits at 36.)
The point of §9: a model is its training data. Match the data to the genre and a free, owned model reads poetry; mismatch it and it hallucinates the only world it knew. Digest study/analysis/gilgamesh_errors.py; figure in community/analysis.wl.
— End of Part I — Building the translator —
— End of Part I — Building the translator —
In Part I we went from raw clay to a working tool: what cuneiform and transliteration are; how a tablet becomes signs, then Latin letters, then English; how a QLoRA fine-tune of Mistral-7B (and later a genre-balanced Gemma-4) was trained to translate Akkadian offline on a laptop; an honest, leakage-aware benchmark (chrF ≈ 45–47); and a literary test on the Epic of Gilgamesh. We now have a translator we own.
Coming next – Part II — A research instrument. With the translator in hand, we stop measuring it and start using it – as an instrument. Part II loads a metadata-rich corpus into Wolfram Language and investigates the flood traditions, the cast of Gilgamesh and the king's correspondents as networks, the spread of texts, omen logic, the geometry of Noah's Ark, an ancient board game made playable, and a 763 BC eclipse used to anchor the chronology.
Coming next – Part II — A research instrument. With the translator in hand, we stop measuring it and start using it – as an instrument. Part II loads a metadata-rich corpus into Wolfram Language and investigates the flood traditions, the cast of Gilgamesh and the king's correspondents as networks, the spread of texts, omen logic, the geometry of Noah's Ark, an ancient board game made playable, and a 763 BC eclipse used to anchor the chronology.
Reproducible end to end – data build, QLoRA training, GGUF/Ollama export, benchmark, batch translator and this notebook (community/build_notebook.wls) are all in the repository. Text data: ORACC (CC BY-SA 3.0) and the electronic Babylonian Library / eBL (CC BY-NC-SA 4.0), reused with attribution for non-commercial research; the derived data and this notebook are shared under CC BY-NC-SA 4.0. Method: Akkademia; Gutherz et al. (2023). Machine translations are leads for specialists, not citable readings.
CITE THIS NOTEBOOK
CITE THIS NOTEBOOK
The computational assyriologist - Part I: translating Akkadian cuneiform to English with a local LLM
by Marco Thiel
Wolfram Community, STAFF PICKS, June 17, 2026
https://community.wolfram.com/groups/-/m/t/3735063
by Marco Thiel
Wolfram Community, STAFF PICKS, June 17, 2026
https://community.wolfram.com/groups/-/m/t/3735063