Loading Real Data

The two-phase loader contract, EventStores, timeline ids, and what a loader builds for you

Loading Real Data

What you will build

You will load Chopin’s Étude Op. 10 No. 3 from MusicXML and a note table exported by ms3, a Python toolkit for working with score corpora. The TSV has one note per tab-separated row; the loading-data guide compares its Ms3Loader with the MusicXML loaders. You will finish with the same kind of you previously built by hand, including timelines and an exact conversion derived from the file.

Before you start

Complete the tutorial on s, Events on a Timeline, first.

from fractions import Fraction

import pandas as pd

from timetoalign import (
    Coordinate,
    CsvLoader,
    EventStore,
    Loader,
    Ms3Loader,
    Music21Loader,
    PartituraLoader,
    PerformanceMidiLoader,
    ScoreMidiLoader,
    TimeUnit,
    TsvLoader,
)
from timetoalign.testdata import ensure_data

VIENNA_DATA = ensure_data("vienna_1x22")
MIDI_DATA = ensure_data("midi")

From hand-built to real

The first three tutorials taught the model by constructing timelines by hand; here, files replace those construction steps. Nothing new appears in the result—only in how it is produced.

musicxml_path = VIENNA_DATA / "Chopin_op10_no3.musicxml"
ms3_path = VIENNA_DATA / "ms3" / "chopin_op10_no3.notes.tsv"
score_midi_path = MIDI_DATA / "score" / "rachmaninoff_piano.mid"
performance_midi_path = MIDI_DATA / "performance" / "rachmaninoff_perf.mid"
source_files = {
    "MusicXML": musicxml_path,
    "ms3 TSV": ms3_path,
    "score MIDI": score_midi_path,
    "performance MIDI": performance_midi_path,
}
source_files

{'MusicXML': PosixPath('/home/laser/git/tta/timetoalign/tests/data/vienna_1x22/Chopin_op10_no3.musicxml'),
 'ms3 TSV': PosixPath('/home/laser/git/tta/timetoalign/tests/data/vienna_1x22/ms3/chopin_op10_no3.notes.tsv'),
 'score MIDI': PosixPath('/home/laser/git/tta/timetoalign/tests/data/midi/score/rachmaninoff_piano.mid'),
 'performance MIDI': PosixPath('/home/laser/git/tta/timetoalign/tests/data/midi/performance/rachmaninoff_perf.mid')}

These are real corpus files resolved by ensure_data(). They are inputs to loaders, not a new kind of timeline result.

The two-phase contract

loader.load(*sources) ingests files and returns the loader; only loader.create_timeline() builds objects in the musical , and it never takes a file path. from_file() is the shorthand for creating a loader and performing the first phase.

two_phase_loader = PartituraLoader()
two_phase_loader.load(musicxml_path)
preview_timeline = two_phase_loader.create_timeline()

partitura_loader = PartituraLoader.from_file(musicxml_path)
phase_objects = {
    "loaded, before timeline creation": two_phase_loader,
    "phase-two result": preview_timeline,
    "from_file result": partitura_loader,
    "follows Loader contract": isinstance(partitura_loader, Loader),
}
phase_objects

{'loaded, before timeline creation': PartituraLoader(sources=1, events=547, unit=quarters),
 'phase-two result': ContinuousLogicalTimeline(id='clt1', length=83/2, unit=quarters, events=0, children=4, cmaps=3),
 'from_file result': PartituraLoader(sources=1, events=547, unit=quarters),
 'follows Loader contract': True}

The first object is a loader holding one parsed source, while the phase-two object is a timeline. from_file() returns another loaded loader, not a timeline; the remaining sections use that shorthand loader.

The EventStore

loader.store holds the parser’s separate note, measure, control, and annotation tables before any timeline exists. This inventory uses store.summary() for counts and reads each table’s exact coordinate range.

score_store = partitura_loader.store
store_contract = isinstance(score_store, EventStore)
store_summary = score_store.summary()
store_tables = store_summary["tables"]

store_rows = []
for category, event_data in score_store.items():
    first_position, last_position = event_data.coordinate_range()
    store_rows.append(
        {
            "category": category,
            "count": store_tables[category]["count"],
            "unit": event_data.unit,
            "first position": Coordinate(first_position, event_data.unit),
            "last position": Coordinate(last_position, event_data.unit),
        }
    )

store_inventory = pd.DataFrame(store_rows)
store_inventory_view = store_inventory.style.format(
    {"first position": repr, "last position": repr}
)
store_inventory_view = store_inventory_view.set_caption(
    f"Parsed table inventory — EventStore contract: {store_contract}"
)
store_inventory_view

Table 1: Parsed table inventory — EventStore contract: True

	category	count	unit	first position	last position
0	notes	498	quarters	Coordinate(Fraction(0, 1), quarters)	Coordinate(Fraction(83, 2), quarters)
1	measures	22	quarters	Coordinate(Fraction(0, 1), quarters)	Coordinate(Fraction(83, 2), quarters)
2	controls	22	quarters	Coordinate(Fraction(0, 1), quarters)	Coordinate(Fraction(40, 1), quarters)
3	annotations	5	quarters	Coordinate(Fraction(0, 1), quarters)	Coordinate(Fraction(65, 2), quarters)

Each row is a parser table with its count, unit, and exact range. The stored endpoints are Fraction values; placing them in Coordinate objects keeps both the rational value and quarters unit visible. summary() is a lightweight digest rather than a coordinate accessor: it reports counts, units and plain-float extents for a quick look at what a store holds, so it is used here for counts while the coordinates come from the typed accessors. The object also satisfies the public EventStore contract: isinstance(score_store, EventStore) is True.

One parsed record

A store table contains parser records rather than timeline children. Select one note now so that its transformation during timeline creation is concrete.

stored_note_data = score_store.notes
stored_note_frame = stored_note_data.to_dataframe()
stored_note_example = stored_note_frame.loc[
    [0], ["id", "name", "event_type", "start", "end"]
].copy()
stored_note_id = stored_note_example.loc[0, "id"]

for position_column in ("start", "end"):
    stored_note_example[position_column] = [
        Coordinate(value, stored_note_data.unit)
        for value in stored_note_example[position_column]
    ]

stored_note_view = stored_note_example.style.format({"start": repr, "end": repr})
stored_note_view

	id	name	event_type	start	end
0	note:000001	B3	Note	Coordinate(Fraction(0, 1), quarters)	Coordinate(Fraction(1, 2), quarters)

This is the first raw note record: B3 begins at zero quarters and ends at one half. Its local id is note:000001. The timeline will preserve these musical fields while placing the record under a queryable notes child.

Making the timeline

create_timeline(uid=...) turns the loaded store into the domain object. The optional uid supplies identity; a separate name can remain readable to a human.

score_timeline = partitura_loader.create_timeline(uid="score:clt1")
score_timeline.name = "Chopin, Étude Op. 10 No. 3"
timeline_identity = {
    "automatic id": preview_timeline.id,
    "role-prefixed id": score_timeline.id,
    "human-readable name": score_timeline.name,
}
timeline_identity

{'automatic id': 'clt1',
 'role-prefixed id': 'score:clt1',
 'human-readable name': 'Chopin, Étude Op. 10 No. 3'}

clt1 means the first continuous logical timeline created by that loader; score:clt1 adds an optional role prefix. The descriptive title is metadata, not identity, so changing it would not change the timeline’s id.

What you got for free

The loader has already separated each event category into a child timeline and attached a quarters-to-ticks .

child_ids = score_timeline.list_children()
stored_categories = tuple(store_inventory["category"])
quarters_to_ticks = score_timeline.get_conversion_map(TimeUnit.ticks)

quarter_position = score_timeline.make_coordinate(Fraction(13, 2))
tick_position = score_timeline.convert_to(quarter_position, TimeUnit.ticks)
loader_payoff = {
    "children": child_ids,
    "stored categories": stored_categories,
    "quarters-to-ticks map": quarters_to_ticks,
    "quarter position": quarter_position,
    "tick position": tick_position,
}
loader_payoff

{'children': ['notes', 'measures', 'controls', 'annotations'],
 'stored categories': ('notes', 'measures', 'controls', 'annotations'),
 'quarters-to-ticks map': ScalarMap(scalar=480, source_unit=quarters, target_unit=ticks),
 'quarter position': Coordinate(Fraction(13, 2), quarters),
 'tick position': Coordinate(3120, ticks)}

The child ids match the four stored categories. The map converts the library’s Coordinate(Fraction(13, 2), quarters) directly to Coordinate(3120, ticks); no rational value is reconstructed for display. Measures are available as the measures child, but this score loader does not also materialize them as objects.

Querying every child

A loaded timeline uses the same get_events() query introduced previously. Ask the parent for every event, then select one representative row from each child for inspection.

loaded_events = score_timeline.get_events()
loaded_event_frame = loaded_events.to_dataframe()
event_examples = loaded_event_frame.drop_duplicates(subset="source_timeline")
event_examples = event_examples.loc[
    :, ["source_timeline", "event_type", "name", "start"]
].copy()
event_examples = event_examples.reset_index(drop=True)
event_examples["start"] = [
    score_timeline.make_coordinate(value) for value in event_examples["start"]
]

event_examples_view = event_examples.style.format({"start": repr})
event_examples_view = event_examples_view.set_caption(
    f"One row from each child among {len(loaded_events)} loaded events"
)
event_examples_view

Table 2: One row from each child among 547 loaded events

	source_timeline	event_type	name	start
0	notes	Note	B3	Coordinate(Fraction(0, 1), quarters)
1	measures	Measure	1	Coordinate(Fraction(0, 1), quarters)
2	controls	TimeSignature	TimeSignature	Coordinate(Fraction(0, 1), quarters)
3	annotations	Text	Lento, ma non troppo	Coordinate(Fraction(0, 1), quarters)

The four rows come from notes, measures, controls, and annotations, so the parent query really does gather events from every child. The selected fields show each event’s source, type, name, and exact start coordinate; the caption records that the complete result contains 547 events.

Narrowing by event type

The same query can keep only notes. Convert a few selected fields to a table so that both note content and exact positions remain visible.

loaded_notes = score_timeline.get_events(event_type="Note")
loaded_note_frame = loaded_notes.to_dataframe()
note_examples = loaded_note_frame.loc[
    :3, ["id", "name", "event_type", "start", "end", "staff"]
].copy()

for position_column in ("start", "end"):
    note_examples[position_column] = [
        score_timeline.make_coordinate(value)
        for value in note_examples[position_column]
    ]

timeline_note_id = f"notes:{stored_note_id}"
assert note_examples.loc[0, "id"] == timeline_note_id
note_examples_view = note_examples.style.format({"start": repr, "end": repr})
note_examples_view = note_examples_view.set_caption(
    f"First four of {len(loaded_notes)} Note events"
)
note_examples_view

Table 3: First four of 498 Note events

	id	name	event_type	start	end	staff
0	notes:note:000001	B3	Note	Coordinate(Fraction(0, 1), quarters)	Coordinate(Fraction(1, 2), quarters)	1
1	notes:note:000002	E4	Note	Coordinate(Fraction(1, 2), quarters)	Coordinate(Fraction(1, 1), quarters)	1
2	notes:note:000003	G♯3	Note	Coordinate(Fraction(1, 2), quarters)	Coordinate(Fraction(3, 4), quarters)	1
3	notes:note:000004	E2	Note	Coordinate(Fraction(1, 2), quarters)	Coordinate(Fraction(3, 4), quarters)	2

These are actual note rows, not an EventData summary. The first is the B3 stored earlier: timeline creation preserves its fields and scopes its local id note:000001 as notes:note:000001. The table shows four of the 498 rows kept by the familiar event_type filter.

The same music, a second way

PartituraLoader.from_file() read the MusicXML above; now Ms3Loader.from_file() reads the neighboring note TSV. Comparing note counts tests whether the two provenances describe the same musical content.

ms3_loader = Ms3Loader.from_file(ms3_path)
ms3_timeline = ms3_loader.create_timeline(uid="ms3:clt1")
ms3_events = ms3_timeline.get_events()
ms3_notes = ms3_timeline.get_events(event_type="Note")

partitura_note_count = len(loaded_notes)
note_counts = {
    "MusicXML via PartituraLoader": partitura_note_count,
    "TSV via Ms3Loader": len(ms3_notes),
}
timeline_classes = (score_timeline.class_name, ms3_timeline.class_name)
assert len(loaded_notes) == partitura_note_count
assert len(loaded_events) >= partitura_note_count
assert len(ms3_events) == len(ms3_notes)
assert len(set(note_counts.values())) == 1
assert len(set(timeline_classes)) == 1
format_comparison = {
    "timeline class": timeline_classes[0],
    "note counts": note_counts,
}
format_comparison

{'timeline class': 'ContinuousLogicalTimeline',
 'note counts': {'MusicXML via PartituraLoader': 498,
  'TSV via Ms3Loader': 498}}

Both routes produce a ContinuousLogicalTimeline and find 498 notes: different provenance, same musical content. File formats do not always preserve the same amount of information, however; the later pitch-and-harmony tutorial shows where their knowledge differs.

The loader families

Choose a loader family according to what the source represents, then choose a concrete loader for its file format.

Family	Public loaders	What it is for
Score	`Ms3Loader`, `Music21Loader`, `PartituraLoader`	Symbolic notation
MIDI	`ScoreMidiLoader`, `PerformanceMidiLoader`	Quantized scores or performed timing
Tabular	`TsvLoader`, `CsvLoader`	Custom delimited research tables
Alignment		Score-to-performance correspondences; introduced later
Graphical	`GraphicalLoader`	Pages, images, and horizontal layout
Format	`JsonLoader`	Structured JSON and XML

Each family has a how-to guide of its own: loading data · tabular loaders · graphical timelines · the Vienna corpus · load anything

loader_families = {
    "score": {
        "loaders": (Ms3Loader, Music21Loader, PartituraLoader),
        "example": source_files["MusicXML"],
    },
    "MIDI": {
        "loaders": (ScoreMidiLoader, PerformanceMidiLoader),
        "examples": (source_files["score MIDI"], source_files["performance MIDI"]),
    },
    "tabular": {
        "loaders": (TsvLoader, CsvLoader),
        "example": source_files["ms3 TSV"],
    },
}
loader_families

{'score': {'loaders': (timetoalign.loader.score.ms3.Ms3Loader,
   timetoalign.loader.score.music21.Music21Loader,
   timetoalign.loader.score.partitura.PartituraLoader),
  'example': PosixPath('/home/laser/git/tta/timetoalign/tests/data/vienna_1x22/Chopin_op10_no3.musicxml')},
 'MIDI': {'loaders': (timetoalign.loader.midi.score.ScoreMidiLoader,
   timetoalign.loader.midi.performance.PerformanceMidiLoader),
  'examples': (PosixPath('/home/laser/git/tta/timetoalign/tests/data/midi/score/rachmaninoff_piano.mid'),
   PosixPath('/home/laser/git/tta/timetoalign/tests/data/midi/performance/rachmaninoff_perf.mid'))},
 'tabular': {'loaders': (timetoalign.loader.tabular.csv.TsvLoader,
   timetoalign.loader.tabular.csv.CsvLoader),
  'example': PosixPath('/home/laser/git/tta/timetoalign/tests/data/vienna_1x22/ms3/chopin_op10_no3.notes.tsv')}}

The class objects show that the score, MIDI, and tabular choices used here are importable public loaders, while the example paths distinguish score MIDI from performance MIDI. The remaining table rows name a concrete public loader and link to the guide that introduces it.

Beyond one timeline

create_timelines() opens the door to loaders that yield several timelines; this leads to the next tutorial on s, Timeline Groups. Alignment-capable loaders also provide create_bundle(), which produces an in Alignment Bundles and MatchClaims.

timeline_list = ms3_loader.create_timelines()
future_doors = {
    "timelines returned": timeline_list,
    "loader families sampled above": len(loader_families),
}
future_doors

{'timelines returned': [ContinuousLogicalTimeline(id='clt1', length=83/2, unit=quarters, events=498, children=0, cmaps=1)],
 'loader families sampled above': 3}

This score loader returns a one-item list; multi-source loaders can return more. The list-shaped interface is the bridge from loading one timeline here to organizing several timelines in the next tutorial.

What you learned

You can replace hand construction with real files without changing the kind of timeline produced.
You can separate load() from create_timeline() and use from_file() as their common shorthand.
You can inspect exact table counts, units, and ranges in loader.store, and read summary() as the counts-and-extent digest it is.
You can distinguish a parser record in an EventStore from the corresponding event placed under a timeline child.
You can assign a type-based timeline id, optionally prefixed by a role, while keeping a readable name separate.
You can inspect loader-created children and exact coordinate conversions, and recognise that this loader does not create regions.
You can display representative rows from an all-child event query.
You can narrow loaded events with the familiar event filter and inspect the resulting note rows.
You can compare MusicXML and ms3 TSV note content through the same timeline API.
You can choose among score, MIDI, tabular, alignment, graphical, and format loader families.
You can use create_timelines() and recognize create_bundle() as entry points to multi-timeline work.

Timeline Groups

Loading Real Data

Loading Real Data

What you will build

Before you start

From hand-built to real

The two-phase contract

The EventStore

One parsed record

Making the timeline

What you got for free

Querying every child

Narrowing by event type

The same music, a second way

The loader families

Beyond one timeline

What you learned

Next

Go deeper