Motivation
Sometimes in Italy, a station has no screen at all, no information about the trains coming. Only the speaker.
After taking the wrong train twice, I decided to develop my own tools to track trains, and since that day I have never taken the wrong one.
The goal is to turn macro data into micro data. The official source only says “this train was seen at this station, N minutes late”. Between two stations, nothing. I want to reduce this large initial uncertainty with a model, and predict as well as possible where a train is along the track, even far from any station.
The result is a small control room for the trains of Liguria: a map and a diagram of the network, with the trains moving on them.
Step one: I need data
After trying different approaches, the best solution is the ViaggiaTreno API (endpoints, useful doc). It is unofficial, but it is the one the Italian railway information itself relies on.
There is no GPS in it. For a train, the API gives the list of its stops, the delay announced at the last detection point, and nothing else. Everything between two stations has to be estimated.
The second problem is that there is no “give me all the trains” endpoint. A station board lists only the trains that call at this station. So the collector works in two phases:
- Discover: read the boards of a few source stations (Genova Principe, Principe sotterranea, Brignole, then the stations around Genoa and on the coast, so that a train which skips Genoa is still found).
- Follow: once a train is found, poll it until its terminus. Often when its next stop is close (every few minutes), rarely when it is far (every 30 minutes).
Each poll is stored as a snapshot, so I keep not only the final delay but how the announced delay evolved during the trip.
trains, stops, snapshots)] OSM[(OpenStreetMap)] --> NET[network
stations, edges, speed limits] DB --> M[model] NET --> M M --> API[API] API --> WEB[map + diagram]
The first run of the collector was a cron job, every 5 minutes. After five days of recording I had about 1,500 trains, 25,000 stops and around 35 polls per train. Enough to start looking at the data.
One thing I learned quickly: the data is not clean. A real arrival time is recorded for only two stops out of three (66 %), the departure for 91 %. The rest of the time the feed simply never says the train arrived.
Step two: Think like a train
The difficulty: the information is sequential, delayed, and only true at one instant. A train announced 10 minutes late may recover part of it, or add more, over the trip. The API’s own estimate is naive: it assumes the delay stays constant until the end.
The model must also stay light: it has to run on a simple VPS, not on a machine with 64 GB of RAM. So for now it does not include complex relations between the data (a late train blocking the next one, for example). The motivation is to build the best simple model.
It is built in layers, each one fixing a weakness of the previous one:
- Rules. A train recovers about half of its delay, up to ~4.4 minutes, and the longer the way, the more it recovers (time constant of 25 minutes). A stop in a regional station lasts about 1.7 min + 0.8 × the scheduled stop.
- Tables. On top of the rules, the model learns the usual error of each pair of stations (last known stop → next stop) from the past days. Some sections are always a bit slower than the timetable says, and this is simply written in a table.
- Inference. The data lags by a minute or two. A train past its predicted arrival time is considered arrived, one past its usual stay is considered gone, up to 4 stops ahead. The real data corrects it when it comes.
- Kinematics. Between two stations, the train accelerates, cruises and brakes. Stretched to the predicted travel time, this gives a position along the track, not a straight line in time: slow, then fast, then slow again.
Does it work?
The test is simple: fit the model on four days (13 to 16 October), then predict the arrival time of the next stops on a day it has never seen (17 October), and compare with what really happened. The error is in minutes, averaged over the 1 to 6 next stops.
| Model | Mean error | 90 % of errors under |
|---|---|---|
| Delay stays constant (what the API assumes) | 2.9 min | 6.5 min |
| Rules | 2.3 min | 5.5 min |
| Rules + tables | 2.0 min | 4.8 min |
The complete model reduces the error by about a third compared to the API’s assumption. For the next stop only, it goes from 2.2 to 1.3 minutes.
It also says how much it does not know. For the next stop, 90 % of real arrivals fall between 3.5 minutes before and 3.6 minutes after the prediction, and the further the stop, the wider the interval (up to +14 minutes for 8 stops ahead). On the test day, these intervals really contain 92 to 96 % of the arrivals.
Be careful with this result: it is five days of data. The tables improve every day, and a more complex model (trees) will wait for at least 14 days of recordings.
Step three: Get access everywhere
A model is useless if it stays in a notebook. The goal is to open it from the platform, from my phone, from anywhere.
Everything runs in Docker on my VPS, deployed with Coolify at each merge on main. One server does it all: it serves the site, polls ViaggiaTreno continuously, and refits the delay model every day on what it recorded. The data lives on the server, outside the containers, so a redeploy never touches it, and it is backed up every night.
The site has three doors, always one click apart:
- Traffic: the control room. A map and a diagram of the network, with the trains moving on it, colored by delay. A slider shows the forecast up to 60 minutes ahead, and a replay lets me watch any recorded moment.
- Panel: the station boards, my first tool, the one that started everything.
- Data: the recorded data in charts (trains per day, delay by hour, by station, the error of the model).
The raw database is also readable in a browser with Datasette, with SQL and CSV export. The site is behind a login.
On top of that, a small Discord bot reads the same database and posts a digest three times a day, line by line and direction by direction. Each way gets a light (🟢 90 % of trains on time or more, down to 🔴 under 60 %), and an alert is sent when two trains run 20 minutes late at the same time.
What I learned
- Record everything, early. The model only became possible because every poll was stored. The question I ask today was not the one I had when I started the collector.
- A simple model with an honest error bar is more useful than a complex one without it.
- The hard part is the data, not the model. Missing arrivals, a feed lagging by two minutes, trains that skip a station and are on no board: most of the work was there.
Next
- Wait for more days of data, and test a model with trees.
- Watch more stations, to see the trains that call nowhere in Genoa.
- Freight trains are not in ViaggiaTreno at all: the traffic I see is the passenger traffic only.