Where to find open data for practice projects

The kinds of sources

National statistics. Demography, economy, health, transport. Reliable data, but the exports are often awkward and updates are slow.

City open-data portals. Public transport, urban maintenance, incidents, residents' requests. The most interesting for practice work: local, comprehensible data that lets you draw conclusions about where you live.

Public service interfaces. Weather, exchange rates, reference data, mapping. Good because the data is current and you can build something that updates itself.

Datasets from analysis competitions. Ready, clean, documented. Convenient for practice but poor for a portfolio: thousands of people have used them.

Collecting your own. Writing a program that gathers data from an open source. Valued most highly, because it shows not only analysis but the ability to obtain data.

Choosing a dataset

Look for a question, not for data. Decide what you want to find out, then look for evidence. The reverse produces work with no conclusion.

Take a field you understand. You read familiar data differently: you notice oddities and know what a figure means.

Check the size. A thousand rows is thin for conclusions. Ten million is heavy for a first piece. The sensible middle: tens to hundreds of thousands.

Check there's something to compare. One month of data is dull. Several years, or several regions, lets a pattern appear.

What to check before starting

Usage terms. Open data usually carries a permissive licence, but some sources impose conditions. Rarely a problem for practice work, but worth a look.

Completeness. Open it and see how many values are missing. A dataset half full of blanks turns the work into torture.

Consistency. The classic ailment: one year a city is written in full, another abbreviated, a third with a typo. That needs cleaning, and it's a normal part of the job — but assess the volume in advance.

Duplicates. Check for repeated records. They corrupt every calculation.

Comprehensible fields. If there's no documentation and a column's meaning is unclear, take another dataset.

About cleaning

Expect it to take more than half your time. That isn't a sign of a bad dataset but the ordinary reality of analytical work.

One thing is mandatory: describe what you cleaned and why. Discarded records, filled gaps, normalised names — all of it is part of the work and belongs in the portfolio. An analyst who speaks honestly about source data quality inspires trust.

Question ideas

To save you searching:

• How has public transport usage changed by district?
• Is there a link between urban maintenance and residents' complaints?
• Which goods are rising in price faster than general inflation?
• How are incidents distributed across the hours of the day and days of the week?
• Does the structure of spending differ between regions?

What they share: a clear question, accessible data, and a conclusion that fits in one sentence.