· ~5 min read
Madrid air quality: clustering, classification and transfer learning
A team project on Madrid air-quality data: K-Means against DBSCAN, SVM against Naive Bayes, and a small neural network that behaved better with transfer learning.
Goal
This was the final project of the Data Analytics and Machine Learning course in my MSc, done as a team project with two classmates. We took the public "Air Quality Madrid" dataset (hourly pollutant readings from the city's monitoring stations) and asked three questions: are there hidden pollution profiles, can a model predict the air-quality class, and can it predict the exact index value?
Preparing the data
- Selection and gaps. We kept the date, the station and seven pollutants: SO2, CO, NO2, NOx, O3, PM10 and PM2.5. Each station's series was treated separately, and isolated gaps were filled by linear interpolation so the time series stayed continuous.
- The PM2.5 problem. PM2.5 had far more missing values than the rest, too many to interpolate. We filled them with a linear regression on SO2, CO and NOx, relying on the correlation between pollutants.
- Features. We computed the sub-indices for NO2, O3, PM10 and PM2.5, the CAQI (Common Air Quality Index) as the largest sub-index, and a CAQI class from 0 (very low) to 4 (very high) for classification.
The exploratory plots shaped the rest of the work. The correlation matrix showed strong positive correlation between the nitrogen oxides and PM10. The histograms were heavily right-skewed: most hours are clean, and a long tail is not. The box plots, on a log scale, showed many extreme days. We kept them, because they are real pollution episodes rather than sensor errors, and they are exactly what a useful model has to handle.
Clustering: distance or density?
We clustered on CO, NO2 and PM10, standardised so that no pollutant dominates through its units, with Euclidean distance.
| Algorithm | Settings tried | Best | Silhouette |
|---|---|---|---|
| K-Means | k = 3, 4, 5 | k = 4 | about 0.41 |
| DBSCAN | min_samples = 20, eps = 0.5, 0.6, 0.8 | eps = 0.6 | above 0.60 |
K-Means was faster and easier to explain, but it forces the data into round clusters of similar size, and pollution does not come in balls. DBSCAN found one dense core of normal conditions and marked the sparse high-pollution hours as noise, which for this problem is the useful answer: it separates ordinary pollution from episodes.
Classification: predicting the CAQI class
| Model | Accuracy | Where it went wrong |
|---|---|---|
| SVM, RBF kernel | 98% | Almost nowhere; the confusion matrix is nearly diagonal |
| Gaussian Naive Bayes | 74% | The middle classes 1-3, e.g. a true "high" predicted as "medium" or "very high" |
Naive Bayes assumes every pollutant is independent of the others, and here they clearly are not: when NO2 rises, PM10 tends to rise with it. The SVM can learn those joint patterns.
Regression: predicting the CAQI value
We built a small feed-forward network in PyTorch: the scaled pollutant features in, two hidden layers of 64 and 32 ReLU units, one output, trained with Adam on mean squared error. Then we compared two ways of training it. The training data was split into a "source" part and a "target" part:
- Custom model: trained from random weights on the target data only.
- Transfer model: pre-trained on the source data, then fine-tuned on the target data with a lower learning rate (0.005) to keep what it had already learned.
| Model (test set, 20%) | R² | MAE |
|---|---|---|
| Custom | 0.88 | 21.29 |
| Transfer learning | 0.90 | 18.10 |
The headline numbers are close, but the plots tell the story. The transfer model's training loss starts low, because it already knows the shape of the problem. And on the real-versus-predicted plot, the custom model loses control at the high end: for true values around 400 it predicted up to 1,700, while the transfer model stayed near the ideal line.
What I learned
The take-home points we wrote were: skewed data with real outliers is the normal case for environmental data; density-based methods suit it better than distance-based ones; models that capture correlations beat models that assume independence; and pre-training made the network far more stable on extreme values.
Looking at it again with a more suspicious eye, these are the checks I would add before trusting the numbers:
- The 98% may be measuring a formula. The CAQI class is computed directly from pollutant sub-indices. If the classifier sees those same readings, it is relearning a deterministic rule, not predicting anything. The honest test is to predict a future hour's class from earlier readings, or to leave out the pollutants the class is computed from.
- Time series need time-based splits. With hourly data, neighbouring hours are nearly identical. If the test set is a random 20%, the model has effectively seen its neighbours during training. Testing on a later period is the fair version.
- Imputed values leak into the target. Our filled-in PM2.5 values came from a regression on other pollutants and then fed the CAQI. Part of what the models learned is our own imputation.
- DBSCAN's silhouette depends on how noise is counted. If the noise points are left out of the score, the clusters that remain look cleaner than the data really is.
A team course project for the MSc in Cybersecurity & Data Science at the University of Piraeus, on a public dataset.
Want the configs or code from this lab, or have a question? Email me at kostaspapas1023@gmail.com