Securing IoT with AI-based cyberattack detection models using real-world testbeds and datasets

AI-enabled cyberattacks continue to evolve rapidly,
challenging traditional centralized intrusion detection systems
deployed in heterogeneous and large-scale IoT environments. This
work presents NorFin Lab, a hybrid IoT cybersecurity testbed
that integrates physical IoT devices, hardware-backed network
stacks, and hardware-induced timing variabilities to generate
realistic benign and adversarial traffic. Using NorFin Lab, we
collected a dataset covering 26 attack samples across brute-force,
injection, reconnaissance, and DoS categories. Lightweight neural
network architectures suitable for edge or resource-constrained
deployments, including FNN, CNN, LSTM, and attention-based
models, were evaluated on this dataset. The models exhibit a
stable and near-perfect convergence on known attack types;
however, their performance degrades substantially when classifying
previously unseen variants, revealing significant generalization
gaps. Analysis of misclassification patterns show an
overlapping feature distribution between reconnaissance, bruteforce,
and injection attacks, highlighting the ambiguity in short
packet-window representations. These findings suggest that models
trained on homogeneous or cleanly separated datasets may
struggle in distributed IoT deployments and are insufficient for
detecting unseen or zero-day attack variants.