C++ for Machine Learning: Practical Capabilities and Limitations

C++ for Machine Learning: Capabilities and Limitations in Practice

When it comes to machine learning, most developers immediately think of Python with its ecosystem of PyTorch, TensorFlow, and scikit-learn. However, C++ remains an indispensable tool for production systems where speed, low latency, and memory control are critical. In this article, we will explore how to use C++ for machine learning in practice, what its strengths are, and where it falls short compared to Python.

Why is C++ Still Relevant for ML?

Python dominates in research and prototyping, but C++ is the language in which the cores of almost all popular ML frameworks (TensorFlow, PyTorch, XGBoost) are written. If you need to:

\n
• Deploy a model on embedded devices (IoT, robotics);
\n
• Process data in real-time (autopilots, trading);
\n
• Achieve maximum performance without interpreter overhead;
\n
• Integrate ML into existing C++ code (games, CAD, simulators),
\n

then C++ becomes not just an alternative, but the only choice.

Practical Capabilities of C++ for ML

1. Working with Linear Algebra (Eigen)

The foundation of any ML is matrix operations. In C++, the Eigen library is used for this. It provides performance close to manual optimizations, but with a convenient syntax.

#include <Eigen/Dense>\n#include <iostream>\n\nint main() {\n    // Create matrices\n    Eigen::MatrixXd X(3, 2);\n    X << 1, 2,\n          3, 4,\n          5, 6;\n    Eigen::VectorXd y(3);\n    y << 2, 4, 6;\n\n    // Solving the least squares problem: w = (X^T X)^{-1} X^T y\n    Eigen::VectorXd w = (X.transpose() * X).ldlt().solve(X.transpose() * y);\n    std::cout << "Model weights: " << w.transpose() << std::endl;\n    return 0;\n}

This code computes linear regression without a single loop — Eigen uses vectorization and multithreading under the hood.

2. Classical ML with Dlib

Dlib is one of the few C++ libraries that provides ready-made algorithms: SVM, k-nearest neighbors, decision trees, and even deep learning.

#include <dlib/svm.h>\n#include <dlib/rand.h>\n#include <iostream>\n\nint main() {\n    using sample_type = dlib::matrix<double, 2, 1>;\n    std::vector<sample_type> samples;\n    std::vector<double> labels;\n\n    // Generate synthetic data\n    dlib::rand rnd;\n    for (int i = 0; i < 100; ++i) {\n        sample_type s;\n        s(0) = rnd.get_random_double();\n        s(1) = rnd.get_random_double();\n        samples.push_back(s);\n        labels.push_back(s(0) + s(1) > 1.0 ? 1.0 : -1.0);\n    }\n\n    // Train SVM with RBF kernel\n    dlib::svm_c_ekm_trainer<dlib::kernel_type> trainer;\n    trainer.set_kernel(dlib::radial_basis_kernel<sample_type>(0.1));\n    auto decision_function = trainer.train(samples, labels);\n\n    // Test\n    sample_type test = {0.8, 0.7};\n    std::cout << "Prediction: " << decision_function(test) << std::endl;\n    return 0;\n}

Dlib supports model serialization, allowing you to save trained models and load them in production.

3. Neural Network Inference Without Python

To run already trained models in C++, use ONNX Runtime or TensorFlow C++ API. Example of loading an ONNX model:

#include <onnxruntime/core/session/onnxruntime_cxx_api.h>\n#include <vector>\n\nint main() {\n    Ort::Env env(ORT_LOGGING_LEVEL_WARNING, "test");\n    Ort::SessionOptions session_options;\n    Ort::Session session(env, "model.onnx", session_options);\n\n    // Prepare input data (example for a 3x224x224 image)\n    std::vector<float> input_tensor_values(3 * 224 * 224, 0.0f);\n    std::vector&l