CNTK provides a directed graph representation for defining neural networks where nodes represent input values, parameters, or matrix operations. It supports feed-forward DNNs, convolutional networks (CNNs), and recurrent networks (RNNs/LSTMs). The toolkit implements stochastic gradient descent with automatic differentiation and parallelization across multiple GPUs and servers. It is written primarily in C++ with Python bindings.