I just saw this paper about distributed Neural Networks: http://www.eecs.harvard.edu/~htk/publication/2017-icdcs-teerapittayanon-mcdanel-kung.pdf
The basic idea is to have small NN in the device, such as a mobile phone and fallback to the cloud if there is no confidence that the small NN is able to classify the example correctly.
It also helps to reduce the bandwidth consumption, since the device might send only a smaller and compressed representation of the data, much like an embedding.
They give an example of a surveillance system. Imagine: using this technique, we might be able to create (of that doesn’t exist yet) something like a God’s eye which can find anyone anywhere.
@andresnds 