It's not 'intelligence' that matters, it's the understanding

This topic contains 5 replies, has 4 voices, and was last updated by  Istvan Makaresz 1 year, 6 months ago.

  • Author
  • #10510

    I think soon machine learning/AI will produce technology that we don’t understand, it will work and work better than what humans can do, but we won’t understand how it works.

    This could relate to new chip designs (which are mainly rules based), material science, etc.


    We (humans) don’t fully understand how deep learning (e.g., CNN) works. but thanks to it machine can now do better than humans in image recognition.

    Not sure when we will fully understand deep neural nets.


    @stevekennedyuk and @jiefeima, we do understand how deep neural networks work (for example see this explanation, it just becomes much more difficult to visualise in higher dimensions!


    @paulharrison, no, we are still far from fully understand how deep neural nets work (or should work). Take CNN as an example, we may know how convolution works, we may know how pooling works, we may also know how gradient descent works. But no one can answer, given a data set, how many kernels we need, what should be the right kernel sizes, how many layers we should build, and etc. Everything is trial-and-error (although past experience may help). There is no mathematical theory backing it. In contrast, think about building a space rocket, each part of the design can be and must be precisely and mathematically calculated.

    In fact, this is a well-known criticism of deep learning in the academic community.


    @jiefeima I see what you mean, although that argument is equally valid for all supervised machine learning methods, not just deep learning; one always takes the same trial and error approach. Are there closed form solutions for any hyperparameter optimisation problems?

     Istvan Makaresz 

    Oh, it seems to be the right place for asking a few questions. (you probably already have provided the answers). I am very interested in this network complexity topic from a biological perspective. In some early work on CNN (Churchland) they found what I guess is the essence of what you debated above:namely given a task (it was differentiating sonic echoes of underwater mines from those of rocks) too simple systems couldn’t even get good results on training sets (Churchland called them ‘dumb’), while overly complicated systems developed dedicated paths on the training sets and then performed poorly on’ natural’ samples (he called these ‘lazy’). So is it still a prevalent problem in CNN design nowadays and there is no way to predict necessary complexity?

Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.

Log in with your credentials


Forgot your details?

Create Account