It's not 'intelligence' that matters, it's the understanding

This topic contains 5 replies, has 4 voices, and was last updated by  Istvan Makaresz 1 year ago.

  • Author
    Posts
  • #10510
     Steve 
    Participant

    I think soon machine learning/AI will produce technology that we don’t understand, it will work and work better than what humans can do, but we won’t understand how it works.

    This could relate to new chip designs (which are mainly rules based), material science, etc.

    #10576
     Jiefei 
    Participant

    We (humans) don’t fully understand how deep learning (e.g., CNN) works. but thanks to it machine can now do better than humans in image recognition.

    Not sure when we will fully understand deep neural nets.

    #10869
     Paul 
    Keymaster

    @stevekennedyuk and @jiefeima, we do understand how deep neural networks work (for example see this explanation, it just becomes much more difficult to visualise in higher dimensions!

    #10933
     Jiefei 
    Participant

    @paulharrison, no, we are still far from fully understand how deep neural nets work (or should work). Take CNN as an example, we may know how convolution works, we may know how pooling works, we may also know how gradient descent works. But no one can answer, given a data set, how many kernels we need, what should be the right kernel sizes, how many layers we should build, and etc. Everything is trial-and-error (although past experience may help). There is no mathematical theory backing it. In contrast, think about building a space rocket, each part of the design can be and must be precisely and mathematically calculated.

    In fact, this is a well-known criticism of deep learning in the academic community.

    #10947
     Paul 
    Keymaster

    @jiefeima I see what you mean, although that argument is equally valid for all supervised machine learning methods, not just deep learning; one always takes the same trial and error approach. Are there closed form solutions for any hyperparameter optimisation problems?

    #12541
     Istvan Makaresz 
    Participant

    Oh, it seems to be the right place for asking a few questions. (you probably already have provided the answers). I am very interested in this network complexity topic from a biological perspective. In some early work on CNN (Churchland) they found what I guess is the essence of what you debated above:namely given a task (it was differentiating sonic echoes of underwater mines from those of rocks) too simple systems couldn’t even get good results on training sets (Churchland called them ‘dumb’), while overly complicated systems developed dedicated paths on the training sets and then performed poorly on’ natural’ samples (he called these ‘lazy’). So is it still a prevalent problem in CNN design nowadays and there is no way to predict necessary complexity?

Viewing 6 posts - 1 through 6 (of 6 total)

You must be logged in to reply to this topic.

Processing...
Thank you! Your subscription has been confirmed. You'll hear from us soon.
ErrorHere

Log in with your credentials

or    

Forgot your details?

Create Account