Hey there… So I enjoyed AI when I studied it, many years ago, but I don’t remember anything. Today I run into some deep learning stuff and well, I just want to know how a neural network with only one hidden layer could be trained. I know that wouldn’t be deep learning, but it doesn’t matter. I guess that there are different approaches, involving different math functions and algorithms for coming up with the right weights (starting from supervised vs unsupervised learning), but would love to see something specific and simple.
This topic genuinely interests me, so if there’s anyone who’s into this stuff it would be great to share thoughts and information.
Last Friday I was talking to my slave about something totally unknown to today’s “developers”, expert systems. Back in the day we had to do something at uni with clips, and I was toying with the idea of some implementation from my own… But I’m soooo lazy
You pretty much answered your own question. The various different training algorithms exist to deal with different problems; it depends what you want the network to achieve and what your data looks like.
If you want to implement a NN from scratch, try gradient backpropagation first. It’s pretty easy.
No, I just find the whole thing interesting. It kinda makes sense to me although the mathematical aspect of it is overwhelming. I wrote once some short story that was similar in essence to what nn are.
For the expert system I do have a project in mind that would use it (or something close enough), but I lack the motivation to do the code of the app, plus it’s an idea that could be a serious project, paid by the government… If they wanted it and didn’t want to just steal the idea
you really mean it’s easy to implement the learning algorithm from scratch? I don’t even understand the formulas haha (although I kinda understand the explanations…).
Last time I actually did that was at university 25 years ago, but yeah, I remember it was straightforward enough. Although my brain worked a bit better back then than it does these days.
If that’s not what you were interested in, there are various software packages that let you build NNs without worrying about exactly what goes on under the hood. Neurosolutions do a free evaluation download.
I think expert systems are still out there - the term just isn’t used anymore. For example I seem to remember there’s an app now that provides an expert system for doctors to make diagnoses. Basically it’s about 10x better than an actual human, so if you see your doctor fiddling with his smartphone, don’t worry - the computer will tell him what to do.
No government will pay for a computer system that works. That’s not what government computer systems are for. The REAL point, typically, is to make sure their buddies get lots and lots of money. For example there is/was a front company in the UK called Crapita (they might have gone bankrupt by now) which is apparently staffed entirely by friends of the gubmint. Their job is to absorb billions of pounds of taxpayers money and give it to the right people. In return, they provide some half-assed software package that doesn’t do anything, wasn’t properly specified in the first place, and could have been designed and written competently by six 23-year-old graduates for 1% of the price.
The UK government isn’t unique in this. If you want to see the purest example of the art, have a look at the “software” churned out by Philippines Government agencies.
Currently my job involves writing neural networks. A simple network with just one hidden layer is the first thing people came up with when it comes to neural networks.
People were working on the single layer networks until some dude pointed out in the 1950s that a single layer neural network could not work out the logic behind XOR.
Then Marvin Minsky, founder of MIT AI Lab, pointed out in 1969 that the electronic neurons at the time could not be scaled to multi-layer networks. That’s when the first wave of artificial neural network hit a wall.
If you do a single hidden layer, the best you can do is train the network to work as a linear classifier, which is exactly why a single hidden layer cannot learn to perform XOR logic.
You need two lines to perform classification for XOR. Single layer can’t do that.
