tag:blogger.com,1999:blog-17558167.post7278944127645876153..comments2024-01-22T04:53:37.483-08:00Comments on A Radial Mind: Tikhonov RegularizationGerard Toonstrahttp://www.blogger.com/profile/17067969645449987498noreply@blogger.comBlogger3125tag:blogger.com,1999:blog-17558167.post-82300269175582420082009-05-06T23:21:00.000-07:002009-05-06T23:21:00.000-07:00The following shows that with different regulariza...The following shows that with different regularization settings, you'll get very different results, yet the overall RMSE is equal. The bias 0.0000 in the middle probably yields the solution for a generalized linreg.<br /><br />0.890898 bias=-0.030049 Ws=[0.295879,0.714400]<br />0.890897 bias=-0.032216 Ws=[0.294100,0.716759]<br />0.890916 bias=0.000000 Ws=[0.285848,0.716473]<br />0.890897 bias=-0.032768 Ws=[0.293651,0.717356]<br />0.890897 bias=-1.619981 Ws=[0.293563,0.717346]Gerard Toonstrahttps://www.blogger.com/profile/17067969645449987498noreply@blogger.comtag:blogger.com,1999:blog-17558167.post-22104925165284584062009-05-06T23:14:00.000-07:002009-05-06T23:14:00.000-07:00I can't possibly answer that question, because it ...I can't possibly answer that question, because it depends on how your predictions are arranged in the total data set. I assume however that for many prediction sets, they are quite well distributed across the actual ratings, if you consider the entire prediction range. Thus, the problem isn't necessarily ill-posed. In that case, the overall rmse is probably very equal. That is why many people state that there's no difference whatsoever.<br /><br />I have a set here which without any regularization gets 0.890916. With regularization it gets 0.890897. So it's minimal, but only because the number of samples is so high.<br /><br />If interested, you can play around with the numbers in the main.cpp. You can see the results for ratings that are generally above or below the actual ratings and their influence on the weights (turn regularization off though).<br /><br />Tikhonov is certainly no silver bullet. It's very likely that it's mostly effective for situations where there are a small number of samples. 140,000 or 1,400,000 doesn't probably qualify as small, plus that their spread is quite regular.<br /><br />The best thing to conclude is that there are bad prediction sets which don't contribute to your overall results and there are good prediction sets which do help. Tikhonov may help you to still nicely blend the not-so-good ones, but it definitely isn't a silver bullet that hammers things into place.<br /><br />Having a number of prediction sets which blend very nicely together with regular normalization is the best way forward, probably.Gerard Toonstrahttps://www.blogger.com/profile/17067969645449987498noreply@blogger.comtag:blogger.com,1999:blog-17558167.post-25045421575260000682009-05-06T17:02:00.000-07:002009-05-06T17:02:00.000-07:00So this begs the obvious question - for the Netfli...So this begs the obvious question - for the Netflix data how much improvement is there over a simple linear combination of predictions?Anonymousnoreply@blogger.com