tag:blogger.com,1999:blog-7233553492253101490.post1032028715460070766..comments2018-08-20T07:15:11.092-04:00Comments on Minimizing Regret: A mathematical definition of unsupervised learning?Elad Hazanhttps://plus.google.com/111108599717601698901noreply@blogger.comBlogger5125tag:blogger.com,1999:blog-7233553492253101490.post-64727743186232950692018-02-27T04:43:29.386-05:002018-02-27T04:43:29.386-05:00This comment has been removed by the author.Wilhelm Duncanhttps://www.blogger.com/profile/04606611102085119621noreply@blogger.comtag:blogger.com,1999:blog-7233553492253101490.post-33199979929372533502016-10-30T19:57:02.595-04:002016-10-30T19:57:02.595-04:00Cool:) Looking forward to it (and I see the follow...Cool:) Looking forward to it (and I see the followup will have a followup:))Csaba Szepesvarihttps://www.blogger.com/profile/13790307935040509983noreply@blogger.comtag:blogger.com,1999:blog-7233553492253101490.post-91835572793612017912016-10-09T02:55:30.209-04:002016-10-09T02:55:30.209-04:00Interesting post. This reminds me of the joke &quo...Interesting post. This reminds me of the joke "Classification of mathematical problems as linear and nonlinear is like classification of the Universe as bananas and non-bananas". Like nonlinear math, unsupervised learning is a very large class of loosly connected ideas so I am intrigued as to what can be said about it. <br /><br />Given that, even mathematically formulating sub-problems like clustering is hard. While some work has been done it only captures part of that we consider clustering.ethan fetayahttps://www.blogger.com/profile/07933110238276416602noreply@blogger.comtag:blogger.com,1999:blog-7233553492253101490.post-82700966903895428522016-10-08T01:07:30.684-04:002016-10-08T01:07:30.684-04:00Thanks Csaba!
Your paper looks cool and in the d...Thanks Csaba! <br />Your paper looks cool and in the direction of changing the statistical assumptions of ICA to some form of "closeness" to a signal in standard input-ICA form. This is certainly in the correct direction of removing statistical assumptions. <br />What I'm arguing for is even more extreme: can we "step out of the model" (ICA in this case) completely, regardless of any special form of the input attain worst-case guarantees? The analogy would be to perform low-rank matrix completion (usually studied under uniform distribution over the inputs and incoherence assumptions) by low-trace (or low max-norm) relaxations. <br />More to come soon...<br /><br /><br /> Elad Hazanhttps://www.blogger.com/profile/14447909674264489117noreply@blogger.comtag:blogger.com,1999:blog-7233553492253101490.post-25090625327931785692016-10-07T19:04:39.318-04:002016-10-07T19:04:39.318-04:00I am really looking forward to the post:)
In the m...I am really looking forward to the post:)<br />In the meanwhile, let me point out one work that I happen to know about as I was part of it (sorry for the plug, I could not help it).<br />So what is happening in this work? Well, we study ICA *without* any generative assumptions. I think what we do can be done in many unsupervised settings, but we choose ICA as it is a setting that seems to be closely tied to generative and stochastic assumptions, so in some sense it looked especially challenging. So what do we do? We derive performance bounds for some algorithm (building on previous work, including Arora's) in terms of how close the data that the algorithm sees is to "ideal data". I find this approach quite satisfactory as finally we don't need to make any generative/stochastic assumptions, and the bounds tell one exactly what we need: how performance (recovery of some mixing matrix in this case) will be impacted by deviations from the ideal situation. As a bonus, one can show that the bounds can also recover the usual bounds available in the generative setting. Details are here:<br /> Huang, R., György, A., and Szepesvári, Cs., Deterministic Independent Component Analysis, ICML, pp. 2521--2530, 2015. https://goo.gl/N1pnMLCsaba Szepesvarihttps://www.blogger.com/profile/13790307935040509983noreply@blogger.com