Видео

thank you 🙏🏻

Very well explained. Thank you.

Nice😎👍

people of different cultures view the world in entirely different ways. It depends on culture, language, genetics etc. for example, people who speak Navajo have an entirely different way of perceiving reality and breaking it down into components than western English speaking people. A shaman would also see the world completely differently to a western man

1. Introduction and Context (0:00 - 1:47) - Ilya Sutskever speaking at an event - Unable to discuss current technical work at OpenAI - Focused on AI alignment research recently - Will discuss old results from 2016 that influenced thinking on unsupervised learning 2. Fundamentals of Learning (1:47 - 5:51) - Questions why learning works at all mathematically - Discusses supervised learning theory (PAC learning, statistical learning theory) - Explains mathematical conditions for supervised learning success - Mentions importance of training and test distributions being the same 3. Unsupervised Learning Challenge (5:51 - 11:08) - Contrasts unsupervised learning with supervised learning - Questions why unsupervised learning works when optimizing one objective but caring about another - Discusses limitations of existing explanations for unsupervised learning 4. Distribution Matching Approach (11:08 - 15:32) - Introduces distribution matching as a guaranteed unsupervised learning method - Explains how it can work for tasks like machine translation - Links to Sutskever's independent discovery of this approach in 2015 5. Compression Theory of Unsupervised Learning (15:32 - 24:43) - Proposes compression as a framework for understanding unsupervised learning - Explains thought experiment of jointly compressing two datasets - Introduces concept of algorithmic mutual information - Links compression theory to prediction and machine learning algorithms 6. Kolmogorov Complexity and Neural Networks (24:43 - 30:52) - Explains Kolmogorov complexity as the ultimate compressor - Draws parallels between Kolmogorov complexity and neural networks - Discusses conditional Kolmogorov complexity for unsupervised learning - Links theory to practical neural network training 7. Empirical Validation: iGPT (30:52 - 35:46) - Describes iGPT as an expensive proof of concept for the compression theory - Explains application to image domain using next pixel prediction - Presents results showing improved unsupervised learning performance 8. Linear Representations and Open Questions (35:46 - 38:27) - Discusses mystery of why linear representations form in neural networks - Compares autoregressive models to BERT for linear representations - Speculates on reasons for differences in representation quality 9. Q&A Session (38:27 - 54:37) - Addresses questions on various topics including: - Comparison to other theories in cryptography - Limitations of the compression analogy - Relationship to energy-based models - Implications for supervised learning - Importance of autoregressive modeling - Relationship to model size and compression ability - Curriculum effects in neural network training

this was a fire talk but the title could be more descriptive. First of all it's misleading and made it sound so daunting that I put it off but it was very informative

This was fantastic. I actually understand these concepts now.

What a humble and nice person Luca is

What a great introduction to tomography and quantum estimation!!

why should this be so profound and how is it relevant to real world?

I am delighted to see such good work is being carried out at Columbia. One question I have as I am midway into your presentation: What you call the effective data set: is it unique? Can you build a confidence or credible set (region) around it?

Maybe preparing first would help sounding like a lecturer instead of a highschooler spitting out random statements.

Fantastic lecture!

I can’t stop talking about the amazing potentials of BDAG, I’m sure of my great yield

Comparison, reflection, abstraction -- Immanuel Kant. Abstraction is the process of creating new concepts or ideas according to Immanuel Kant. Creating new concepts is a syntropic process -- teleological. Syntax is dual to semantics -- languages or communication, data. Large language models are using duality, if mathematics is a language then it is dual. Sense is dual to nonsense. Right is dual to wrong. "Only the Sith think in terms of absolutes" -- Obi Wan Kenobi. "Sith lords come in pairs" -- Obi Wan Kenobi. Concepts are dual to percepts" -- the mind duality of Immanuel Kant. The intellectual mind/soul (concepts) is dual to the sensory mind/soul (percepts) -- the mind duality of Thomas Aquinas. Your mind/soul converts perceptions or measurements into conceptions or ideas, mathematicians create new concepts all the time from their observations, intuitions or perceptions. The mind/soul is actually dual. Mind (syntropy) is dual to matter (entropy) -- Descartes or Plato's divided line. Your mind converts entropy or average information into syntropy or mutual information -- information (data) is dual. Concepts or ideas are therefore syntropic in form or structure. Teleological physics (syntropy) is dual to non teleological physics (entropy) -- physics is dual. Syntropy (prediction) is dual to increasing entropy -- the 4th law of thermodynamics! Duality creates reality! "Always two there are" -- Yoda. Physics is all about generalization or abstraction -- a syntropic process, teleological. Truth is dual to falsity -- propositional logic or Bayesian logic. Absolute truth is dual to relative truth -- Hume's fork. Truth is dual.

The hypothesis for learning convergence is false. There’s no universal representation of knowledge. There’s though confirmation, data and anthropomorphic biases to be learned here. See umwelt and differing latent space representations even for exact same Transformers. That’s enough to refute these claims.

It makes perfect sense for different AIs to learn similar representations of the same reality. This is similar to how science works.

So far I’m only 24 minutes into watching this, but, this seems like a really important idea? In addition to the clear practical application, it seems to me like this should also imply something about like, the theory of the development of scientific theories ? If we replace the ML model with, e.g. Newton’s law of universal gravitation (regarding “observations Isaac Newton knew about before publishing anything about the universal gravitation” as the “training set” used to produce that theory, when we consider the requirement that the gold-standard data be independent of the training set... uhh... I guess this should give a... hm, maybe this isn’t as applicable to this case as I thought. Still, seems very important!) Edit: I suppose the points at 33:00 - 35:15 should temper my uh, somewhat wild imaginings for how widely this could be applied .. and also she goes on to point out connections to previous literature dealing with somewhat similar things that I hadn’t at all heard of, I guess one can tell that I haven’t really studied statistics in much depth.. Nonetheless, I continue to be of the opinion that this is *very* cool.

'we can talk offline about this' -the evolution of language in the information age

έλα μωρε τώρε βρε

Poor man

@11:56 when the mouth detection algorithm isn't sure if it is the mouth

This was very shortly before the OpenAI crisis when he tried to get altman fired.

It is so wonderful to listen to Prof. Andrew!

what is the opposite of confirmation bias?

this could be sparks of SSI

Dear Prof Hayden, I found you a few years when I was following ProfLenSus work. I am a huge fan now and follow you avidly. Thank you for your work and I assure you some day I see you becoming one of the greatest scientists of our time. Keep up the amazing contributions and we are appreciative!

Rest in Peace, Luca.

How about tinnitus?

I was eight minutes in before I understood the title!

Cool! Not that I know enough to have real reasons to have suspected one thing or another, but, I was rather surprised that the pattern had the “local order without global order” thing in contrast to when moving on a 2D surface. I would have guessed either “columns for horizontal positions” or a 3D global order. I guess the “putting hard balls into a region will tend to produce a local optimum for packing far from the global optimum” and the bit about the interaction potential thing producing similar patterns, maybe suggests something about how the cells come to fire when? Not the specifics maybe, but, something like, “there being cells that fire at location x causes other cells to be less likely to start firing at locations near x”? I think I’d like to read a bit about the relationship between the 2D random walks and the hexagonal grid. Did I hear correctly that that was connected to representation theory? Or maybe that was just saying representations more generally and not about representation theory. 18:39 : “they see!” :)

This presale is ready to go, having just released their most recent keynote.

@ 52:55 It would be nice to know what that "Attention" block is, such as, if K-means clustering is utilized there... @43:14 That "advantage function" is interesting, would be great to have more details on it, it almost seems like two Fuzzy values. @ 34:05 That looks like a Euclidean distance (like K-means clustering).

Great talk! Thanks Irit :)

“You know…” No, Scott, I have no freaking idea. You have alien intelligence.

when tf did my feed go from neo nazi shit to this. I fw it though, didn't know what a Laplacian was until now. Always hated graph theory back in Uni cus I thought it was shallow. I wonder why we weren't taught this stuff?

Thank you for the lecture

BlockDAG has raised $49.7million in the presale That's a huge progress from this project

*"set programming guide to stupid humans always the wholly totally and illegal War option every time all the time"* so who's the fucking idiot again? Sure isn't the hardly idiotic computer and computer people!

how could one predict whether the system will reach some equilibrium state or frustrated state?

If i ever visit the USA i must attend one of these lectures. Thanks for uploading! i have been watching so many interesting lectures since i subscribed

where lens?

Fauci go homes ❤

My goodness, (playful trumpet sound ) this is fantastic! Thanks so much for sharing this publically. Curious about what goes on in their minds!

BlockDAG is the next big thing

BlockDAG has been for year so they are not just new to the game

clement is incredible :)

Awesome video

BlockDAG's advancements in mining technology open up new possibilities for efficient and profitable cryptocurrency mining

Wow guys could atleast put up some subtitles or something 😮