If these values really meant anything, then Anthropic should stop working with Palantir entirely given their work with ICE, domestic surveilance, and other objectionable activities.
PyTorch was partly inspired by the python Autograd library (circa 2015 [1]) to the point where they called their autodiff [2] system "autograd" [3]. Jax is the direct successor of the Autograd library and several of the Autograd developers work on Jax to this day. Of course, for that matter, PyTorch author Adam Paszke is currently on the JAX team and seems to work on JAX and Dex these days.
Direct indexing is WAY more valuable to US citizens living in the EU than US citizens living in the USA because of the painful intersection of MiFID II rules and US tax law (PFIC tax cancer makes buying EU domiciled funds a non-starter). Brokers will not sell US domiciled ETFs to US citizens living in the EU unless they can opt out of the consumer disclosure rules (e.g. by becoming an elective professional client of their broker under MiFID II rules). So these 2 million US expats have no choice but to manage a portfolio of individual stocks or pay exorbitant AUM fees. The first half-way decent direct indexing product that accepts US expats residing in the EU will make a killing!
Igor Markov, along with Sat Chatterjee, seem to be pursuing a bizarre vendetta (after Sat failed to take over their project) against the lead authors of the chip placement work, not some sort of intellectually honest critique.
This was covered previously in the press and on social media, with statements from a variety of prominent researchers (e.g.
[1][2][3]).
Why not try reading the paper written by Igor and try to find single instance where he launches a personal attack on the researchers, calls them names etc?
Note just the tone difference as you read Igor’s work to the stuff of his detractors. One immediately goes personal, tries to figure out motivations of the opposing counsel, talks about harassment and sounds emotional to say the least. The other has an extremely objective tone, only focuses on the subject matter, and in general reads more like a maths theorem than an activist essay.
>> Google stands by this work published in Nature on ML for Chip Design, which has been independently replicated, open-sourced, and used in production at Google.
>> The results in the Nature paper were independently replicated and validated by my team, the results were used in actual chips and Sat and his collaborators know it.
>> Furthermore, the code was open-sourced.
>> It is sad that you are providing a platform for someone's resentments.
The claims about independent replication refer to Google's circuit_training repository[1]. The UCSD team has conclusively shown this claim was materially false (see section 3 of their paper[2]).
BTW, Prof. Andrew Khang, who headed the UCSD effort, initially wrote an exteremely favorable editorial about the Nature paper[3].
The matter is way past superficial personal accusations. And the people at these Twitter links have no technical background in chip design (why would anyone listen to them?). Sergio's and Zoubin's tweets are obviously and verifyably wrong.
"The AlexNet paper is generally recognized as the paper that sparked the field of Deep Learning"
Uh not really. That would have been "A fast learning algorithm for deep belief nets" in 2006.
Also weird how this list completely ignores speech recognition. Deep learning's success in speech recognition predates AlexNet and motivated Google to create TPUs [1], and, more generally, invest in deep learning.
One paper will be the most cited. One paper will be the earliest.
Unless they are the same there is not _one_ answer, what you get is an efficient set (the earliest paper with at least X citations / the most cited paper as of D).
And of course there is the problem of making the list of relevant papers in the first place.
You are mistaken. Ladleys papers detailing the explorative method and his figurative routines adopted by “A fast learning algorithm for deep belief nets” was the firecracker that started deep learning.
Also anyone who references wired.com should be shown the door
> anyone who references [Source] should be shown the door
Well - (exportable input) -, we are past the age of editorial boards (but for The Economist, probably) and in the age of independent journalists lending efforts inconsistently to different publishers.
Honest question, which paper is this? The 2012 one? The 2006 Hinton paper was the trigger surely - that's what I used and was filled with things that worked very quickly on GPUs.
Going from 25% top-one error to 22% top-one error is a massive jump on ImageNet and very meaningful in a lot of applications. That said, there is no reason to believe attention-based models are the only way forward on image classification. The humble ResNet-50 can get near 22% top-1 error when trained properly.
Whether we need attention or not is a more interesting question on seq2seq models on text data.
Julia provides no meaningful advantages. PyTorch and JAX are too good. For typical deep learning workloads, Julia will not easily have a speed advantage. Everything goes down to the same cuDNN kernels anyway.
Julia seems like an attempt at a better matlab, but the machine learning world moved to python first.
(Also 1-based indexing is almost as obnoxious as the 24/7 Julia shill brigade.)
> (Also 1-based indexing is almost as obnoxious as the 24/7 Julia shill brigade.)
You had me at 1-based indexing. This is sadly an anti-feature of Julia as far as I'm concerned.
I know that 1-based indexing is used in Fortran, MATLAB and Lua (and other places) -- but I just find 0-based indexing more natural to me. If I grew up using 1-based indexing I'd probably be saying the opposite but this is what I prefer now.
Julia is probably a more cleaner system overall. But then so is Deno compared to Node. The problem is that the alternative needs to be many times better not merely better. The "network" effort is quite strong in Python scientific computing this is still outweighing some of the real benefits of Julia.
It is always good to have alternatives. The quality of GCC improved when Clang came along. Julia's presence will keep Python on its toes. Both platforms will keep improving.
But, once again, Julia's 1-based indexing is something that adds unnecessary friction, for me at least.
Julia has this. It’s called OffsetArrays.jl. Ironically, it’s a common source of bugs because library authors don’t always anticipate them and loop over 1:length(A) rather than eachindex(A).
There are plenty of free, open access journals that are reputable.
JMLR http://www.jmlr.org/ is quite successful. There are some fields, such as machine learning, that are not dominated by for-profit journals. Why is this possible in some fields and not others? My answer would be that it is possible in all fields, but incumbency advantages can be very strong and coordination across academic volunteers can be more difficult when they are individually less secure.
reply