Hello and thank you for tuning in to Issue #510!
Once a week we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.
Seeing this for the first time? Subscribe here:
If you find this newsletter helpful to your job, consider becoming a paid subscriber here: https://datascienceweekly.substack.com/subscribe :)
If you don’t find this email useful, please unsubscribe here.
And now, let's dive into some interesting links from this week :)
Champion-level drone racing using deep reinforcement learning
First-person view (FPV) drone racing is a televised sport in which professional competitors pilot high-speed aircraft through a 3D circuit. Each pilot sees the environment from the perspective of their drone by means of video streamed from an onboard camera. Reaching the level of professional pilots with an autonomous drone is challenging because the robot needs to fly at its physical limits while estimating its speed and location in the circuit exclusively from onboard sensors. Here we introduce Swift, an autonomous system that can race physical vehicles at the level of the human world champions. The system combines deep reinforcement learning (RL) in simulation with data collected in the physical world…
Google Gemini Eats The World – Gemini Smashes GPT-4 By 5X, The GPU-Poors
Google had all the keys to the kingdom, but they fumbled the bag. A statement that is obvious to everyone. The statement that may not be obvious is that the sleeping giant, Google has woken up, and they are iterating on a pace that will smash GPT-4 total pre-training FLOPS by 5x before the end of the year. The path is clear to 20x by the end of next year given their current infrastructure buildout. Whether Google has the stomach to put these models out publicly without neutering their creativity or their existing business model is a different discussion. Today we want to discuss Google’s training systems for Gemini, the iteration velocity for Gemini models, Google’s Viperfish (TPUv5) ramp, Google’s competitiveness going forward versus the other frontier labs, and a crowd we are dubbing the GPU-Poor…
Reinforced Self-Training (ReST) for Language Modeling
Reinforcement learning from human feedback (RLHF) can improve the quality of large language model's (LLM) outputs by aligning them with human preferences. We propose a simple algorithm for aligning LLMs with human preferences inspired by growing batch reinforcement learning (RL), which we call Reinforced Self-Training (ReST)…Our results show that ReST can substantially improve translation quality, as measured by automated metrics and human evaluation on machine translation benchmarks in a compute and sample-efficient manner…
Alternative investments are gaining momentum, with 67% of institutional investors predicting that a portfolio including 20% alternatives will outperform the traditional 60/40 stock-bond investment mix with lower volatility.
Hedonova, a hyper-diversified hedge fund open to accredited investors has outperformed the S&P 500 by 17%.
Hedonova invests in various alternative assets like equipment finance, litigation finance, startups, wine, art, etc. The SEC regulated fund is backed by the likes of Morgan Stanley and is open to accredited investors with a low minimum investment of $5,000.
The fund was also awarded Best Multi-Strategy Hedge Fund at Hedgeweek European Awards 2023.
* Want to sponsor the newsletter? Email us for details --> team@datascienceweekly.org
Yet Another ICML Award Fiasco
I wrote a blog post on the ICML award fiasco: They gave an outstanding paper award to the D-Adaptation paper, that contains worse results that the ones in papers from 9 years ago. Also, this is not the first time that ICML gives awards to questionable or even plainly wrong papers. I believe this might start a serious conversation about "stochastic" awards, and the super noisy reviews in machine learning conferences…
Influencers in data are doing no justice to the industry [Reddit Discussion]
A few of them aside, most are writing stuff just to fill the gaps. Nothing meaningful, just piece after piece of barely important content…Data Twitter, on the other hand, is much more cliquey. Guarding and almost gatekeeping their world. They don't even like the LinkedIn data influencers and sometimes even hate on people in other parts of Data Twitter too…
RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitation
For robots to be useful outside labs and specialized factories we need a way to teach them new useful behaviors quickly…In this work we explore dense tracking as a representational vehicle to allow faster and more general learning from demonstration. Our approach utilizes Track-Any-Point (TAP) models to isolate the relevant motion in a demonstration, and parameterize a low-level controller to reproduce this motion across changes in the scene configuration. We show this results in robust robot policies that can solve complex object-arrangement tasks such as shape-matching, stacking, and even full path-following tasks such as applying glue and sticking objects together, all from demonstrations that can be collected in minutes…
GPT Pilot – a PoC for a scalable dev tool that writes production ready apps from scratch as the developer oversees the implementation
In this blog post, I will lay out a PoC (proof of concept), with a prototype open sourced on Github, for a dev tool that uses GPT-4 to code an entire, production-ready app…The main idea is that now AI can write most of the code for an app, even 95%, but an app doesn’t work at all if all code doesn’t work completely. So, until we get to AGI, the developer will be needed to oversee the app development, acting as a team lead who gets involved when the AI developer gets stuck. I started this project thinking about a couple of things…
Inside a Google Cloud TPU Data Center [Video]
Get an inside look at the magic of Google Cloud TPUs, including a rare inside view of the data centers where it all happens. Customers use Cloud TPUs to run some of the world's largest AI workloads and that power comes from much more than just a chip. In this video, take a look at the components of the TPU system, including data center networking, optical circuit switches, water cooling systems, biometric security verification and more…
GPU Puzzles - Solve puzzles. Learn CUDA
GPU architectures are critical to machine learning, and seem to be becoming even more important every day…This notebook is an attempt to teach beginner GPU programming in a completely interactive fashion. Instead of providing text with concepts, it throws you right into coding and building GPU kernels. The exercises use NUMBA which directly maps Python code to CUDA kernels. It looks like Python but is basically identical to writing low-level CUDA code. In a few hours, I think you can go from basics to understanding the real algorithms that power 99% of deep learning today…
Tensor Puzzles - Solve puzzles. Learn PyTorch
When learning a tensor programming language like PyTorch or Numpy it is tempting to rely on the standard library (or more honestly StackOverflow) to find a magic function for everything. But in practice, the tensor language is extremely expressive, and you can do most things from first principles and clever use of broadcasting. This is a collection of 21 tensor puzzles. Like chess puzzles these are not meant to simulate the complexity of a real program, but to practice in a simplified environment. Each puzzle asks you to reimplement one function in the NumPy standard library without magic…
Geographic data analysis in R and Python: comparing code and outputs for vector data
In this blog post, we talk about our experience teaching R and Python for geocomputation. The focus of the blog post is on geographic vector data, meaning points, lines, polygons (and their ‘multi’ variants) and the attributes associated with them. Geographic data analysis is a broad topic and in a later post we will cover raster data, meaning gridded data such as satellite images…
Teaching with AI
We’re releasing a guide for teachers using ChatGPT in their classroom—including suggested prompts, an explanation of how ChatGPT works and its limitations, the efficacy of AI detectors, and bias…
For the Gucci Global Data Science team based in Milan, we are currently seeking an English speaking Senior Data Scientist.
In this role, you will report to the Global Corporate Director of Data Science and help the business in central decision making processes, have the opportunity to lead the technical development of a small team of bright and driven data scientists, collaborate with teams across different regions and areas of the business leveraging Gucci’s rich data sources, infrastructure and the power of machine learning and advanced analytics.
Influential, innovative and progressive, Gucci is reinventing a wholly modern approach to fashion.
The Gucci Data Science team is the new kid on the block, bringing fresh perspectives and a new way of working that will help the company in continuing its innovation path leveraging the power of data and ML.
Apply here
Want to post a job here? Email us for details --> team@datascienceweekly.org
GPT in 60 Lines of NumPy
In this post, we'll implement a GPT from scratch in just 60 lines of numpy. We'll then load the trained GPT-2 model weights released by OpenAI into our implementation and generate some text…
A Visual Introduction to Neural Networks
Here, we’re going to explore neural networks. Neural networks are a type of machine learning model inspired by the structure and function of the human brain. They are designed to recognize patterns and learn from data in a way that enables them to make predictions. They do this using a method called backpropagation. Before diving into the full details, let’s take a bit of a look at the history of neural nets and where they were first used…
My take on “How much math do I need?” [Reddit Discussion]
I see this question come up so many times, I wanted to write up my own “comprehensive” answer to this. Really… it depends. The answer is on a spectrum, and I wish to break down this spectrum in this post from the simplest, to the most advanced…
* Based on unique clicks.
** Find last week's issue #509 here.
Thank you for joining us this week :)
All our best,
Hannah & Sebastian
P.S.
If you found this newsletter helpful to your job, please consider becoming a paid subscriber here: https://datascienceweekly.substack.com/subscribe :)
Copyright © 2013-2023 DataScienceWeekly.org, All rights reserved.
