An easytounderstand introduction to neural networks how can a computer learn to recognize patterns and make decisions like a human brainThe Google Technology Stack Michael Nielsen. Posts. Page. Rank and Map. Reduce. Data mining and machine learning. About. Part of what makes Google such an amazing engine of innovation is their internal technology stack a set of powerful proprietary technologies that makes it easy for Google developers to generate and process enormous quantities of data. According to a senior Microsoft developer who moved to Google, Googlers work and think at a higher level of abstraction than do developers at many other companies, including Microsoft Google uses Bayesian filtering the way Microsoft uses the if statement Credit Joel Spolsky. This series of posts describes some of the technologies that make this high level of abstraction possible. The technologies Ill describe include The Google File System a simple way of accessing enormous amounts of data spread across a large cluster of machines. From Developer to Time Series Forecaster in 7 Days. Python is one of the fastestgrowing platforms for applied machine learning. In this minicourse, you will. Acts as a virtual file system that makes the cluster look to developers more like a single machine, and eliminates the need to think about details like what happens when a machine fails. Bigtable Building on the Google File System, Bigtable provides a simple database model which can be run across large clusters. This allows developers to ignore many of the underlying details of the cluster, and concentrate on getting things done. Map. Reduce A powerful programming model for processing and generating very large data sets on large clusters, Map. Reduce makes it easy to automatically parallelize many programming tasks. It is used internally by Google developers to process many petabytes of data every day, enabling developers to code and run simple but large parallel jobs in minutes, instead of days. Together, these technologies make it easy to run large parallel jobs on very big data sets. Running on top of this technology stack are many powerful data mining and machine learning algorithms. Ill describe at least two of these and may describe more, including statistical approaches to spell checking and machine translation, and recommendation algorithms. In addition to understanding how these technologies work, well also Develop toy implementations of some of the technologies. Investigate some related open source technologies, such as Hadoop, Couch. DB, Nutch, Lucene, and others. Ive never worked at Google. The posts are based on the published literature, especially some key technical publications released by Google describing the Google File System, Bigtable, Map. Reduce, and Page. Rank. Im sure there are some significant differences between the material I will describe, and the current internal state of the art. Still, we should be able to build up a pretty good picture of a data intensive web company, even if its not accurate about Googles particulars. Exercises and problems The posts will contain exercises for you to do to master the material, and also some more challenging and often open ended problems. Note that while Ive worked on many of the problems, I by no means have full solutions. Friend. Feed room There is a Friend. Feed room for the series. Accompanying lectures Im basing a lecture series in Waterloo, Ontario, on the posts. Heres some details about the lectures. Backpropagation Program Ma' title='Backpropagation Program Ma' />Audience The audience will include both theoretical physicists and developers the lectures should be accessible to both sets of people. Time Lectures will be held every Tuesday, 7 0. The exception is the first lecture, which will be Thursday, December 4, 2. Location Aide. RSS have kindly offered us a room for the course, at 5. King Street South, Waterloo its opposite the Brick Brewery, and near the Hospital. To get into the building, you need someone from Aide. RSS to let you in. If no one from Aide. RSS is by the entrance, call 5. The lectures are based primarily on the following sources Original Page. Rank paper The Page. Rank Citation Ranking Bringing Order to the Web, by Lawrence Page, Sergey Brin, Rajeev Motwani, and Terry Winograd 1. Textbook on Page. Rank and related ideas Googles Page. Rank and Beyond The Science of Search Engine Rankings, by Amy N. Langville and CarlD. Meyer 2. Overview of Googles early architecture The Anatomy of a Large Scale Hypertextual Web Search Engine, by Sergey Brin and Lawrence Page 1. More recent overview of Googles architecture Web Search for a Planet The Google Cluster Architecture, by Luiz Barroso, Jeffrey Dean, and Urs Hoelze 2. Original Google File System paper The Google File System, by Sanjay Ghemawat, Howard Gobioff, and Shun Tak Leung 2. Original Bigtable paper Bigtable A distributed storage system for structured data, by Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber 2. 00. 6. Original Map. Reduce paper Map. Reduce Simplified Data Processing on Large Clusters, by Jeffrey Dean and Sanjay Ghemawat 2. Related resources include First of five lectures on Map. Ibis Computer Program there. Reduce. Lecture course on scalable systems at the University of Washington, covering many similar topics. Videos of the lectures are included. Many more resources will be posted in the Friend. Feed room for the course. Schedule Deep Learning Summit Montreal. Lisha Li. Amplify Partners. WELCOMECONVOLUTIONAL NEURAL NETWORKS. Jean Franois Lalonde. Universit Laval. Deep Learning for Computer Graphics Learning to Estimate Lighting From Photographs. Jean Franois Lalonde. Universit Laval. Deep Learning for Computer Graphics Learning to Estimate Lighting From Photographs. We propose an automatic method to infer high dynamic range illumination from a single, limited field of view, low dynamic range photograph of a scene. In contrast to previous work that relies on specialized image capture, user input, andor simple scene models, we train an end to end deep neural network that directly regresses a limited field of view photo to HDR illumination, without strong assumptions on scene geometry, material properties, or lighting. This allows us to automatically recover high quality HDR illumination estimates that significantly outperform previous state of the art methods. Consequently, using our illumination estimates for applications like 3. D object insertion, we can achieve compelling, photorealistic results. Jean Franois Lalonde is an Assistant Professor in Electrical and Computer Engineering at Laval University, Quebec City, since 2. Previously, he was a Post Doctoral Associate at Disney Research, Pittsburgh. He received a B. Eng. Computer Engineering with honors from Laval University, Canada, in 2. He earned his M. S at the Robotics Institute at Carnegie Mellon University in 2. Prof. Martial Hebert and received his Ph. D., also from Carnegie Mellon, in 2. Profs. Alexei A. Efros and Srinivasa G. Narasimhan. His Ph. D. thesis won the 2. CMU School of Computer Science Distinguished Dissertation Award. After graduation, he became a Computer Vision Scientist at Tandent, Inc., where he helped develop Light. Brush, the first commercial intrinsic imaging application. He also introduced intrinsic videos at SIGGRAPH 2. Tandent. His research focuses on lighting aware image understanding and synthesis by leveraging deep learning and large amounts of data. Jasper Snoek. Google Brain. Deep Learning on DNA. Jasper Snoek. Google Brain. Deep Learning on DNAI will talk about some recent work on deep learning applied to genomics. Specifically, we use deep convolutional networks to automatically annotate the regulatory regions of genes that govern many important biological processes. The hope is then to unlock new insights in genomics, such as the connection between genetic mutation and disease, by reverse engineering the predictions of a very accurate model. Jasper Snoek completed his Ph. D in machine learning at the University of Toronto in 2. He subsequently held postdoctoral fellowships at the University of Toronto, under Geoffrey Hinton and Ruslan Salakhutdinov, and at the Harvard Center for Research on Computation and Society, under Ryan Adams. Jasper co founded the machine learning startup Whetlab, which was acquired by Twitter in 2. Currently, he is a research scientist at Google Brain in Cambridge, MA. Eric Humphrey. Spotify. Advances in Deep Architectures and Methods for Separating Vocals in Recorded Music. Eric Humphrey. Advances in Deep Architectures and Methods for Separating Vocals in Recorded Music. Source separation of audio mixtures, with an emphasis on the human voice, remains one of the enticing unsolved challenges in audio signal processing. This challenge is amplified in the context of recorded music, where often many sound sources are intentionally correlated in both time and frequency. In this talk, we present recent advances in the state of the art for separating singing voice and accompaniment in popular music audio recordings, leveraging semi supervised datasets mined from a large commercial music catalog. In addition, we explore the effects of combining deep convolutional U Net architectures with multi task learning for vocal separation. Eric J. Humphrey is a research scientist at Spotify, and acting Secretary on the board of the International Society for Music Information Retrieval ISMIR. Previously, he has worked or consulted in a research capacity for various companies, notably THX and Muse. Ami, and is a contributing organizer of a monthly Music Hackathon series in NYC. He earned his Ph. D. at New York University in Steinhardts Music Technology Department under the direction of Juan Pablo Bello, Yann Le. Cun, and Panayotis Mavromatis, exploring the application of deep learning to the domains of audio signal processing and music informatics. When not trying to help machines understand music, you can find him running the streets of Brooklyn or hiding out in his music studio. Hugo Larochelle. Google Brain. Generalizing From Few Examples With Meta Learning. Hugo Larochelle. Google Brain. Generalizing From Few Examples With Meta Learning. Deep learning has been successful at many AI tasks, largely thanks to the availability of large quantities of labeled data. Yet, humans are able to learn concepts from as little as a handful of examples. Ill describe a framework that has recently been used successfully to address the problem of generalizing from small amounts of data, known as meta learning. In meta learning we develop a learning algorithm that itself can produce and train a learning algorithm for some target class of problems. Ill review some examples of successful use of meta learning to produce good few shot classification algorithms. Hugo Larochelle is Research Scientist at Google and Assistant Professor at the Universit de Sherbrooke Ude. S. Before, he was working with Twitter and he also spent two years in the machine learning group at University of Toronto, as a postdoctoral fellow under the supervision of Geoffrey Hinton. He obtained his Ph. D. at Universit de Montral, under the supervision of Yoshua Bengio. He is the recipient of two Google Faculty Awards. His professional involvement includes associate editor for the IEEE Transactions on Pattern Analysis and Machine Intelligence TPAMI, member of the editorial board of the Journal of Artificial Intelligence Research JAIR and program chair for the International Conference on Learning Representations ICLR of 2. Kyunghyun Cho. New York University. Deep Learning, Where Are You GoingKyunghyun Cho. New York University. Deep Learning, Where Are You Going There are three axes along which advances in machine learning and deep learning happen. They are 1 network architectures, 2 learning algorithms and 3 spatio temporal abstraction. In this talk, I will describe a set of research topics Ive pursued in each of these axes. For network architectures, I will describe how recurrent neural networks, which were largely forgotten during 9. I continue on to discussing various learning paradigms, how they related to each other, and how they are combined in order to build a strong learning system. Along this line, I briefly discuss my latest research on designing a query efficient imitation learning algorithm for autonomous driving. Lastly, I present my view on what it means to be a higher level learning system. Under this view each and every end to end trainable neural network serves as a module, regardless of how they were trained, and interacts with each other in order to solve a higher level task. I will describe my latest research on trainable decoding algorithm as a first step toward building such a framework. Kyunghyun Cho is an assistant professor of computer science and data science at New York University.