Data science is a discipline that uses scientific methods, statistics and computer science in order to solve business problems using data. Recently it has become incredibly popular, thanks to the multitude of open source frameworks, cheaper compute resources and especially the increase in the amount of data that is being generated and captured. Data science can be applied to a variety of areas, but its biggest impact recently has been in the context of business analytics and decision processes.
It was a fantastic learning experience. That’s how I would describe the week I spent there. The Massachusetts Institute of Technology is one of the most prestigious universities. And the Supply Chain department is widely considered one of the best in the world. It would have been hard to imagine that the online course I started almost three years ago resulted in studying in person at MIT, in a fantastic learning experience.
During one of my recent flights I completed a short book from Seneca, “On the shortness of life”. Seneca (the Younger) was a roman philosopher, who eventually became an advisor to emperor Nero. “On the shortness of life” is a letter to Paulinus, his father in law, in which he reviews many timeless lessons on how to live life.
Two years and five online courses later I finished another degree: A Micromasters in Supply Chain. This is very meaningful for me because the time studying Supply Chain has matched almost perfectly the time that I was part of the Supply Chain team at Google. I was able to learn about the big picture vision of Supply Chains as well as the details of the day to day in my team. I worked with many others in order to build data centers and deploy compute capacity.
I used Ghost for a while. It’s a simple and straightforward platform that simply gets out of the way and lets you write and publish a blog. And I’ve been a happy customer of the hosted version, paying a bit less than $100 a year for it. However, recently Ghost decided to raise prices by more than 100% and target a higher end market. At which point it’s hard for me to justify the expense. So I began looking for alternatives.
This is right in the intersection of Supply Chain and Artificial Intelligence.
This is one of the most interesting visualizations I have seen in a while. The folks at Kiln created a map of merchant ships around the world, including a breakdown of the contents of the ships, as well as CO2 emissions as a proxy for environmental impact. You can see a fully functioning version of the map below:
I found out that Google is open sourcing (more or less) the build system used internally, Blaze. The open source version is called Bazel.
The Astronauts, the team behind The Vanishing of Ethan Carter, just published a great summary of how they are using photogrammetry in order to create some of the most realistic 3D scenes ever seen in a game.
Dave Cutler is one of the best software engineers in Microsoft, and arguably in the entire world. The Microsoft News Center recently published a long feature on Cutler’s contributions to the company, which covers his many decades of revolutionary work in the industry.
Right before the end of last year I finished the second Supply Chain Course on edX. This was a fantastic course, easily the best one I’ve taken online, and probably the most useful for my day to day job (in Supply Chain of course).
Springer, the academic publisher, has been maintaining an open access program for a while. They have released thousands of books in their website, free to download. Apparently it’s been going on for a while but I only found out recently.
Andrew Ng is one of the world’s top experts in Machine Learning, and had a tremendous influence making Deep Learning a popular technique for problems in speech recognition, image captioning and many others. In particular, he was one of the founders of the Google Brain group a few years ago.
A few days ago I read an article on Nature Scientific Reports about Fungi in the brain of Alzheimer’s Disease patients.
Lately it’s been “supply chain this” and “supply chain that” for me. So here is another blog post about Supply Chain! I’ve been studying for two online courses in EdX: I talked a bit about the first course in a previous blog post, and I just started the second part.
Please check also the first blog post in this series about Supply Chain.
I can’t believe I didn’t read Snow Crash until now. Snow Crash is a novel by Neal Stephenson in which he introduced the Metaverse. What we would call today a virtual reality world. While I was reading it, I had to remind myself that the book was published in 1992. More than 10 years ahead of Second Life and WOW.
I recently switched jobs at Google. After two years working in Ads I decided that I needed a change, and I joined the Supply Chain team. Five months later I can start to recap my experience.
The last few months I’ve been using a Jawbone tracker. It’s a small bracelet which keeps track of steps and sleep patterns. And it’s been very useful to help me sleep a bit longer. From what I understand most people don’t get enough sleep, so I recommend checking it out. A while back I achieved a nice milestone. A thousand hours of sleep and one million steps.
I recently ported the blog to Ghost. Now it’s time to say goodbye to AppEngine. I had to make several updates in order to keep the application running, and the amount of work was just too big to justify. Now I happily let Ghost run the blog and I just write.
Yesterday I finally finished porting over the blog content from the old platform into Ghost. And today I set up a new mailing list using MailChimp. The updates to the blog are almost complete…
Let’s see how this works out. New blogging platform. Hello Ghost!
Everybody who has spent some time working in a business of any size knows that Excel is ubiquitous. It’s surprising that there are so few companies out there working in the spreadsheet world.
A new detailed simulation of the universe:
A raytracer in the back of a business card! Here is the code:
One of my favorite features from github is the activity chart that appears in the user profile. For example, this one from user jmcnamara:
Hello, internets! I just redesigned the look & feel of the blog. The new design has bigger font sizes and less distractions. It's optimized for mobile so it should look much better than before on Android and iOS phones, and especially on iPad.
A few weeks ago I had the chance to read Masters of Doom by David Kushner and found it very interesting. It tells the story of John Carmack and John Romero, the founders of id Software and authors of some the most revolutionary videogames ever.
A while back I found a beautiful website on Mathematics and I wanted to share the link: Mathigon has a series of introductory articles and many images, figures and interactive visualizations on a variety of mathematical topics.
I guess this is the advanced version of the companion cube:
I haven't done any kind of 3D design in many years, but I was thinking last week that it would be fun to get back to it. And while I was looking at software available for Linux, I found Blender.
A primality test in one line of Perl:
I haven't had much time to write lately, work is keeping me busy. But today I wanted to share a site I found recently and I think is really interesting. It's essentially a git tutorial, but with a twist.
A team from USC engineered microscopic markers that light up synapses in a living neuron in real time.
Project Loon is a network of balloons traveling on the edge of space, designed to connect people in rural and remote areas, help fill coverage gaps, and bring people back online after disasters.
A few days ago I found out about codesearch. It's a group of command-line tools to index and search code using regular expressions. It was originally developed by fellow googler Russ Cox. It uses the same algorithms that were behind Google Code Search.
A stop-motion movie made with two-atom molecules.
A few weeks ago I gave a small webinar for a group of students from MIT.
Over the last few months I have been going to a few technology meetups here in Seattle, and I enjoyed it very much. It's great to see all the cool technology that local companies are creating. Here are three of the most popular tech meetups in Seattle:
Card Games, Complete Subgraphs, Constant Weight Error Correcting Codes and Finite Projective Geometries
I was recently in a store buying a gift for a friend and found a card game that seemed interesting. The cards have small symbols which are arranged in a way such that between two cards there is always one symbol in common.
Probably everyone that is reading this post knows that Google has decided to discontinue Google Reader. This comes as a surprise to many because of the large and very passionate user base, including myself.
An electron microscope has imaged threads of DNA directly for the first time:
On December 3rd, the world’s governments will meet to update a key treaty of a UN agency called the International Telecommunication Union (ITU). Some governments are proposing to extend ITU authority to Internet governance in ways that could threaten Internet openness and innovation, increase access costs, and erode human rights online.
A few days ago I had the chance to watch a talk by Steve Souders, who is one of the world's top experts on website performance. I started thinking about how to apply some of the techniques that he mentions to my site, and I decided to replace the icons that appear at the top. These are links to subscribe to blog updates, the RSS feed, the feedburner email subscription, and links to twitter and Google+, which I update when there are new blog posts. I decided to replace these image icons with icons made using only CSS.
I've been using emacs for a few years now. And even though I am not an expert I appreciate how powerful it is. It never ceases to amaze me how many configuration options are available.
Today, I'm just going to do a little bit of self-promotion. A few days ago Google announced the launch of the new Chromebook.
Today I would like to share a Youtube video course on the History of Mathematics. I haven't watched all of it, but I liked the part that I saw. Here is one of the videos, on projective geometry:
Like many Software Engineers and Computer Scientists, I've always been interested in Artificial Intelligence (AI). When people outside of the field think about AI, they normally think about computers that can behave like humans in the real world. This branch of AI is called Artificial General Intelligence. And despite how popular it is in science fiction, in the real world there has been little progress towards the goal of creating an intelligent machine.
Today I'm just going to share a little bit of code to download content from the DataStore in Google AppEngine. A few days ago I was worried about not having a recent backup of my blog. So I decided to write some code to download all the blog content in a convenient ZIP file. Here it is:
I was recently in the Denver area, and while driving around I saw directions to a place called Mesa. This reminded me of Black Mesa, which is the location of the original Half Life game. The game actually starts after some kind of nuclear accident, which opens an inter-dimensional portal.
There is a wide variety of Mathematics used at Google. For example Linear Algebra in the PageRank algorithm, used to rank web pages in search results. Or Game Theory, used in ad auctions, or Graph Theory in Google Maps. At Google there are literally dozens of products which use interesting Mathematics. These are not just research prototypes, but real Google products; in which Mathematics play a crucial role.
Recently I've been learning a little bit about Google Guice (pronounced like juice) and dependency injection. In case you are not familiar with dependency injection, a good introduction is this article by Martin Fowler.
I would like to share a quick note about some writing tools that I've been using lately.
In an international scientific breakthrough, a Griffith University research team has been able to photograph the shadow of a single atom for the first time.
At some point or another, pretty much everybody has had to bulk process all the images in a directory. Either to resize them to a smaller size to share them, or do some other simple transformation.
Normally I write all the posts in this blog manually, including all the HTML, paragraphs, links, images, etc. But a few days ago I found a great in-browser HTML5 editor, http://xing.github.com/wysihtml5/.
If you are not familiar with Synthetic Biology, you should watch the following video. It's a very well made introduction to one of the most promising fields in technology.
Welcome to the new sleek and minimalistic design of the blog!
Two very cool pieces of technology that I saw in the news recently:
During the 20th century the life expectancy grew from an average of 47 years to about 70-80 years, in part because of improved healthcare and hygiene as well as the development of antibiotics and vaccines. In some places even greater incidence of extraordinary life spans has even been observed, like the Japanese from Okinawa, who have the highest number of centenaries in the world.
Just for trivia, this is how I implemented the SOPA protest page yesterday.
I've been reading a little bit lately about robots. It's amazing how fast robotics has advanced in the last ten years. On the one hand we have several million robots doing useful tasks in people's homes. I'm talking of course about the Roomba.
Cryptography studies how to process data to transmit it securely, even if someone access the information in transit. For thousands of years cryptography has been a key military technology. For example, Julius Caesar used a simple encryption technique to communicate with his generals. Starting from a message text, the method replaces a letter for another letter in the alphabet that comes after one or several positions.
Google just announced that App Engine is going to be out of preview soon. And there is a new pricing model as part of the changes. So I decided to take a look at the resource usage of this blog and found out that the RSS feed is the most expensive request.
Human interest in the possibility of rejuvenation begins in ancient times. Around 200BC the Chinese emperor Qin Shi Huang became obsessed with his own mortality and sent the famous alchemist Xu Fu to find the Peng Lai islands (where immortal beings were believed to live) and return with the elixir of life. Starting in the Middle Ages, European alchemists began a quest for a substance that was not only able to transform metal into gold, but also rejuvenate whoever held it: the philosopher's stone. These myths have reached the present day, and feature in works such as J. K. Rowling's (hugely successful) first Harry Potter novel.
I figured out how to add authorship markup in the blog! There is a nice tutorial here, and complete documentation in the official Webmaster Tools authorship help page. But it basically comes down to this: There are two options, with 2 or 3 links.
Just wanted to leave a note on two new cool gadgets that I found online.
I read a few days ago an interesting post about what are the best fonts for programmers (monospaced fonts) at Hivelogic. And I decided to try out one of the fonts, Inconsolata. Here you have a screenshot of emacs with it.
Google+ is finally here! Google+ is the new social network from Google. It has a lot of cool features that make it better than anything else out there, like for example, the hangouts, which are like Skype, but with many friends at the same time.
Google just made the +1 button available for websites. The +1 button is a gadget that allows people to recommend content on the web.
Ive been a long time user of Google Reader. It worked great for me until a few weeks ago, when I started noticing it becoming slower. Also, I realized that many of the feeds that I frequently read were updated less frequently that I wanted. And it will only keep unread posts for up to 30 days. Anyway, my biggest issue with Reader is that its a web based RSS aggregator. Which means that it can be accessed from anywhere. Which really means that I may spend precious time reading feeds while I should be doing more useful stuff (like working...).
The Riemann Zeta Function
I decided to read again Transcend, and this time I took notes. Here are a few tips, in very synthesized form, to have better health and ensure greater longevity.
Complex Analysis is the branch of mathematics that studies functions of complex numbers. These numbers are those given by a + bi, where i is the imaginary unit, the square root of -1. We can think of complex numbers as points in a plane, where the x coordinate indicates the real component and the y coordinate indicates the imaginary component.
There are several ways to create fault tolerant systems. One of the most common is simply to duplicate the system. For example in a web application stack, we could have a master database and a slave database, which contain the same duplicated data. Also two web servers and a load balancing system that splits the load among the servers. In case of a hardware failure in a web server or a database the remaining server may be able to handle all the load while the broken one is being repaired. Replication of data and redundancy of servers is one of the reasons that some sites are highly available.
Recently I have been reading about the best way to design a graph / visualization / infographic to express a particular message. Some cases are easier than others. To show distribution of a total into parts one chart that works well is the Pie Chart. Take a look at this example:
This one is going to be a small post. A friend of mine told me about the Facebook Like buttons. One can put them in a site or in a blog post, and people can click on it to indicate that they like that item. Then the blog post or the site will appear in the Facebook homepage of that person. I checked it out and its actually pretty easy.
This one if off topic, but I couldn't resist posting it. If you have traveled by air recently you know about the porno-scanners. The TSA is forcing everyone to pass through the scanner or be subject to a thorough pat-down. So they will either watch you naked or touch all your body, including your private parts. And this includes you, your wife, your children and your grandma.
Google recently launched a new promotion for Webmaster Central and Webmaster Tools, which are undoubtedly the most important resources that the webmasters of the entire world have to optimize their sites for search (Disclaimer: I work at Webmaster Tools so Im kind of slightly biased).
During the past week shocking news have stormed through the world of Theoretical Computer Science. A researcher from HP Labs, Vinay Deolalikar claimed that he had a proof that P NP.
The folks at Zero Hedge published a very interesting post about high frequency trading and market manipulation. I am not going to discuss economy or finance now, but there is something about that post that is quite shocking. The images which show the prices and volumes of the orders routed to the market have a certain aesthetic appeal.
I just watched this fascinating talk from Murray Gell-Mann. The core idea of the talk is that in Physics, as in so many other areas of Science and Mathematics, incredibly complex ideas can be expressed in a simple and concise way. This is a remarkable property of the universe in which we live, and in that fact there is an amazing beauty.
Last week I attended Edward Tufte's course on data visualization here in Seattle. For those who don't know him, Tufte is one of the world's top experts on information visualization. Here is a short description, from his website:
A few weeks back I had a conversation with a friend about different places in the US where she was considering moving. And one of the factors was the dating scene in each one of those places. This brought to my memory a graph that probably many of you have already seen:
This quarter I am leading a study group in Machine Learning at Google's Kirkland Office. And while I was looking for datasets and resources I found Andrew Ng's course in Machine Learning at Stanford. All the lectures are available online at YouTube. You can find the links in the Stanford Engineering Everywhere Machine Learning course page.
Over the last few decades hundreds of companies, thousands of jobs and billions of dollars of real wealth were created by startups with the support of angel investors. The possibility of a small company to obtain seed funding with little more than an idea is indeed one of the most important wealth creation engines of America. Now this possibility, this engine, is at stake.
Infographics, or information graphics are visual representations of data. In a good visualization or representation of a data set, the author expresses an idea that is deeper that the data itself. A good visualization conveys a message that is clear and helps to extract conclusions, but also a message that is precise and based on the data, without transforming or manipulating the data in dishonest ways.
SEO stands for Search Engine Optimization, and is the process of improving a website's structure and content in order to make it easy for search engines to gather the pages and display them in search results in the best position possible.
Mathematical research is traditionally seen as a one-man job. To quote Jean Dieudonn in The Music of Reason:
Google recently released a new set of tools for graphics and interactive visualizations called Google Chart Tools. Google Chart Tools replaces the previous Charts API (for static images) and Visualization API (for dynamic graphics). And it combines both APIs within a single framework. Here is a link to the official announcement.
Over the last few weeks this blog has changed dramatically. It looks pretty much the same as when it started but under the covers the code of the blogging platform, Nounoublog is very different. I am going to talk about three of the features that I have been working on lately:
My brother David is a film director. He has been making short film for a few years, he has even won a few prizes. Most of his work is at tpmpictures.com. Today I just wanted to show his last piece of work, a science fiction short about the end of the world. The short is in Spanish but with English subtitles.
A couple of days ago I started reading TRANSCEND, the new book from Ray Kurzweil and Terry Grossman. The book starts from the principle that our knowledge of medicine and biology is increasing to a point where we can start to control effectively how fast our own bodies age. And even more important the amount of knowledge that we gather is increasing over time. If the trend continues we may reach a point where we can effectively reverse engineer our bodies in order to avoid aging.
This Monday I published my article on MapReduce for integer factorization in arXiv. The article is essentially the same that can be downloaded in the research section of this site. So if you have already checked it out, you won't find anything new. However I am very excited because it is my first addition to arXiv.
Recently I published the code of MapReduce for Integer Factorization. It is available under the Apache 2.0 License in Google Code. It includes everything necessary to run in Apache Hadoop, as well as the numerical libraries used. It has no dependencies apart from the last version of Hadoop.
If you have never made a web application it may seem daunting. There are hundreds of alternative technologies and frameworks out there. And web apps development is quite different from client applications, which is what most developers are used to.
Visualizations are simply ways of representing data. But if they are good, they can bring us deep insights, that go well beyond what is possible to understand by simply looking at the raw data.
Hello everybody. This is the first post in my new blog. This is not your common Wordpress or Blogger blog. It runs on a custom blogging platform made from scratch, on top of Google App Engine. Soon I will add a couple of posts about how it’s done, and I will release the code of the platform.
Here are some of the projects that I worked on over the last few years:
FinanceAI is an open source project with the goal of providing advanced Artificial Intelligence, Statistical and Mathematical tools for amateur and sophisticated investors. The purpose was to develop a complete algorithmic trading platform with comprehensive AI and Quantitative Finance libraries. It would also provide high performance algorithms. I started this project in early 2008, but so far it didn’t get past the first version, mostly because of lack of time.