MapReduce and Pregel
Tech Talk
Posted
by
Jeena
Today I was in Gotenburg at the Chalmers University of Technology at my first Google Tech Talk:
Abstract: This talk will present two methods for processing of really large data sets. The first approach, MapReduce, is a data parallel computing framework and probably Google's most important infrastructure bit. The second method, Pregel, is a new graph computing model allowing data propagation between billions of nodes. Both technologies are suitable for large scale applications running on thousands of machines.
Speaker: Jochen Hollmann, Software Engineer, Google
Bio: Jochen Hollmann has been a Google engineer since 2006. Before joining Google he has been an PhD student in the high performance computing research group at Chalmers.
It was really interesting bur far to short, only one hour. The funny thing is that they think and speak in quite different quantities then normal people do. For example if you are a google engineer and need to use 10.000 computers for 3 days to solve a problem? No problem, there are frameworks exactly for that purpose like MapReduce or Pregel and there are hundreds of thousands of computers for your disposition.
During the FAQ session the most common answer was "I'm sorry that is a secret." :D (edit: For questions like: "Do you use GPUs for computing this?")
It was cool to see a former Chalmers fellow student go give a talk at his old University, but the main purpose seemed to be headhunting. The last 15 minutes of his talk he was talking about their internships and new graduate opportunities and how different the hiring process is from other companies because other engineers are hiring you instead of some human resources staff.
Interesting fact: There are (only?) 1500 people working for Google across Europe.