Select Website 

Recruitment Directory's Blog - Australia's #1 Recruitment Technology Blog!

Back to Menu Back to Menu

Data Applications and Infrastructure at LinkedIn

Posted By: Thomas Shaw, 8:30am Friday 06 August 2010    Print Article

At a recent presentation by Jay Kreps of LinkedIn at the Hadoop Sumit 2010. Jay describes how LinkedIn crunches 120 billion relationships per day and blends large scale data computation with high volume, low latency site serving.

The Search, Network, and Analytics (SNA) team at LinkedIn works on LinkedIn's information retrieval systems, the social graph system, data driven features, and supporting data infrastructure. The system uses a number of open source software products such as
  • Apache Hadoop - framework for running applications on large clusters of commodity hardware.
  • Zoie - Faceted search
  • Bobo - Real-time search indexing
  • Decomposer - Very large matrix decomposition routines
  • Norbert - Partition aware cluster management & RPC
  • Voldermort - Key/Value storage
  • Kamikaze - Compression package
  • Sensei - Distributed realtime search
  • Azkaban - Hadoop workflow



Article URL: http://www.recruitmentdirectory.com.au/Blog/data-applications-and-infrastructure-at-linkedin-a389.html

Article Tags: linkedin linkedin infrastructure data management database hadoop zoie bobo decomposer norbert voldermort kamikaze sensei azkaban apache data warehousing open source software

Comments Hide Comments (0)

Feel free to join in on the conversation. All comments are moderated before publishing. Comments posted by subscribers don't necessarily reflect the views of Recruitment Directory.

Your Name: * Required
Your Email Address: * Required
Website URL:
Comments: * Required
Refresh
Enter the code you see in the image above (case sensitive). Click on the image to refresh it.
 


Back to Menu Back to Menu



Random Blog Articles

MyCareer.com for sale
Published: 3:43pm Saturday 06 June 2009

Green Collar Jobs
Published: 11:52pm Monday 16 February 2009

Review of Oracle Recruiter
Published: 12:02am Thursday 22 January 2009

Creating an Effective Social Media Strategy, Part 7 - Find Quality Followers to Engage, Don’t Worry About Numbers
Published: 3:07pm Sunday 16 August 2009

Interview with Tim de Jardine from Hirewall
Published: 4:45pm Tuesday 13 January 2009