For VendorsBlog
Amazon EMR for Yelp community
Phase: Finished

Amazon EMR for Yelp community

Vendor verified

Additional information

The project has been delivered on schedule
The budget has not been exceeded
Functionality complies with task

Implementation details

Проблемы Problem Solving
  • Low quality of customer service
  • Total high cost of ownership of IT infrastructure (TCO)
  • Shortage of inhouse IT resources
  • High costs of routine operations
  • Low employee productivity
  • Low quality of customer support
  • Risk of lost access to data and IT systems
Проблемы Business Objectives
  • Reduce Costs
  • Enhance Staff Productivity
  • Ensure Security and Business Continuity
  • Improve Customer Service

About Product

The Challenge

Yelp has established a loyal consumer following, due in large part to the fact that they are vigilant in protecting the user from shill or suspect content. Yelp uses an automated review filter to identify suspicious content and minimize exposure to the consumer. The site also features a wide range of other features that help people discover new businesses (lists, special offers, and events), and communicate with each other. Additionally, business owners and managers are able to set up free accounts to post special offers, upload photos, and message customers.

The company has also been focused on developing mobile apps and was recently voted into the iTunes Apps Hall of Fame. Yelp apps are also available for Android, Blackberry, Windows 7, Palm Pre and WAP.

Local search advertising makes up the majority of Yelp’s revenue stream. The search ads are colored light orange and clearly labeled “Sponsored Results.” Paying advertisers are not allowed to change or re-order their reviews.


Why Amazon Web Services

Yelp originally depended upon giant RAIDs to store their logs, along with a single local instance of Hadoop. When Yelp made the move to Amazon Elastic MapReduce (Amazon EMR), they replaced the RAIDs with Amazon Simple Storage Service (Amazon S3) and immediately transferred all Hadoop jobs to Amazon Elastic MapReduce.
“We were running out of hard drive space and capacity on our Hadoop cluster,” says Yelp search and data-mining engineer Dave Marin.

Yelp uses Amazon S3 to store daily logs and photos, generating around 1.2TB of logs per day. The company also uses Amazon EMR to power approximately 20 separate batch scripts, most of those processing the logs. Features powered by Amazon Elastic MapReduce include:

  • People Who Viewed this Also Viewed
  • Review highlights
  • Auto complete as you type on search
  • Search spelling suggestions
  • Top searches
  • Ads

Their jobs are written exclusively in Python, while Yelp uses their own open-source library, mrjob, to run their Hadoop streaming jobs on Amazon EMR, with boto to talk to Amazon S3. Yelp also uses s3cmd and the Ruby Elastic MapReduce utility for monitoring.

Yelp developers advise others working with AWS to use the boto API as well as mrjob to ensure full utilization of Amazon Elastic MapReduce job flows. Yelp runs approximately 250 Amazon Elastic MapReduce jobs per day, processing 30TB of data and is grateful for AWS Support that helped with their Hadoop application development.


The Benefits

Using Amazon Elastic MapReduce Yelp was able to save $55,000 in upfront hardware costs and get up and running in a matter of days not months. However, most important to Yelp is the opportunity cost. “With AWS, our developers can now do things they couldn’t before,” says Marin.“Our systems team can focus their energies on other challenges.”

Comments (0)

No comments found!