RR 439: Human Powered Rails: Automated Crowdsourcing In Your RoR App with Andrew Glass

00:00 44:14
Download MP3

Andrew Glass is a Brooklyn based Rubyist operating a small independent devshop called Bang Equals. He has held many ‘enrichment jobs’, including being a ball person at US Open for 5 years, traveling for judging Guinness World Record attempts, and will be a balloon holder in the Macy’s Thanksgiving Day Parade this year. Today the panel is discussing his about his 2018 RailsConf talk, Human Powered Rails: Automated Crowdsourcing In Your Ruby on Rails App. In his talk, he shows the audience how to use Amazon Mechanical Turk. Amazon Mechanical Turk lets you post tasks, set a price point, and then people can go and complete the task. This is often done with tasks that can’t be done with machine learning and to train machine learning algorithms. In his talk he goes into What it is, how it’s used, and how we can use Ruby to automate the process. In his apps, he uses it for lead generation, qualification, enrichment, and some video and photo tagging. More specific uses include recording items from a picture of a shopping list, identifying specific things in a video, categorizing businesses and items, sentiment analysis of text or image. Overall, Mechanical Turk is used for things that machine learning can’t handle yet. The panel discusses some different uses for crowdsourcing and how to submit something to Mechanical Turk. There are multiple ways to ensure accuracy in your surveys, including setting up multiple stages to your task, having more than one person complete your task, and creating a qualified worker pool based on tests to determine their aptitude and skill.

The panel discusses some of the controversy surrounding Mechanical Turk, citing an article in the New York Times (see links). The big issue is wages and worker rights. Wages can be very low, and it is ripe for abuse by companies as they could easily refuse all work and withhold pay. It is also important for the companies to give an accurate time estimate for the task and a reasonable reimbursement. Mechanical Turk attracts a variety of people, from people that do it for fun to people to actually do it for a living, so it is vital that companies use the tool responsibly.

Andrew talks more about how his app works. His apps are built on RTurk, Turkee, and Mechanical Turk, and he talks about how they work. The tricky part is figuring out the logic for what answers they will accept. Andrew talks about how to get started with Mechanical Turk and how to validate the work you get back. To ensure you get accurate information, he suggest that you make it happy for your users, make the UX simple and usable, and use a lot of formatting in your forms so that you get good information in. They preface their results with an accuracy score to help determine what is true. Andrew talks about where he wants to go from he. His Turking days are behind him, but his days of coordinating the efforts of many using software show promise.

Panelists

  • Dave Kimura
  • Charles Max Wood

Guest

  • Andrew Glass

Sponsors

____________________________________________________________

"The MaxCoders Guide to Finding Your Dream Developer Job" by Charles Max Wood is now available on Amazon. Get Your Copy Today!

____________________________________________________________

Links

Picks

Dave Kimura:

Charles Max Wood:

Andrew Glass:

FAQ

What is Amazon Mechanical Turk?

From the MTurk website; “Amazon Mechanical Turk (MTurk) is a crowdsourcing marketplace that makes it easier for individuals and businesses to outsource their processes and jobs to a distributed workforce who can perform these tasks virtually. This could include anything from conducting simple data validation and research to more subjective tasks like survey participation, content moderation, and more. MTurk enables companies to harness the collective intelligence, skills, and insights from a global workforce to streamline business processes, augment data collection and analysis, and accelerate machine learning development.”


What can Mechanical Turk be used for?

Mechanical Turk has a variety of applications, including lead generation, qualification, enrichment, and some video and photo tagging. More specific uses include recording items from a picture of a shopping list, identifying specific things in a video, categorizing businesses and items, sentiment analysis of text or image. Overall, Mechanical Turk is used for things that machine learning can’t handle yet.


What are some ways to ensure accuracy with Mechanical Turk?

There are multiple ways to ensure accuracy in your surveys, including setting up multiple stages to your task, having more than one person complete your task, and creating a qualified worker pool based on tests to determine their aptitude and skill.


What is the controversy surrounding Mechanical Turk?

The controversy stems mainly from wages and worker rights. There are many opportunities for companies who use the service to abuse workers. They could set low wages, withhold payment, and set inaccurate time estimates, among many other concerns.


Sign up for the Newsletter

Join our newsletter and get updates in your inbox. We won’t spam you and we respect your privacy.