Machine learning · Natural language processing

How can I use ML algorithms, NLP and website data extraction technique to develop something concret?

SHARIQUE NISAR Consultant- Business Intelligence | Marketing Strategy | B2B Leads | Automation | Digital Innovation

February 9th, 2016

Pls share your experiences.

Scott McGregor Advisor, co-founder, consultant and part time executive to Tech Start-ups. Based in Silicon Valley.

February 9th, 2016

This is very broad, so I will just give one example in each of the 3 categories: 1) In 2001, my team took 22 years of “Columbia House Record and Video Club” offer and purchase data and built a Bayesian network for prediction. In doing so, we were able to predict which new recording people would most likely purchase based on past offers and orders. We also were able to figure out whether they would be more likely to respond positively to a “get two for the price of one" or "50% off” offer, or any other offer. We were able to predict which times of the year they were most likely to purchase (not only Christmas and Valentines day) 2) A few years ago,Adam Pease, father of SUMO, created an NLP program that reads reviews of TripAdvisor.com and does Sentiment analysis, as well as segment analysis, so you can tell if a hotel was liked by business travelers (quiet, good separate living room for meetings…) or by family vacation travelers (Pool in courtyard open all night! Live Music all day long!). (Note that all this information is in online reviews so it could have been captured by website extraction) 3) Indeed.com started initially by spidering company websites to find you information about jobs not on the main job boards. All these are concrete applications illustrative of things that can be used to create real business value.

Jan Mizgajski Lead Machine Learning Engineer (Contract) at Spoken Communications

February 9th, 2016

I'm not sure what are you asking for your question is very vague and probably better suited for Quora. Regardless, some things you would already find in the wild:

- semantic advertising (Adamantx)

- content discovery (Outbrain, Taboola)

- inferring social networks (kantwert, NetReveal)

- stock market predictions using social and online data (Bloomberg)

- dev tools for exploring unstructured data (AlchemyAPI)

Mona Sabet

February 9th, 2016

agree with what's already been posted. We used machine learning to understand the content in video. There are a thousand current applications for ML and NLP. I wouldn't start with the technology. I'd start with a problem that needs to be solved that you are fascinated by, and then see if your technology can be applied to it.

SHARIQUE NISAR Consultant- Business Intelligence | Marketing Strategy | B2B Leads | Automation | Digital Innovation

February 9th, 2016

Thank you guys for your response- much appreciated.

The idea which led me to ask this question revolves around---

Recently, we were working with a client who was in need of processing large amount of information to find insights. We built a system with the help of ML which led us to carry out text analysis with NLP. With this, we were also able to:

1. Process large text to extract key information, keywords, specific data etc. Example, we analysed an investor meet script to draft: a.) New announcement, b.) Facts and figures etc instantly.

2. We could track specific data set. Example, a product review or review from blogs, etc.

3. Perform News Analysis & Sentiment Analysis

5. Carry out text coding and analysis etc.

Now what- Since we also expert in extracting data from any website via automation and perform automatic and manual analysis of those data (concurrent review of data) using ML and NL, we thought of building something as our NEW offerings.

Although, a number of thoughts are already playing in our mind, I was wanting to take experts advice on this.

Any other inputs would be appreciated.

Thanks- :)

Viktor Pekar Research Scientist at University of Birmingham

February 9th, 2016

I suggest you speak with someone who has experience with NLP and ML, about the specific problem you are trying to solve. Some problems can be solved with high accuracy, high accuracy can be expected within very specific domains/verticals/industries, where a lot of annotated data is available (e.g., sentiment analysis of movie reviews). Some problems are just not feasible, e.g. trying to develop some complex NLP functionality for all possible domains and verticals. Also, some NLP tasks are easier than others: e.g. keyword extraction and named entity extraction are relatively easy, document summarization is very hard.

Question stats