Content Aggregation: How do we do it?

Its a bit of discovery for me that so many people who wanted to use our review script are also interested in some kind of a reviews/listings aggregation from other sites so that they can build a useful database of a specific niche.

What is Content aggregation anyway and what it means for Crowd Vox?
Crowd – Vox, that name is worth 6 hours of debate and conversation (silly! I know), we were trying to find out a name which would say it loudly that this product offers you to build a website where users (Crowd) can express themselves or say their views (Vox – latin for voice). So anyway, back to the topic, while it does that job – but that ain’t enough. Why? you may ask. In today’s busy world only a proportion of internet traffic got time to post reviews – that makes running a review site pretty difficult (I can see existing customers nodding now). So what? Wouldn’t it be great if you could use reviews or business listings or any other thing for that matter from existing websites and aggregate to your own? Think about that, what if you’re building a book review website of a specific niche, say, Business – and you are given all the user reviews from Amazon? Wouldn’t that be great? Ok, you get the idea I hope – but don’t get too excited because its still in our brains.

So now the question comes:

How do we do it?
I’ll cut this section short, my concerns:

1. Every user will want to aggregate content from different websites. So it is necessary to come up with pricing which fits all.

2. How to build something generic that fits need of every customer?

3. Legal Issues

Suggest anything you like. We believe this could be a huge thing for Crowd Vox as a product, and we’re putting our money where our mouth is. Yes we are, We’re offering a revenue share for those geniuses among you, who have their unique ideas to make this thing happen. You don’t have to code anything on your own, but just suggest your idea. If we found your idea to be truly innovative and feasible to execute, we’d love to offer you a revenue share from the sales of this module.

Update: CSV Import feature is added in release 1.0.11

  • Andrew Mills

    I have an idea about aggregation and it is to have the ability to Import CSV Files, so you can populate site with relevant products from an affiliate program you are with, and it will import all the products data and you affiliate link from the CSV file.

    I think this could be very powerful as most people starting or looking to start a review site understand the potential for affiliate earnings, and this could be a selling feature if it is done right.

    maybe have it so that categories etc can be imported and created from the CSV file so it would be one click creation and population of the site.

    there would also be no legal problems as affiliate prgrams freely offer the use of their content to help sell the products etc.

    I think this could be powerful and useful as there are many affiliate programs offering CSV feeds so almost everyone could benefit from it.

    I leave it to you to determine best how to implement it and integrate the CSV file into the site, so it displays right. I would make one suggestion and that is to make it so the affiliate link is prominent on the page and maybe make it so that the product image is clickable and when clicked takes the visitor to the affiliate link for that product.

    just some ideas:)

  • admin

    Andrew, Good point – there’s a CSV import module which will be ready pretty soon. However, what we’re suggesting here is fetching data from sites which do not allow such feeds.

  • William

    Hi. I will recommend using existing RSS feeds from review websites. Secondly, since Google is also showing reviews and ratings in its searches, we can find websites that have been configured to allow Google to index their reviews. Great right?
    And add some cron job function to retrieve these inf every few minutes. :) .

    • admin

      Secondly, since Google is also showing reviews and ratings in its searches, we can find websites that have been configured to allow Google to index their reviews.

      William, This is surely a nice idea. Here’s what we’re planning to do in order of their priority:

      1. CSV – Plain and simple. This will help webmasters who are affiliated with various merchant networks.

      2. RSS/XML Feeds – This would be configurable module where you can specify your feed URL and fields to look data from. Currently, we are discussing this and most probably this feature will be there in next to next release.

      3. Web Scrapping (your google idea) – This is somewhat complicated to design and fit in needs of every customer. So, this requires some special debate and discussion. or maybe we can keep this as a service. Any ideas will be greatly appreciated.

      4. Twitter (This is not validated yet) – One of our customer also suggested to parse twitter statuses and extract reviews per business/listing and classifying that review as ‘good’ or ‘bad’. This is pretty big to implement though. But what do you think?

      The first item is coming already in next release, but rest are still in discussion phase (just to be clear).

      Thanks for your feedback.

  • Mark

    I think this is a great idea, our site will only allow reviews that have been verified and authenticated whilst it is a time consuming process it means that our site does not contain fake or defamatory reviews. We are about to aggregate other sites that actually verify their reviews onto KwikChex.

    As a suggestion about Web Scraping and Google can you use the microformats and rich Snippets to aggregate the reviews from other sites and create a ‘Mashup’?

  • admin

    Yes, Mark, That is same as point 3 of my previous comment.