We concentrate on making data processing as fast and easy as possible. To complete the dataflow, Integrate.io integrates with a plethora of services that can store, analyze, or visualize data. One of these services is Chart.io, a popular service for data visualization. You can use Integrate.io to process the data and then visualize the results in Chart.io. Here’s how.

Processing Data with Integrate.io

In the example below, we used Integrate.io to process public GitHub data and find the total commit sizes per day in June 2014.

Dataflow

thumbnail image

  1. github_source — loads the relevant GitHub data from our public S3 directory
    thumbnail image

  2. filter_commits — filters only PushEvents (commits)
    thumbnail image

  3. select_fields — selects the commit size field while typecasting it to an integer and changes the date to a string format via the ToString(created_at,'yyyy-MM-dd') function
    thumbnail image

  4. sumcommitsize — calculates the total commit size per day
    thumbnail image

  5. output_redshift — saves the results to Redshift so they can be loaded in Chart.io (here’s how to connect Integrate.io to Redshift). Integrate.io can also save the results as CSV files or to relational databases, which can all be integrated with Chart.io.

    thumbnail image

Visualizing Results with Chart.io

Once the above job is executed on a cluster, the results will be stored in Redshift. To load this data in Chart.io:

  1. Log in to your Chart.io account

  2. Go to “Settings” and “Data Sources” to connect a new data source and select “Amazon Redshift.”
    thumbnail image

  3. Enter all the relevant Redshift connection details and click “Connect.”
    thumbnail image

  4. Start visualizing your data.

Summary

Data can be easily processed with Integrate.io and then visualized in Chart.io. We showed an example of how GitHub data can be aggregated and visualized, but we’re really curious to see what you would be able to do with your data.