GSoC 2108 Project [ Advance features in daru-view ] discussion

GSoC GSOC 2018
41

Summary of the meeting (12th June):

  1. ChartEditor: There is a bug that Dialog box of chart editor is appearing behind the IRuby notebook. Still trying some way to figure this out.
  2. Exporting of HighCharts: Keeping the online exporting in IRuby notebook and offline in web frameworks. Also need to try this in console.
  3. Importing data from Google spreadsheet: Need to add more tests to check the validation of URL. Further, there might be some way to retrieve the data from the URL as opening the link provides the JSON data.
  4. Rails examples: More examples needs to be added. There should be three types: simple, intermediate and advanced.
42

Update on daru-view:
This week I worked on Charteditor and displaying of multiple charts in googlecharts.

  1. Displaying of multiple charts in a single cell in IRuby notebook and in a single row in web frameworks has been implemented and the PR is open for review along with the Chartwrapper PR. Check out the examples here.
  2. ChartEditor: It is implemented in this PR (must be reviewed after Chartwrapper gets merged) and is working fine in web framewroks but in IRuby notebook it appears behind the elements of notebook like this:charteditor
    I tried a lot to figure out this problem but it still persists. The div of this dialog box is generated dynamically when we click on the edit button and it remains until we click on the ok or cancel button.
    What I tried:
    a. Increasing the z-index of dialog box in developer’s tools which worked well, so increasing the z-index of the dialog box is a solution.
    b. Increasing the z-index of the dialog in the chart_div using class name(as dialog box does not have id).
    c. Incresing the z-index of the dialog in the button click listener.
    d. Creating async function in listener and then increasing the z-index (as I was unable to get the dialog so I used sleep in it).
    e. Reducing the z-index of other charts.
    Nothing worked out. I was unable to get the dialog box element through the script as when the dialog box is generated it makes aria-hidden="true" for other divs.
43

Minutes of the meeting (23rd June):

  1. Exporting of GoogleCharts: GoogleCharts provides the chart.getImageURI() method which in turn provides the access to a PNG image of a chart only. This has been implemented but for more types like pdf, jpeg and png, there are other solutions. For pdf specific, we can use jspdf which is working fine in rails app but again is producing an error in IRuby notebook as the jspdf dependency is not loaded properly in IRuby notebook. I tried it in this PR. (See example)
    Another solution is using cloudformatter (examples) but again this does not work in IRuby notebook. It provides exporting options to jpg, png and pdf. I tried it in this PR.
    Right now, I am keeping the png format only as it works for both.
  2. Handling events (adding listener) in googlecharts: JS will be provided in the string format in the third parameter or as plot.add_listener(event, callback).
  3. Multiple Charts in a row: Need to create a separate class as Plot class is for creating a single plot.
  4. There is an issue that whenever we update the jsapi in googlecharts, it causes an error.
44

Update on daru-view:
For the last few days I’ve been working on the following tasks:

  1. Exporting of GoogleCharts: I’ve implemented the export to PNG in googlecharts (for corecharts and geocharts). GoogleCharts official does not provide any method to export the chart and some other methods are not working properly with the IRuby notebook. I have opened this PR for the PNG implementation.
    Further, I’ve kept this PR for the future reference. It uses jspdf (for exporting to PDF) and cloudformatter (for exporting to pdf, png and jpeg).

  2. Event Handling: With this feature, a user can apply multiple listeners to a google chart/table to handle various events. I have opened this PR for its implementation.
    Event handling can be done in Highcharts easily through the highcharts options like this, so in googlecharts also, we are handling events through options (in third parameter).

  3. ToolBar: I was going through the js code of toolbar to incorporate it in daru-view. However, in the documentation, it is mentioned,

    To use a toolbar, your visualization must get its data from a URL; you cannot pass in hand-populated DataTable or DataView objects. You will pass the URL of the data used to populate your visualization into the drawToolbar() method.

    So, I think it would be of less use as our primary aim is to visualise the data from Daru::DataFrame and Daru::Vector.

45

I was digging more into the Google Charts website and learnt about the formatters in Google DataTables. Here is the document that describes how the API might look in daru-view. @shekharrajak can you please go through it once and also will this feature be good to have in daru-view?

46

Minutes of the meeting (1st July):

  1. Multiple Charts: Plot class should inherit PlotList and each plot object in the list should work according to its adapter. Thus, each plot object should call its own div.
  2. Export GoogleChart/ HighChart in terminal: to export the chart in terminal using export method, we can export the HTML file to download the chart and then will open it in browser through code. This will generate the popup to download the chart. We need to find a way to distinguish whether we are running code in console or web frameworks (for rails we can use if defined? Rails)
  3. Formatters: There is already a class created for formatters in googlevisualr. We can use that class to add this feature in daru-view.
  4. Chartwrapper: Need to create a separate module and use it for both table and chart class.
47

Minutes of the meeting (10th July):

  1. Another method Daru::View.init_ruby should be created to load the libraries (right now, at object level init_iruby can be used).
  2. Need to add more examples of handling events in google charts.
  3. Add more examples and screenshots of examples in demo_daru-view.
  4. Styling should be done in rails examples.
  5. Name of daru-data_tables should be changed to daru-view_datatables.
  6. Proper documentation and testing needs to be done in daru-data_tables.
  7. Fill up the progress report for second phase.
48

Update on daru-view:
For the past few days, I have been working on the DataTables (PR #104 in daru-view and PR #20 in daru-data_tables) and have achieved the following in it:

  1. Use javascript Array rather than HTML table in DataTables: Right now, data is converted in the form of HTML table. I have shifted it to javascript array for the further server-side implementation (as datatable can take Array of Array).
  2. Resolved this issue in DataTables.
  3. Loading large set of data piece by piece: I have implemented server-side processing (using ajax option in DataTables) in DataTables for more than 50000 data rows and client-side processing below that.
  4. Updated the API of daru-data_tables: I have updated and refactored the API of daru-datatables.
  5. Added tests and documentation: I have added specs for the daru-data_tables gem and datatables.rb (in daru-view) along with the documentation.

Apart from these, I added some rails examples in demo_daru-view, committed on some old PRs and added load_libs_in_iruby method in this PR to load dependencies of multiple adapters in IRuby at once.

49

Minutes of the meeting (21st July):

  1. Add datatables examples in rails in demo_daru-view.
  2. Add a constants file in daru-data_tables also.
  3. Create another method dependent_scripts which will call load_libs_in_iruby (similar thing for dependent_script).
  4. Along with rails examples, add sinatra and nanoc examples too in demo_daru-view.
  5. Write a description of each PR in the comments along with the examples.
  6. Add rails examples in daru-data_tables.
50

Minutes of the meeting (31st July):

  1. Need to create separate files for separate modules (GoogleVisualr, Display, DisplayJavaScript).
  2. Add real-world examples in daru-view_io_rails repo.
  3. Remove a bunch of lines in the head tag. Understand the internal working of rails (railtie).
  4. Change Daru::DataTables::DataTable to Daru::View::DataTable in daru-data_tables.
51

Minutes of the meeting (4th Aug):

  1. The landing page of demo_daru-view must be improved.
  2. Include real-world examples in daru_examples_io_view.
  3. Implement 3rd parameter in HighCharts (chart_class, CSS, modules).
  4. While renaming the DataTables, init_script should be called using DataTables.
  5. Include rails examples where dependent JS are included from application.js file in demo_daru-view.
52

Update on daru-view:

  1. Formatters in GoogleCharts (PR #110):
    The Google Visualization API provides formatters that can be used to reformat data in a visualization.
    For more details, visit this page.

  2. Added a method to load dependent scripts for multiple adapters (PR #107):
    Have a look at these examples.

  3. Rake Task to add new adapter templates (PR #112):
    Added a rake task to generate the sample template for the adapter.

  4. Rename Daru::DataTables::DataTable to Daru::View::DataTable (PR #114 in daru-view, PR in daru-data_tables):
    As daru-data_tables is a plugin gem for daru-view, it needs to be initialized in a different way. Now, it can be instantiated as Daru::View::DataTable.new(data, options).

  5. Reduce a bunch of lines due to JS files in source HTML in rails (PR #115 in daru-view, PR #23 in daru-data_tables):
    This aims to remove a bunch of lines that occur in the head tag in rails. In rails, the one line of code we actually need to write in daru-view is for a class called Engine, this is going to inherit from Rails::Engine, and this is more or less saying that when rails loads this gem, it’s going to look for this Engine class, and if it’s defined, it’s going to treat this gem’s directory structure a little bit differently. It’s going to work sort of like your rails application, almost as if you had a rails app inside of a rails app.
    To carry this out, dependent JS and CSS files are shifted to vendor/assets/javascripts and vendor/assets/stylesheets respectively, so that, JS can be included in the app/assets/javascripts/application.js file as:

//= require highcharts/highcharts                                                           
//= require highcharts/highcharts-more
//= require highcharts/map
//= require jquery-latest.min
//= require jquery.dataTables

and CSS files can be included as:

 *= require jquery.dataTables

Include the below line in the head of the layout file:

<%= javascript_include_tag "application" %>
<%= stylesheet_link_tag "application" %>

Note: The previous method to use <%=raw Daru::View.dependent_script(:highcharts) %> will also work.

Difficulties:
Right now this has only been done for rails. Other web frameworks do not provide much flexibility and there are fewer resources available to search. I am trying to reduce lines in Sinatra and nanoc too but still facing many difficulties. Loading JS and CSS files in the app (by keeping these files in the app) can be done, but loading these files from daru-view in the sinatra/nanoc/other frameworks is not an easy task. I have also opened this issue in the sinatra repo.

  1. Improved Code Quality (PR #116, PR #76, PR #73, PR #74):
    Improved code quality of the project by fixing various rubocop comments in these PRs and also during working in other PRs.

  2. Added examples in web frameworks:
    Rails examples:

  • Importing data from google spreadsheet in google charts - #2
  • HighCharts, HighStock, HighMaps examples - #3
  • Custom CSS in highcharts example - #4
  • ChartWrapper examples - #6
  • Multiple Charts using PlotList examples - #8
  • DataTables examples - #10
  • ChartEditor examples - #9
  • Formatters examples - #11
  • Handling events examples - #12

Sinatra Examples:

  • Importing data from google spreadsheet in google charts - #5 commit
  • HighCharts, HighStock, HighMaps examples - #14
  • Custom CSS in highcharts examples - #13
  • ChartWrapper examples - #5 commit
  • Multiple Charts using PlotList examples - #5 commit
  • DataTables examples - #5 commit
  • Handling events examples - #5 commit
  • Formatters examples - #16
  • ChartEditor examples - #17