GSoC 2108 Project [ Advance features in daru-view ] discussion

[shekharrajak]

@Prakriti, We can discuss about the progress and doubts on 19th may after 9pm.

[Prakriti]

I will be available. Thanks.

[Prakriti]

Summary of today’s (19th May) meeting:

1. Loading of dependent js files: We have a bunch of dependent js files to load for highcharts, highstock, highmaps and their modules. So, for now, we plan to load dependent js files for just highstock, highmaps and some of the common modules and might be able to provide some option for the others while creating the chart.
2. Data of HighMap in Daru::DataFrame/ Daru::Vector format: Some of the examples like Honeycomb Map and Distribution Map uses data which is difficult to incorporate Daru::DataFrame/ Daru::Vector format. So, for now, I am trying on how to get the data in the required format.
3. I will add the required rails examples in demo_daru-view repo.

[Prakriti]

Report for 19th May:
I worked on improving the coverage report of Highcharts.rb through this commit.

[shekharrajak]

Thanks @Prakriti, for writing Minutes of meeting .

I feel that you need to learn about SOLID design principles , software design patterns, better exception handling, extendable & modular and some other concepts to code like a pro.

Meanwhile keep writing examples in demo-daru_view (learn rails, sinatra, nanoc, hanami) repo.

Regards,
Shekhar

[Prakriti]

I was thinking of a way to load dependent js files of modules separately. We can provide another option modules for the user but that would create problems:

1. In web frameworks dependent js files must be loaded in head tag and fetching js files in the head tag would be difficult from that option.
2. In IRuby notebook, we can also set options as plot.chart.options = options, so we’ll be loading dependent js when show_in_iruby will be called (as we are not getting options in the plot object here) which might create conflicts in display of chart before loading appropriate js files.

So, I thought of including the modules (by the user as a separate parameter) when the plotting_library or dependent_script is set but that would create design problems as it is highcharts specific. Any other idea that we could try?

[shekharrajak]

head tag would be difficult from that option.

Please add more lines for clarification. How it would be difficult.

In IRuby notebook, we can also set options as plot.chart.options = options , so we’ll be loading dependent js when show_in_iruby

Hm, then it will load the same js code whenever show_in_iruby is called.

I hope you will come with some good idea before friday.

[Prakriti]

Report for 22nd May:
I spent the day working on custom styling CSS.
There are two modes of Highcharts: Classic and Styled.
Classic mode provides some custom CSS styling and does not require any additional dependencies while styled mode uses js dependency (like https://code.highcharts.com/stock/js/highstock.js) replacing the older one + the CSS dependency and provides full custom CSS styling.
I have implemented the Styled mode for web frameworks but as there is no method to load CSS files in IRuby notebook, so I have kept classic mode for IRuby notebook where some styling can be provided.
User needs to provide the required CSS styling in CSS option.
I have opened a PR regarding the same.
Difficulties: For loading CSS in IRuby notebook, custom.css could be a rescue but then everytime user needs to load the CSS files in custom.css. I am searching for some other way out (refer).

[Prakriti]

Report for 23rd May:

1. I added examples of the custom styling CSS in IRuby notebook in this PR.
2. I worked on separating the dependent js files. User can provide the required js files in modules option as a list of Strings. In IRuby notebook, these will be loaded when show_in_iruby is called (working fine now, had some problem earlier). I have implemented it in this PR. I have modified the examples of HighMaps and HighStock accordingly.
   In web frameworks, div method is called. So I tried loading the dependent js in to_html method which is working correctly. My concern is that the dependent files of the modules are not loaded in the head tag instead they are loaded into the body before the script of the chart, so will it be a problem?
3. Added HighMap example for data as Daru::DataFrame in this commit.

Also, have a look at the revised blog post for the first week.

[Prakriti]

Minutes of the meeting (26th May’ 2018) :

1. HighMaps: We need to add more examples of HighMaps with data in Daru::DataFrame or Daru::Vector format. All the examples of HighMaps can be found here.
2. Transfer of all the js and CSS dependencies to the assets folder.
3. The folder structure of the js dependencies should be maintained.
4. All the constants should be kept in a separate file.

[shekharrajak]

@Prakriti, write down your proposed ideas on GoogleCharts, your current research, what do you want to implement and what not.

We recommend you to explore more about googlecharts from official site and let us know your research.

[Prakriti]

Minutes of the meeting (3rd June’ 2018):

1. GoogleCharts: We would be implementing the following in GoogleCharts:
   a. ChartWrapper class that wraps around the chart and does all the loading and data source querying.
   b. ChartEditor class that provides runtime editing.
   c. Exporting of GoogleCharts to different formats.
   d. Multiple Charts in a single row (Needs to be done for all adapters).
   e. Importing data from google spreadsheets. (Have already opened a PR. Facing some problems earlier but now fixed).
   f. Adding Toolbar in GoogleCharts to export the data in CSV and HTML format(Optional).
   g. Adding listeners in GoogleCharts (Optional).
   h. Adding more methods of DataTables (Optional - will be doing only if we get some solid use case).

2. 3rd parameter: Will be adding 3rd optional parameter that will contain the hash of options created by us like modules, chart_class and CSS in HighCharts. This will be done in a separate PR.

3. Updating Daru-view wiki: Wiki needs to be updated with the highcharts features added, with examples and the future possibilities in highcharts.

[shekharrajak]

Hello @Prakriti,

Please let us know the update and fill this wiki page, mentioning what is completed till phase 1, TODO part (challenges you faced), and next plan. You will see the edit button for wiki page once it is public.

Regards,
Shekhar

[Prakriti]

Sure, I will update the wiki page.

[Prakriti]

Update on the work in daru-view:
For the past two weeks, I worked on the following tasks:

1. Exporting of HighCharts in different formats: Through this feature, user can export the HighChart in various formats (PDF, PNG, JPEG, and SVG) using the code. It is running fine in web frameworks. Progress can be tracked via this PR.
2. ChartWrapper: It facilitates the use of view option and paves the way to implement ChartEditor. I have opened a PR to implement this feature.
3. Import data from Google spreadsheet in google charts: I opened this PR earlier but was facing some issues which have been resolved now.
4. ChartEditor: Right now I am working on it which provides the facility to edit the Chart on the fly. Check out this PR for its implementation.

Challenges and difficulties:

1. Exporting of HighChart in IRuby notebook can be done either offline and online and both have their pros and cons. I am still confused which one should be implemented. Right now, I have kept the online exporting.
2. I am facing some problem in the display of ChartEditor dialog box in IRuby notebook. I tried a lot regarding the same but am still facing the issue.
3. In GoogleCharts, js script of the Chart is generated in two ways, i.e., with script tag and without script tag while we need only the latter one.

[Prakriti]

Summary of the meeting (12th June):

1. ChartEditor: There is a bug that Dialog box of chart editor is appearing behind the IRuby notebook. Still trying some way to figure this out.
2. Exporting of HighCharts: Keeping the online exporting in IRuby notebook and offline in web frameworks. Also need to try this in console.
3. Importing data from Google spreadsheet: Need to add more tests to check the validation of URL. Further, there might be some way to retrieve the data from the URL as opening the link provides the JSON data.
4. Rails examples: More examples needs to be added. There should be three types: simple, intermediate and advanced.

[Prakriti]

Update on daru-view:
This week I worked on Charteditor and displaying of multiple charts in googlecharts.

1. Displaying of multiple charts in a single cell in IRuby notebook and in a single row in web frameworks has been implemented and the PR is open for review along with the Chartwrapper PR. Check out the examples here.
2. ChartEditor: It is implemented in this PR (must be reviewed after Chartwrapper gets merged) and is working fine in web framewroks but in IRuby notebook it appears behind the elements of notebook like this:
   
   
   I tried a lot to figure out this problem but it still persists. The div of this dialog box is generated dynamically when we click on the edit button and it remains until we click on the ok or cancel button.
   What I tried:
   a. Increasing the z-index of dialog box in developer’s tools which worked well, so increasing the z-index of the dialog box is a solution.
   b. Increasing the z-index of the dialog in the chart_div using class name(as dialog box does not have id).
   c. Incresing the z-index of the dialog in the button click listener.
   d. Creating async function in listener and then increasing the z-index (as I was unable to get the dialog so I used sleep in it).
   e. Reducing the z-index of other charts.
   Nothing worked out. I was unable to get the dialog box element through the script as when the dialog box is generated it makes aria-hidden="true" for other divs.

[Prakriti]

Minutes of the meeting (23rd June):

1. Exporting of GoogleCharts: GoogleCharts provides the chart.getImageURI() method which in turn provides the access to a PNG image of a chart only. This has been implemented but for more types like pdf, jpeg and png, there are other solutions. For pdf specific, we can use jspdf which is working fine in rails app but again is producing an error in IRuby notebook as the jspdf dependency is not loaded properly in IRuby notebook. I tried it in this PR. (See example)
   Another solution is using cloudformatter (examples) but again this does not work in IRuby notebook. It provides exporting options to jpg, png and pdf. I tried it in this PR.
   Right now, I am keeping the png format only as it works for both.
2. Handling events (adding listener) in googlecharts: JS will be provided in the string format in the third parameter or as plot.add_listener(event, callback).
3. Multiple Charts in a row: Need to create a separate class as Plot class is for creating a single plot.
4. There is an issue that whenever we update the jsapi in googlecharts, it causes an error.

[Prakriti]

Update on daru-view:
For the last few days I’ve been working on the following tasks:

1. Exporting of GoogleCharts: I’ve implemented the export to PNG in googlecharts (for corecharts and geocharts). GoogleCharts official does not provide any method to export the chart and some other methods are not working properly with the IRuby notebook. I have opened this PR for the PNG implementation.
   Further, I’ve kept this PR for the future reference. It uses jspdf (for exporting to PDF) and cloudformatter (for exporting to pdf, png and jpeg).

2. Event Handling: With this feature, a user can apply multiple listeners to a google chart/table to handle various events. I have opened this PR for its implementation.
   Event handling can be done in Highcharts easily through the highcharts options like this, so in googlecharts also, we are handling events through options (in third parameter).

3. ToolBar: I was going through the js code of toolbar to incorporate it in daru-view. However, in the documentation, it is mentioned,

   To use a toolbar, your visualization must get its data from a URL; you cannot pass in hand-populated DataTable or DataView objects. You will pass the URL of the data used to populate your visualization into the drawToolbar() method.

   So, I think it would be of less use as our primary aim is to visualise the data from Daru::DataFrame and Daru::Vector.

[Prakriti]

I was digging more into the Google Charts website and learnt about the formatters in Google DataTables. Here is the document that describes how the API might look in daru-view. @shekharrajak can you please go through it once and also will this feature be good to have in daru-view?