Machine Learning Plugins for Data Science
|
This site is the new docs site currently being tested. For the actual docs in use please go to https://www.jenkins.io/doc. |
Goal: Create a new plugin for integrating Jenkins with one of Machine Learning tools (e.g. Jupyter Python, TensorBoard, or Sacred)
Status: Completed
Team
Student:
Loghi Perinpanayagam
Mentor(s):
Bruno P. Kinoshita
Ioannis Moutsatsos
Marky Jackson
Shivay Lamba
Details
The main goal of this project is integrating Machine Learning workflow including Data preprocessing, Model Training, Evaluation and Prediction with Jenkins build tasks. This plugin will be capable of executing code fragments via IPython kernel as currently supported by Jupyter Notebook. Version control will be handled as an added advantage of this project.
Deliverables
-
An IPython plugin with pipeline compatible
-
Improved config.jelly for the plugin
-
File parsers for ipynb, py and text(containing python code) files
-
Code segment extractor
-
Implement a code editor
-
Output console with static visual components
Community Bonding
The community bonding period is all about, well, community bonding. Rather than jumping straight into coding, I’ve got some time to learn about Jenkins’s processes - release and otherwise - developer interactions, codes of conduct, etc.
-
Blog post
Phase 1
-
Implement the configuration part of Ipython kernel interpreter
-
Implement file parser
-
Improve current config.jelly
-
Improve form validations
-
Increase the test coverage and documentation
-
Blog post
-
Change log
Phase 2
-
Implement code editor
-
Extend the editor for test purposes with pre-tested codelets
-
Testing and bug fixing
-
Improve the documentation
-
Discuss with mentors about phase 3 coding
-
Blog post
-
Change log
Phase 3
-
Design and implement code extractor
-
Multiple language kernel support
-
Implement Visual generator for the results
-
Integrate whole previous work and testing
-
Git integration for code segments (optional)
-
User documentation and examples
-
Blog post
-
Change log
Future Improvements
-
Improving performance of the plugin
-
Try to implement https://issues.jenkins.io/browse/JENKINS-63377
-
Support parameterized definitions in Notebooks https://issues.jenkins.io/browse/JENKINS-63478
-
Increasing testing code coverage
-
Implementing Interactive visualizer for data sets and results.
