Tuesday, 21 June 2011

Sakai OAE Column Storage Driver Project

Hello, I am Aadish Kotwal, a now final-year-entering computer engineering student from Mumbai University. This is the first time I have entered into GSOC program and was (and am) very excited to associate myself with Sakai. Sakai Foundation is a great open-source organization. I had been through the past year’s projects and the blogs and thought working here would be a great learning experience. I have been working with many of the technologies Sakai uses for development and working here gives me a very big platform to take my interests and knowledge to a bigger level. I am associated with Sakai OAE, which in their own words is “A completely new system that incorporates all of the values of the Sakai CLE, and reimagines a new vision for academic collaboration.

My project is titled “Sakai OAE Column Storage Driver”.

Put in Sakai’s words, Sakai OAE user content uses a storage mechanism based on a sparse map concept(for more on Sparse Map Content concept refer link [1]) representing column database type storage with a memory window onto the storage. This abstraction has allowed to create traditional RDBMS representations of the storage system with a MySQL driver capable of doing sharded storage over with 1 write/many read DB clusters. The original column driver was based on Apache Cassandra which need some updating to keep it in sync with the latest developments, and Sakai would also like to create a driver for another column DB (eg HBase, Raik, Mongo, CouchDB, etc or some network protocol approach, eg protocol buffer, thrift).

This idea involves modifying the existing Cassandra driver(why Cassandra? See link [2]) Sakai uses to include all the features present in the JDBC driver and then to create a new driver for a database which will be decided during the term of the project.

I have been in regular discussions with Ian Boston(my mentor). Based on his guidance, I came up with an analysis and a flow of the project which is described below.

What we currently have:
Cassandra driver for the sparsemapcontent with incomplete implementation

What I aim to achieve by the end of GSoC term:
A complete Cassandra driver and a new driver for a NoSQL database implemented from scratch.

The project flow which has been framed after discussion with project mentor is as follows:

  • Finding all the methods and unit tests that require implementation and completing them.
  • Working on the analysis of new driver.
  • Start with coding API for the new driver with a structure similar to that of existing NoSQL driver. Also take feedback after completing this phase if any additional features specific to this database are expected.
  • After finishing with a working API, I would start with the soak tests and will test the results by implementing it on local machines.
  • Implement unit tests for the new driver to test functions such as content-addition, deletion, etc. on a local instance of database.
  • Write integration tests for the new driver to check if it works well with the existing implementation of sparsemapcontent.
  • Document the new driver and the implementation details at the confluence.

I have started working on technologies and codebase and have understood the basic structure. I plan to finish the entire learning phase prior to the coding stage and will also try my best to complete work on the existing database driver as early as possible so that I get more time for analysis and thoughts on implementation of new driver.

Progress till date:

1. My first task was to get serialization issues of Cassandra to be sorted, and this being my first task, Ian guided me wonderfully through it and made sure I stick to Sakai standards while focussing on efficiency. The task essentially was to write methods to convert an Object to ByteStream and vice versa. A quick link for the same for interested is: https://github.com/ieb/sparsemapcontent/blob/master/src/main/java/org/sakaiproject/nakamura/lite/types/Types.java

2. My second task was to implement indexing for efficient retrieval. This too under Ian’s guidance got implemented. This task included to find columns that were supposed to be indexed. The implementation links for the same are:

a) https://github.com/ieb/sparsemapcontent/blob/master/src/main/java/org/sakaiproject/nakamura/lite/storage/cassandra/CassandraClient.java

b) https://github.com/ieb/sparsemapcontent/blob/master/src/main/java/org/sakaiproject/nakamura/lite/storage/cassandra/CassandraClientPool.java

c) https://github.com/ieb/sparsemapcontent/blob/master/src/main/java/org/sakaiproject/nakamura/lite/ConfigurationImpl.java

3. My third task includes implementation of find on the Cassandra driver. The task is still under way.

My primary interest is to create a product of great significance which would be useful and accepted by the entire community.

A note on my mentor (Ian Boston):

GSOC on onset seemed very intimidating, but it was Ian’s support which really held me through. A thorough, detailed-oriented person, Ian has always been supportive and encouraging in writing efficient codes. The fact that such a busy man explaining the concept in so much detail really enthrals me. Any mistake and Ian not only rushes to correct me, but also shows alternative and better approach of implementation. Working under his guidance is going to be really fruitful this summer, and not to mention fun.

I will soon post of any further progress, and of course would love comments. My e-mail address is kotwal.aadish@gmail.com .

Looking forward for an awesome summer... :)


[1]: https://confluence.sakaiproject.org/display/KERNDOC/Sparse+Map+Content+-+Developer+Information

[2]: http://oreilly.com/catalog/0636920018537

Wednesday, 1 June 2011

Sakai OAE native mobile app

First of all, I would like to introduce myself, I am Mayte Giménez a student of Computer Science and Fine Arts from Spain. I am doing my specialization in languages and artificial intelligence. I have been working with Sakai for the last 3 years as scholar with David Roldán (recently named fellow, big kudos for him!), and Sakai gave me this big opportunity to develop a mobile application for Sakai OAE.

Before I will start talking about the project, I would like to say how impress I am about the big work the community have done with this branch of Sakai. The architecture is really strong and all technologies used are awesome.

So, let's talk about the GSoC project!
During this summer, I will develop, with the big help of Carl Hall, a mobile application for Sakai OAE. After research our different options, we are going to develop a native application for the main mobile platforms: Android and iOS. So I will be switching from one environment to the other every week.
Since Sakai OAE services returns JSON we can take advantage from native application, specially with the Android background services. (I still have to investigate with iOS, any help?)

I have created some mock ups about how the application should look like. These are my two proposals:

Mockup 1:

  • Login window
  • If login is correct we arrive to my dashboard, where we can select where to go. If the user selects "My membership" he will get a list of places where he can access and when he selects one site he will see the information about this site.
  • If the user selects "My profile" he will see his profile and he could edit it.

  • Mockup 2:

    • Login window
    • If login is correct we arrive to my dashboard, where the user will see some tabs with all the basics of sakai oae. First tab will be the profile, and he will be able to edit his profile. If the user selects "My membership" tab he will get a list of places where he can access and when he selects one site he will see the information about this site.
    • If the user is at "My profile", and he press a button (in Android: menu button, in iOs I should investigate which one will be better) he will see all the options of his profile and he could change the view.

    I would love to have some feed back about these. So please comment! :)

    Quite big for a short introduction, isn’t it? Well I won’t bore you much more.

    I have began with the configuration screen on the android environment, so I will post soon about it.

    Thanks for reading. Please, comment. And we see us soon.

Improve Sakai CLE WebDav Support

Hello,I am Manoj Inukolunu a 4th year student(graduated) from Bits-Pilani Goa, India.

My project is to rewrite the existing sakai CLE webdav code for better maintainability using Milton.

Milton is a great choice because its open source,it has great support(the mailing list and the author of milton are very helpful).The main problem with the existing code is maintainability due to its large size. All the code is written in a single class and its pretty old based on an older tomcat version.
with milton we dont have to worry about maintenance as it handles all the bookkeeping .We only have to implement the interfaces for the functionality.milton is agnostic about the nature of the data.
From my understanding I created 4 classes SakaiFolderResource which implements the FolderResource(com.bradmcevoy.http.FolderResource) SakaiFileResource which implements the FileResource(com.bradmcevoy.http.FileResource) and SakaResourceFactoryImpl which implements the ResourceFactory(com.bradmcevoy.http.ResourceFactory) and a helper class SakaiDavHelper for code re-usability.

Finally the best place for ideas is the confluence page and I can always be reached at manoj0011989@gmail.com for any query's