HOME / NEWS /

Sid G.9 Released

June 28, 2019

This is a limited first release that puts processes and infrastructure in place to support an expanded scope and an expanded, geographically dispersed team.

Work was concentrated in these areas:

 

  • Formal environments for Development, Testing, and Production
  • Reproducibility of instantiation of Development, Testing, and Production environments
  • Process for on-boarding additional team members
  • Formalized code management
  • Formalized release process
  • UI related issues
  • Addition and updating of supported applications

User facing Features of this release Include:

  • Unix desktop application with - Python( including Matplotlib, Numpy, Pandas) - R - Emacs (with Ista Zahn’s customizations) - LyX  - TexStudio 
  • Jupyter Notebook
  • RStudio
  • Application access to Harvard personal google drive mount
  • Support for Authentication and Authorization using HARVARDKEY and Grouper

 

 

Details of the Release:


Limitations:

 

  • Confidential data storage and processing are not supported. #128
  • Users are limited to 10 running jobs each, with up to 4 CPUs and 7GB of RAM per job. If the cluster is full, jobs will remain in a "Pending" state until resources are freed. #416 #180 #179 #68 #361
  • End user documentation is limited. (project:SidWebPresence)
  • Google Drive for g.harvard accounts is utilized for storage. The storage will be mounted at /mnt/google-drive. Note! Any data not saved to cloud storage will be deleted when the job ends. #141 #142 #311
  • There is no global mailing list for Sid users #182
  • Google Drive implicitly backs up data. Explicit backup of research data not available. #522 
  • For at least G.8 and G.9 Heroku will be running the web frontend on a single Dyno #526

Usability notes - Sid Job Launcher:

  • When launching a job, if an Application or Version has not yet been selected, then the "Connected storage" options are grayed out, and hovering the cursor over them changes it to a circle-with-slash-through, which indicates that the UI elements are disabled. However, clicking the grayed-out Google Drive option causes the Google authorization popup to open if Google Drive has not previously been authorized. #446
  • If timeout occurs between the the Dyno and Redis the Sid front end will have to be restarted. Running Jobs are not impacted. #531
  • Users may see: "This application has not been verified by Google" when attaching my Google storage." It's safe to move forward. #506

Usability notes - Sid Desktop Application:

  • From a Mac, on Sid Desktop, the Command key will function as Control and the Control key will function as Meta (Alt). #451
  • Logout or Quit from within an app will have undesirable results, such as rendering the job inaccessible. Don't do this. #458 Jupyter #422
  • When accessing the Sid Desktop over Chrome and Firefox, resizing the browser window does not resize the desktop window. #449
  • Up arrow not working in Sid Desktop Application from Mac. This has been observed in QTerminal and Emacs. #470
  • man command not found in Sid Desktop #471
  • In the desktop, Ctrl+C/Ctrl+V/Ctrl+A/Ctrl+E key combinations for copy, paste, first letter of line, last letter of line, are switched.  #451
  • If you remove the Sid App authorization from your Google Account Settings, you will not be able to re-authorize the Sid App for access to your Google Drive. This can be fixed but requires administrator intervention. #448

Usability notes - Jupyter Application:

  • Quitting Jupyter causes a job to return to the "Initializing" status. It should be deleted instead, or moved to a "Terminated" status #422

Usability notes - local development environment:

Note: minikube is software that allows a developer to the Sid Backend on their mac laptop.
**Issues targeted for G.9 that did not make the cut and are targeted for inclusion in the next minor release:**

  • Sometimes when launching a job in local dev after starting minikube, the job remains in Initializing state #528
  • I can only run one job with Google Drive storage connect #424
  • Need to reduce resource usage in dev env#371
  • Google storage prevents job from initializing on minikube >0.27.0 #337
  • Create development environment documentation for Windows #284
  • Create development environment instructions for Linux #283
  • Simplify the creation of a Sid sandbox #176
  • Investigate a minikube replacement #475

The following are non-documentation related issues that are completed but did not make it through review and QA and so are not included in this release.

  • Update terraform-provider-aws to 2.15.0 #476
  • The LXQT theme for Sid Desktop has minimize, maximize and close buttons which may not be obvious to the user. #452

Maintenance and Upgrades:

Three of the four components of the system have been targeted for continuous deployment.  Updates to these components can be done outside of a maintenance window and will not impact running jobs:

  • FrontEnd 
  • Sid Middleware 
  • Applications. 

Updates to the BackEnd cannot be implemented in a continuous deployment scenario and impact all running jobs.  The Sid team reserves a weekly maintenance window from Monday at 5PM through Tuesday at 9AM during which BackEnd maintenance will occur.

Not every window will have activity and when possible, users will be given 24 hour notice prior to a maintenance window being utilized.  However, users should expect their applications to be terminated without warning during the maintenance period.