Cornerstone Development Meeting

  • Erik Ziegler Merge current dev branches, then the rest of these PRs go into dev

Outstanding Pull Requests

Cornerstone

IssueReasonPlan
imageCache :: memory leak (critical)Large DICOM files with sharedCacheKey will crash the browserMerge into dev branch
Fix #82: drawInvalidated() should pass trueRequired for requestAnimationFrame support
Display the images that have only VOI LUT Sequence properlyFixes bug with VOI LUT support
  • Former user (Deleted) will review Evren Ozkan's PR and if it is unacceptable, he will close it. He has investigated this issue and has another fix which may be more appropriate.


Making the canvas, context and its rendering data specific to each enabledElementFixes race condition with renderingMerge immediately
WIP: Prototype of multi-layer compositionDevelopment PRLeave open
Make cornerstone Promise/A+ compliantNot sure we intend to make Cornerstone Promise compliant
  • Everywhere else can return promises
  • Fix documentation to specify imageLoaders should return Deferred
  • Why do we want to use Deferreds?:
    • So we can cancel the image loading
  • Start moving towards the removal of the jQuery dependency
    • Future: allow the dev to specify what type of Promises / Promise library Cornerstone should use
    • Create an example of how to use Cornerstone without jQuery for the Promises
Rendering loopUses requestAnimationFrame.
Add possibility to display image staticallyUseful for PreviewsReview / Merge
Rendering using css transform instead of canvas transformDevelopment PRLeave open (or close?)
Added canvasToPixel function for easier use in cornerstoneToolsHelper function which will help us clean up Cornerstone ToolsMerge immediately
WIP: WebGL renderingAdds optional WebGL rendering path. This is stable and has been used in OHIF for months.
  • Erik Ziegler and Former user (Deleted) Look into non-linear LUT in WebGL branch. That may be the last issue remaining before it can be merged. Non-linear LUTs will be used in large images.
  • Erik Ziegler Force developer to explicitly handle WebGL failure via an event or something like that
  • Erik Ziegler Document the fact that WebGL branch is not the 'final solution' and will change and may be unstable. This is why it is opt-in.
  • Does WebGL cause too much overhead when switching images?
  • Look into changing Cornerstone's API to allow devs to plug in new rendering types to support pyramid type rendering or JF's rendering type

CornerstoneTools

IssueReasonPlan
Purge image cache issue (stackPrefetch)

stackPrefetch should not trigger exceptions for disabled elements removed from cache

Merge immediately
Setting a default value for resetPrefetchDelayRequired to fix issues with stackPrefetch for multi-frame instances during CINE playMerge immediately
Fix issues with rectangleRoi toolBrings rectangleRoi to parity with other toolsMerge immediately
Fix comments for the timeSeriesPlayer toolMinor comment issueMerge immediately
Cornerstone playClip tool now supports playing variable frame rate instancesNecessary for multi-frame supportMerge immediately
use buttons instead of which in the mouse event apiMinor change for Firefox supportMerge immediately
for bower installation dev branchVery minor change to fix bower installation of dev branchMerge immediately
WIP: Added simple paintbrush tool & exampleDevelopment PR, not to be mergedLeave open
Level of Detail Window Level and CenterAlternative to WWWC tool which downsamples during window/levelNot sure? Useful for many users?

dicomParser

IssueReasonPlan
New method to parse VR=AT fields + TestsRequired for multi-frame instance support since it enables Frame Increment Pointer (0028,0009)Merge immediately
Heap performancePerformance improvement
Don't include tag if not not neededUnclear

Cornerstone WADO Image Loader


IssueReasonPlan
Support for loading frames using WADO-RS (octet-stream)Required for WADO-RSMerge immediately
wadors (loadimage) must return a deferred like wadouri doesRequired for WADO-RSMerge immediately
Reject the promise if parsing the dicom file failsAdds error handlingMerge immediately
Add bitsStored into image object to be used for VOI LUT in cornerstoneUsed for VOI LUT. Not certain if we want this in image object or in metadataProvider?Review
Speeds up RLE decoding significantly by replacing DataView by Int8Array (~10x speedup in Chrome)Performance improvementMerge? Rollback if issues arise?
Add support for binary encoded DICOM Segmentation filesUsed for DICOM Seg supportMerge immediately
Fix CornerstoneImageLoadProgress for multi-frame imagesUseful for multi-frame imagesReview
[DicomFile loader] multiframe loading brokenUseful for local dicom file multi-frame supportNot sure this is still relevant. Review

Cornerstone Web Image Loader

IssueReasonPlan
gh-pages branch still exists
Delete branch, not necessary anymore


Open Issues

Cache Size Issue

If the cache size is set to 1GB and you load 1GB of data cornerstone will consume at least 2GB. That's because the dataset is cached by dataSetCacheManager and also a color/grayscale image will be created for each frame using a closure keeping the imageData (bytes) allocated in the memory (see getPixelData, getImageData and getCanvas methods). The problem is because imageCache doesn't count the images (frame) size but just the dataset size. Then if you load a 100MB dataset that contains 100 frames it will store 100MB (dataset) + 100 frames (1MB each approx). The size of the images created by wadouri image loader is set to the dataset size and we don’t know the real image size what makes it difficult to know how many MB there are in the cache.

Three ways to fix this issue:

  1. The dataset should have a different structure, storing only the header bytes (removing the pixel data) and the frames converted into a color/grayscale images instead of the raw pixel data. Then each image would re-use the same frame data instead of creating a copy of it. Cornerstone uses the dataset in too many places and maybe this will not be so easy to change.
  2. All caches (dataSetCacheManager, imageCache, etc) should add an entry to a Global Cache responsible for counting and removing items from these caches (like datasetCacheManager and imageCache do through decache method). This way we should be able to count dataset + image + anything else sizes. And also, to avoid removing the dataset before its images from the cache each cache entry could have a priority (property) then sorting by priority/timeStamp would solve this problem.
  3. The image should not create a copy of the bytes but creates the color/grayscale image on-the-fly whenever getCanvas is called. This process saves memory but results in a performance impact because these images would have to be re-created too many times (scroll / cine).
  • Erik Ziegler Put this on the OHIF roadmap
  • Figure out a way to make the Cache manager 100% accurate for how much memory Cornerstone is using.
    • Look into color images
    • Look into compressed images
    • Look into CINE usage and cache involvement

RequestPoolManager Improvements

  • Erik Ziegler Put this on the OHIF roadmap
    Allow parallel downloads when loading multi frame instance
    If a multi frame series starts loading using wadouri all frame promises will be added to the requestPoolManager but all threads will waiting for the same dataset. If you open Chrome DevTools and try to prefetch a different series (we developed a StudyPrefetcher on OHIF) you will see that only the first one will be in progress. The second series will be downloaded only when the first one is done and all its promises are resolved.

  1. Abort a download when its promise is rejected.
    When a 100MB series is being loaded and user clicks on another series he/she will have to wait for the first one to be loaded before starting loading the second one what can result in a delay on UI impacting the user experience.

General Issues

  • Viewers / Cornerstone should use requestAnimationFrame loop to display images
  • Multi-layer composition work is ongoing



Anonymous
February 3, 2017

..Me, jpambrun.