The following section provides an introductory overview of the WebSDK.
Overview / Design Patterns
The design pattern to
create a WebSDK server connection, register callbacks, and trigger
events is minimal and can be easily incorporated into any
JavaScript/Typescript framework. Moreover, the mechanism for
triggering and handling of events is similar for both the display of a 2D stack of DICOM
images or interacting with a MPR/3D volume. The below pseudocode describes the
minimal JavaScript code required to establish a server connection, load
a DICOM series, and display an image in the viewport. Each line is
described below.
The first line references the minimized WebSDK client API.
The second line is optional and references helpful utilities methods that
can be reused by your application. The third line references your
JavaScript app, and the fourth defines an HTML canvas element
with an ID of "fovia" that is referenced below in the JavaScript.
JavaScript
/// <reference path="../../foviaAPI.d.ts"/>
Fovia.FoviaAPI.requestServerConnection(async function (err) {
let scanDirResults = await Fovia.ServerContext.scanDICOMDir(filePath);
let sdc = getImageSeries(scanDirResults);
let foviaViewport = new Fovia.UI.HTMLViewport2D("fovia", windowWidth, widthHeight);
await foviaViewport.init(sdc,onDrawCallback, onImageMetaDataReceived);
foviaViewport.renderFinal();
}
let onImageMetaDataReceived = function (data) {
}
let onDrawCallback = function (foviaViewport, canvas, width, height, onDrawCallbackComplete) {
}
Make sure you include foviaAPI.d.ts to enable type checking and IntelliSense inside your editor.
Fovia.FoviaAPI.requestServerConnnection
establishes the initial socket.io connection to the server. The
first parameter defines a callback that is invoked following a
successful connection. Once connected, the application can make subsequent
calls into the API Next, Fovia.ServerContext.scanDICOMDir
reads all DICOM files in the supplied directory and subdirectories and
returns a JSON of all DICOM objects found, along with a subset of DICOM
tags, organized by patient, study, series (further organized by logical
subseries), and images. The application will read the JSON to
determine which series to load and how to hang / layout the
study. Loading from other data sources, including DICOM Web are
described below. The same JSON object structure is returned regardless
of the data source.
getImageSeries is a helper class in common/studyUtils.ts that by default returns the first viewable DICOM image series as a seriesDataContext (sdc). Fovia.UI.HTMLViewport is an abstract class that is
associated with an HTML5 Canvas element ID "fovia" referenced in the HTML. Concrete classes include Fovia.UI.HTMLViewport2D , Fovia.UI.HTMLViewport3D, Fovia.UI.HTMLViewportMPR,
including DoubleBuffer varients for each. Associated with each
HTMLViewport object is a renderEngine that performs the
rendering. Based on the class, its init method is invoked and is initilized with the seriesDataContext (or volumeDataContext
in the case of MPR/3D). It also take two important
callbacks: onMetaDataReceived and onDrawCallback described
below and provide the main hook into the event loop. The renderFinal
operation is called to trigger the initial render of the viewport.
When an image has been generated and is ready to be drawn onto the viewport
(canvas), the onMetaDataReceived callback is invoked with a data object
that includes context information associated with how the image was
rendered (render state, transformation, window/level, etc.) The
application can add its code to the onMetaDataReceived to trigger events
to other viewports or perform some action. For example, it may
need to update the localizer lines in other viewports or it may have to
update other synchroized, linked, or grouped viewports.
Once onMetaDataReceived returns, the onDrawCallback is
invoked with the image rendered onto the viewport canvas (or double
buffered canvas). At this point, the application can then draw
its elements onto the canvas, such as DICOM overlay, annotations, or UI
elements. Once it is finished, it MUST call the last parameter
onDrawCallbackComplete.
This informs the HTMLViewport object that the aqpplication drawing
is complete, and if necessary the HTMLViewport can handle the double
buffer operation.
This onMetaDataReceived and onDrawCallback methods are both called exactly one for each call to setRenderParams, render,renderFinal
, and buiilt-in UI adaptors that trigger an internal render. By way of example, this is illustrated in the simple 2D and
3D examples below.
.
Basic HelloFovia Application -- 2D
Click on http://localhost:8088/apps/hellofovia/2d.html (JavaScript) or
http://localhost:8088/apps/hellofovia/typescript2d.html (Typescript) to
run the simple 2D example. This loads and displays a stack of
DICOM image using the standard DICOM-compilant pipeline and provides
interactions through the default set of UI adaptors (mouse/touch).
Heavily documented source for each can be found at ./foviaserver/public/apps/hellofovia/2d.js (JavaScript) or
./foviaserver/public/apps/hellofovia/typescript2d.ts (TypeScript)
Basic HelloFovia Application -- 3D
Click on http://localhost:8088/apps/hellofovia/hellofovia.html (JavaScript) or
http://localhost:8088/apps/hellofovia/typescript.html (TypeScript) to
run the simple 3D example. This loads and a 3D volume for the saem
dataset and provides interactions through the default set of UI
adaptors(mouse/touch). Heavily documented source for this example
can be found at ./foviaserver/public/apps/hellofovia/hellofovia.js (JavaScript) or
./foviaserver/public/apps/hellofovia/typescript.ts (TypeScript)
To add 3D support, the following lines marked in green have been modified:
let scanDirResults = await Fovia.ServerContext.scanDICOMDir(filePath); let sdc = get3DAbleImageSeries(scanDirResults); let vdc = await Fovia.ServerContext.loadVolumeFromSeries(sdc); let foviaViewport = new Fovia.UI.HTMLViewport3D("fovia", windowWidth, widthHeight); await foviaViewport.init(vdc,Fovia.RenderType.parallel,onDrawCallback, onImageMetaDataReceived);
foviaViewport.renderFinal();
get3DAbleSeries is a helper class in common/studyUtils.ts that returns the first DICOM series tht is "3D-able". This seriesDataContext is then used by Fovia.ServerContext.buildVolumeFromSeries to generate the volumeDataContext (vdc). A Fovia.UI.HTMLViewport3D is created and the init
method is invoked with the vdc. Nearly identical design pattern
as the 2D example, but just uses a different renderEngineContext and
different set of UI adatpors.
Client / Hybrid / Server Rendering
Fovia.UI.HTMLViewport2D,
defaults to Fovia.RenderLocation.Server which uses the server DICOM
pipeline for rendering images. To render using the client-side
DICOM pipeline, you should use specify
Fovia.RenderLocation.Hybrid. By using hybrid mode, rendering will occur on
the server and automatically switch to client rendering once images are
downloaded.
One can force pure client-side rendering by
specifying Fovia.RenderLocation.Client, but keep in mind, no images
will be rendered until the DICOM images are downloaded to the
client. Presumably, this is only used if your
application knows image data has already been
downloaded. To modify the renderLocation, modify the above
2d.js app as follows:
// default render location in 7.0 is Fovia.RenderLocation.Server
// this can be changed to Fovia.RenderLocation.Hybrid or Client
let renderLocation = Fovia.RenderLocation.Hybrid
// for hybrid/client rendering, trigger the download to the browser cache so images
// will be automatically rendered on the client once image data is found in the cache
if (renderLocation == Fovia.RenderLocation.Client || renderLocation == Fovia.RenderLocation.Hybrid) {
sdc.downloadData();
}
let repaintable = false;
let useLosslessPNG = false;
// pass in renderLocation, and the two other flags to the constructor
let foviaViewport = new Fovia.UI.HTMLViewport2D("fovia", width, height, repaintable, useLosslessPNG, renderLocation);
The 2d app can be executed in all three modes by passing in either server (default), hybrid, or client via URL parameter: http://localhost:8088/apps/hellofovia/2d.html?renderLocation=hybrid
Heavily documented source for this example can be found at ./foviaserver/public/apps/hellofovia/2d.js
UI Event Handling
HTMLViewport defines default
behavior for all the common mouse operations (left-click, right-click,
middle-click, wheel scroll and wheel drag operations) and can be
associated with keyboard modifiers (shift, ctrl, alt, and meta keys).
The application can change this mapping by invoking setMouseAdaptor with Fovia.UI.MouseAdaptors, Fovia.UI.MouseButton and Fovia.UI.KeyboardModifier. For instance, to assign the 2D scroll adaptor to the left-click meta key, do the following:
foviaViewport.setMouseAdaptor(Fovia.UI.MouseAdaptors.scroll,
Fovia.UI.MouseButton.left, Fovia.UI.KeyboardModifier.meta);
• Window/Level
• Zoom
• Pan
• Page (navigate stack of 2D images one image at a time)
• Scroll (quickly navigate full stack of 2D images based on drag from top/bottom of viewport)
let voxelAdaptor = new VoxelValueAdaptor(foviaViewport, vdc);
foviaViewport.setCustomMouseAdaptor(voxelAdaptor, Fovia.UI.MouseButton.left, Fovia.UI.KeyboardModifier.ctrl);
// example class for creating a custom mouse adaptor that prints out the voxel value on the screen and console
function VoxelValueAdaptor(foviaViewport, vdc) {
this.foviaViewport = foviaViewport;
this.vdc = vdc;
}
VoxelValueAdaptor.prototype.down = function (event) {
}
// clear voxel print variable and force a repaint() of the viewport
VoxelValueAdaptor.prototype.up = function (event) {
voxelValue = "";
foviaViewport.repaint();
}
// during a move operation, obtain the voxel value (via shootRay) and force a repaint() of the viewport
VoxelValueAdaptor.prototype.move = async function (event) {
var rayStopInfoList = await
this.foviaViewport.getRenderEngine().shootRay([new
Fovia.Util.Point(event.clientX, event.clientY)]);
var rayStopInfo = new Fovia.RayStopInfo(rayStopInfoList[0]);
var displayUnits = rayStopInfo.voxelValue * this.vdc.rescaleSlope + this.vdc.rescaleIntercept;
voxelValue = "(" +
rayStopInfo.volCoodinate.x.toFixed(0) + ", " +
rayStopInfo.volCoodinate.y.toFixed(0) + ", " +
rayStopInfo.volCoodinate.z.toFixed(0) + ") = " + displayUnits + " HU";
console.log("voxelValue", voxelValue );
foviaViewport.repaint();
}
The mapping between the mouseAdaptorID or the
Fovia.UI.MouseAdaptorInterface object and mouseButton /
keyboardModifier can be reassigned at any time by invoking
setMouseAdaptor or setCustomMouseAdaptor again.
If the mouse should trigger an event for a particular area of the screen such as for:
• hot spot detection to change cursor when user moves over particular region
• clicking on cross-reference lines (adjust MPR oblique view)
• clicking on slab control lines (adjust MPR slice thickness)
For example, to create a ResetHotspotAdaptor, do the following:
// example class on how to implement a hot spot through the mouse override adaptor
function ResetHotspotAdaptor(renderEngine) {
this.renderEngine = renderEngine;
}
ResetHotspotAdaptor.prototype.down = async function (event) {
// capture mouse event in the upper corner of the viewport, and when detected
// call resizeWindow to force a reset of the window (by passing in true as last parameter)
if (event.clientX < 240 && event.clientY < 40) {
resetWindow(window.innerWidth, window.innerHeight);
// consume event
return false;
}
// propagate event
return true;
}
// only those methods needed must be impelemtned (up, move, scroll can all be ignored)
// // ResetHotspotAdaptor.prototype.up = function (event) { // // propagate event // return true; // }
Example
Click on http://localhost:8088/apps/hellofovia/mouseadaptors2d.html to
run a simple 2D application that demonstrates the above mentioned
concepts. Heavily documented source for this can be found at /foviaserver/public/apps/hellofovia/mouseadaptors2d.js
Click on http://localhost:8088/apps/hellofovia/mouseadaptors.html
to run a simple 3D application that demonstrates the above mentioned
concepts Heavily documented source for this can be found at /foviaserver/public/apps/hellofovia/mouseadaptors.js
Quick Start Guide for the WebSDK
The following section provides an introductory overview of the WebSDK.Overview / Design Patterns
The design pattern to create a WebSDK server connection, register callbacks, and trigger events is minimal and can be easily incorporated into any JavaScript/Typescript framework. Moreover, the mechanism for triggering and handling of events is similar for both the display of a 2D stack of DICOM images or interacting with a MPR/3D volume. The below pseudocode describes the minimal JavaScript code required to establish a server connection, load a DICOM series, and display an image in the viewport. Each line is described below.HTML
<html><script src="/foviaAPI.js"></script>
<script src="/apps/common/studyUtils.js"></script>
<script src="/apps/YOUR_APP/YOUR_APP.js"></script>
<canvas id="fovia"></canvas>
</html>
The first line references the minimized WebSDK client API. The second line is optional and references helpful utilities methods that can be reused by your application. The third line references your JavaScript app, and the fourth defines an HTML canvas element with an ID of "fovia" that is referenced below in the JavaScript.
JavaScript
/// <reference path="../../foviaAPI.d.ts"/>Fovia.FoviaAPI.requestServerConnection(async function (err) {
let scanDirResults = await Fovia.ServerContext.scanDICOMDir(filePath);
let sdc = getImageSeries(scanDirResults);
let foviaViewport = new Fovia.UI.HTMLViewport2D("fovia", windowWidth, widthHeight);
await foviaViewport.init(sdc,onDrawCallback, onImageMetaDataReceived);
foviaViewport.renderFinal();
}
let onImageMetaDataReceived = function (data) {
}
let onDrawCallback = function (foviaViewport, canvas, width, height, onDrawCallbackComplete) {
}
Make sure you include foviaAPI.d.ts to enable type checking and IntelliSense inside your editor. Fovia.FoviaAPI.requestServerConnnection establishes the initial socket.io connection to the server. The first parameter defines a callback that is invoked following a successful connection. Once connected, the application can make subsequent calls into the API Next, Fovia.ServerContext.scanDICOMDir reads all DICOM files in the supplied directory and subdirectories and returns a JSON of all DICOM objects found, along with a subset of DICOM tags, organized by patient, study, series (further organized by logical subseries), and images. The application will read the JSON to determine which series to load and how to hang / layout the study. Loading from other data sources, including DICOM Web are described below. The same JSON object structure is returned regardless of the data source.
getImageSeries is a helper class in common/studyUtils.ts that by default returns the first viewable DICOM image series as a seriesDataContext (sdc). Fovia.UI.HTMLViewport is an abstract class that is associated with an HTML5 Canvas element ID "fovia" referenced in the HTML. Concrete classes include Fovia.UI.HTMLViewport2D , Fovia.UI.HTMLViewport3D, Fovia.UI.HTMLViewportMPR, including DoubleBuffer varients for each. Associated with each HTMLViewport object is a renderEngine that performs the rendering. Based on the class, its init method is invoked and is initilized with the seriesDataContext (or volumeDataContext in the case of MPR/3D). It also take two important callbacks: onMetaDataReceived and onDrawCallback described below and provide the main hook into the event loop. The renderFinal operation is called to trigger the initial render of the viewport.
When an image has been generated and is ready to be drawn onto the viewport (canvas), the onMetaDataReceived callback is invoked with a data object that includes context information associated with how the image was rendered (render state, transformation, window/level, etc.) The application can add its code to the onMetaDataReceived to trigger events to other viewports or perform some action. For example, it may need to update the localizer lines in other viewports or it may have to update other synchroized, linked, or grouped viewports. Once onMetaDataReceived returns, the onDrawCallback is invoked with the image rendered onto the viewport canvas (or double buffered canvas). At this point, the application can then draw its elements onto the canvas, such as DICOM overlay, annotations, or UI elements. Once it is finished, it MUST call the last parameter onDrawCallbackComplete. This informs the HTMLViewport object that the aqpplication drawing is complete, and if necessary the HTMLViewport can handle the double buffer operation.
This onMetaDataReceived and onDrawCallback methods are both called exactly one for each call to setRenderParams, render, renderFinal , and buiilt-in UI adaptors that trigger an internal render. By way of example, this is illustrated in the simple 2D and 3D examples below.
.
Basic HelloFovia Application -- 2D
Click on http://localhost:8088/apps/hellofovia/2d.html (JavaScript) or http://localhost:8088/apps/hellofovia/typescript2d.html (Typescript) to run the simple 2D example. This loads and displays a stack of DICOM image using the standard DICOM-compilant pipeline and provides interactions through the default set of UI adaptors (mouse/touch). Heavily documented source for each can be found at ./foviaserver/public/apps/hellofovia/2d.js (JavaScript) or ./foviaserver/public/apps/hellofovia/typescript2d.ts (TypeScript)
Basic HelloFovia Application -- 3D
Click on http://localhost:8088/apps/hellofovia/hellofovia.html (JavaScript) or http://localhost:8088/apps/hellofovia/typescript.html (TypeScript) to run the simple 3D example. This loads and a 3D volume for the saem dataset and provides interactions through the default set of UI adaptors(mouse/touch). Heavily documented source for this example can be found at ./foviaserver/public/apps/hellofovia/hellofovia.js (JavaScript) or ./foviaserver/public/apps/hellofovia/typescript.ts (TypeScript)
To add 3D support, the following lines marked in green have been modified:
let scanDirResults = await Fovia.ServerContext.scanDICOMDir(filePath);
let sdc = get3DAbleImageSeries(scanDirResults);
let vdc = await Fovia.ServerContext.loadVolumeFromSeries(sdc);
let foviaViewport = new Fovia.UI.HTMLViewport3D("fovia", windowWidth, widthHeight);
await foviaViewport.init(vdc,Fovia.RenderType.parallel,onDrawCallback, onImageMetaDataReceived);
foviaViewport.renderFinal();
get3DAbleSeries is a helper class in common/studyUtils.ts that returns the first DICOM series tht is "3D-able". This seriesDataContext is then used by Fovia.ServerContext.buildVolumeFromSeries to generate the volumeDataContext (vdc). A Fovia.UI.HTMLViewport3D is created and the init method is invoked with the vdc. Nearly identical design pattern as the 2D example, but just uses a different renderEngineContext and different set of UI adatpors.
Client / Hybrid / Server Rendering
Fovia.UI.HTMLViewport2D, defaults to Fovia.RenderLocation.Server which uses the server DICOM pipeline for rendering images. To render using the client-side DICOM pipeline, you should use specify Fovia.RenderLocation.Hybrid. By using hybrid mode, rendering will occur on the server and automatically switch to client rendering once images are downloaded.One can force pure client-side rendering by specifying Fovia.RenderLocation.Client, but keep in mind, no images will be rendered until the DICOM images are downloaded to the client. Presumably, this is only used if your application knows image data has already been downloaded. To modify the renderLocation, modify the above 2d.js app as follows:
// default render location in 7.0 is Fovia.RenderLocation.Server
// this can be changed to Fovia.RenderLocation.Hybrid or Client
let renderLocation = Fovia.RenderLocation.Hybrid
// for hybrid/client rendering, trigger the download to the browser cache so images
// will be automatically rendered on the client once image data is found in the cache
if (renderLocation == Fovia.RenderLocation.Client || renderLocation == Fovia.RenderLocation.Hybrid) {
sdc.downloadData();
}
let repaintable = false;
let useLosslessPNG = false;
// pass in renderLocation, and the two other flags to the constructor
let foviaViewport = new Fovia.UI.HTMLViewport2D("fovia", width, height, repaintable, useLosslessPNG, renderLocation);
The 2d app can be executed in all three modes by passing in either server (default), hybrid, or client via URL parameter: http://localhost:8088/apps/hellofovia/2d.html?renderLocation=hybrid
Heavily documented source for this example can be found at ./foviaserver/public/apps/hellofovia/2d.js
UI Event Handling
HTMLViewport defines default behavior for all the common mouse operations (left-click, right-click, middle-click, wheel scroll and wheel drag operations) and can be associated with keyboard modifiers (shift, ctrl, alt, and meta keys).The application can change this mapping by invoking setMouseAdaptor with Fovia.UI.MouseAdaptors, Fovia.UI.MouseButton and Fovia.UI.KeyboardModifier. For instance, to assign the 2D scroll adaptor to the left-click meta key, do the following:
foviaViewport.setMouseAdaptor(Fovia.UI.MouseAdaptors.scroll, Fovia.UI.MouseButton.left, Fovia.UI.KeyboardModifier.meta);
For HTMLViewport2D.setMouseAdaptor, mouseAdaptor enums include
• Window/Level
• Zoom
• Pan
• Page (navigate stack of 2D images one image at a time)
• Scroll (quickly navigate full stack of 2D images based on drag from top/bottom of viewport)
For HTMLViewport3D.setMouseAdaptor, mouseAdaptor enums include
• Window/Level
• Zoom
• Pan
• Rotate (Parallel rendering mode)
• Flythrough (Perspective rendering mode)
• Scroll (MPR/MIP/MinIP rendering modes)
The application can define its own user input behavior by creating an object that implements the Fovia.UI.MouseAdaptorInterface and assigning it via HTMLViewport2D.setCustomMouseAdaptor / HTMLViewport3D.setCustomMouseAdaptor with a mouseButton / keyboardModifier. For example, to create a custom 3D voxelValueAdaptor, do the following:
let voxelAdaptor = new VoxelValueAdaptor(foviaViewport, vdc);
foviaViewport.setCustomMouseAdaptor(voxelAdaptor, Fovia.UI.MouseButton.left, Fovia.UI.KeyboardModifier.ctrl);
// example class for creating a custom mouse adaptor that prints out the voxel value on the screen and console
function VoxelValueAdaptor(foviaViewport, vdc) {
this.foviaViewport = foviaViewport;
this.vdc = vdc;
}
VoxelValueAdaptor.prototype.down = function (event) {
}
// clear voxel print variable and force a repaint() of the viewport
VoxelValueAdaptor.prototype.up = function (event) {
voxelValue = "";
foviaViewport.repaint();
}
// during a move operation, obtain the voxel value (via shootRay) and force a repaint() of the viewport
VoxelValueAdaptor.prototype.move = async function (event) {
var rayStopInfoList = await this.foviaViewport.getRenderEngine().shootRay([new Fovia.Util.Point(event.clientX, event.clientY)]);
var rayStopInfo = new Fovia.RayStopInfo(rayStopInfoList[0]);
var displayUnits = rayStopInfo.voxelValue * this.vdc.rescaleSlope + this.vdc.rescaleIntercept;
voxelValue = "(" + rayStopInfo.volCoodinate.x.toFixed(0) + ", " + rayStopInfo.volCoodinate.y.toFixed(0) + ", " + rayStopInfo.volCoodinate.z.toFixed(0) + ") = " + displayUnits + " HU";
console.log("voxelValue", voxelValue );
foviaViewport.repaint();
}
The mapping between the mouseAdaptorID or the Fovia.UI.MouseAdaptorInterface object and mouseButton / keyboardModifier can be reassigned at any time by invoking setMouseAdaptor or setCustomMouseAdaptor again.
Override Mouse Adaptor (UI glass pane)
While any given mouseButton / keyboardModifier combination can only have one assigned mouse operation, it is possible to allow multiple events with the same combination by creating an object that implements the same Fovia.UI.MouseAdaptorInterface and assigned it to the HTMLViewport2D.setOverrideMouseAdaptor() / HTMLViewport3D.setOverrideMouseAdaptor().
If the mouse should trigger an event for a particular area of the screen such as for:
• hot spot detection to change cursor when user moves over particular region
• clicking on cross-reference lines (adjust MPR oblique view)
• clicking on slab control lines (adjust MPR slice thickness)
For example, to create a ResetHotspotAdaptor, do the following:
// example class on how to implement a hot spot through the mouse override adaptor
function ResetHotspotAdaptor(renderEngine) {
this.renderEngine = renderEngine;
}
ResetHotspotAdaptor.prototype.down = async function (event) {
// capture mouse event in the upper corner of the viewport, and when detected
// call resizeWindow to force a reset of the window (by passing in true as last parameter)
if (event.clientX < 240 && event.clientY < 40) {
resetWindow(window.innerWidth, window.innerHeight);
// consume event
return false;
}
// propagate event
return true;
}
// only those methods needed must be impelemtned (up, move, scroll can all be ignored)
//
// ResetHotspotAdaptor.prototype.up = function (event) {
// // propagate event
// return true;
// }
Example
Click on http://localhost:8088/apps/hellofovia/mouseadaptors2d.html to run a simple 2D application that demonstrates the above mentioned concepts. Heavily documented source for this can be found at /foviaserver/public/apps/hellofovia/mouseadaptors2d.js
Click on http://localhost:8088/apps/hellofovia/mouseadaptors.html to run a simple 3D application that demonstrates the above mentioned concepts Heavily documented source for this can be found at /foviaserver/public/apps/hellofovia/mouseadaptors.js
Touch Support
Similar design patterns are available for touch UI support by setting the desired Fovia.UI.TouchAdaptors to HTMLViewport2D.setTouchAdaptor / HTMLViewport3D.setTouchAdaptor and creating custom HTMLViewport2D.setCustomTouchAdaptor / HTMLViewport3D.setCustomTouchAdaptor by implementing the Fovia.UI.TouchAdaptorInterface.