top of page
Search
  • Writer's pictureGrant Carroll

Creating and distributing Esri custom widgets (Part 1)

Updated: Jul 19, 2022

Esri custom widget development with Azure DevOps

This is the first of four posts where I will describe an end to end workflow for developing a custom Esri JavaScript widget and distributing it across multiple locations. Specifically showing how to:

  1. Build a single stand alone custom Esri JavaScript API widget.

  2. Use the widget within an Experience Builder application.

  3. Use the widget within a stand alone Esri JavaScript Application.

  4. Publish the widget to ArcGIS Enterprise 11.0 (To be done once 11.0 is released).

An example of the widget can be viewed below.


Inspiration for this came from the blog posts by Gavin Rehkemper on the Esri GeoNet site:

The details of how to publish to npm, were modified from the details in this blog post

from Georgi Parlakov here.

Overview


In this first post we will:

  1. Create a repository for our widget

  2. Add tools to support building and testing our widget

  3. Automatically build and publish the widget to npm

The widget we are building is a live weather widget, which will take the current location on the map, and return the current weather at that location, additionally, if we use it with a SceneView, we can use the latest tools from the Esri JavaScript API, to create a realistic view of the location.



In order to use the weather API, you will need an API key, you can create a free account at WeatherAPI.com to get one.


You will also need an npm account as well, these are free, however if you also want to create an organization and create private packages, then this will cost $7 USD per user a month.

 

1: Create your repository


If you do not have an Azure DevOps organisation, you can sign up and create one for free.

Within Azure DevOps, create a new project then navigate to the Repos->Files page.





Your project may already have a repository in it, from the drop box, select New repository



Enter a name for your repository and accept the rest of the defaults.



Now that we have the repository created, we need to clone it on to our local machine so we can start developing.

In the top right of the screen click the ellipsis and select Clone.


Next select the IDE that you want to use, in this example we will use VS Code. (Ensure you have VS Code installed already, link to download is here)


Allow the extension to open the URI.


Enter a folder location to store the repository, its a good idea to have a single folder for all your development projects.



VS Code will now clone the repository and ask you if you want to open it, select Open.



Within the root of the repository, build out a folder structure as below.

widget
 --src
    --model
    --view-model

The widget folder will hold all of the files for our custom widget. The src folder contains the .tsx and ts files for our widget. The model folder will hold any files relating to any data related classes that we will create. The view-model folder will hold any view model classes that we create.

 

2: Get setup with npm


The next step in creating the custom widget is to install some of the libraries we will need to support our widget. We will use NPM to scaffold out the details of our widget and allow us to install these libraries.

NPM will create a package.json file that will store the details of our widget, which extra libraries we will need to install to support it and it can also be used to build and publish our application.

Open a command prompt or use the terminal window in VS Code (if your are using it) and navigate to the root of your widget, if you have followed the folder structure suggested in the first section, this will be the /widget directory.

  1. Run the command

npm init

The first question will be the package name, give your widget a name, use lower case letters and use the - character for spaces. As we are publishing to an npm organisation, we add in the name of the organisation (scope) and then the name we want to give to our widget.

Accept the default for the version. Enter a meaningful description for the widget, this will be displayed in the npm website when the package is published. For the entry point enter dist/index.js Accept the default entry for the test command. Hit Enter to accept for the default git repository. (if you have created the repo this should be pre-populated) Enter some keywords that will make your widget searchable. Hit Enter to accept for the default for the licence.

You should see a read out of what npm is about to create, hit enter accept it.

You should now see a package.json in your folder.

Open the package.json file and add in the following.

"module": "dist/index.es.js",
"jsnext:main": "dist/index.es.js",
"source": "./src/widget.tsx",
"type":"module",
"types": "dist/widget.d.ts",

The next step is to install the libraries that will be used allow us to create and build the desired functionality in the widget.


Open a command prompt/terminal window and ensure that you are in the widget folder.

Enter the following command to install the Esri JSAPI.

npm install @arcgis/core

rollup.js is a build system that will package our widget in to an output module which we can then publish to npm.

Enter the following command and hit enter.

npm install --save-dev rollup 
npm install --save-dev rollup-plugin-commonjs
npm install --save-dev rollup-plugin-peer-deps-external
npm install --save-dev rollup-plugin-node-resolve
npm install --save-dev rollup-plugin-typescript2

To ensure we are writing good quality code, we should install a test system so that we can write unit tests to help prevent regressions coming in to our code. We will use the jest library for testing.


Enter the following commands.

npm install --save-dev jest typescript
npm install --save-dev ts-jest @types/jest
npm install --save-dev ts-jest
npm install --save-dev ts-node

As we want to report the details of our test results to DevOps, we should also install JUnit which allows us to produce output that can be read by DevOps.

npm install --save-dev jest-junit

Finally, we will install copyfiles which will allow us to push files from our build in to our demo app where we can debug it.

npm install --save-dev copyfiles
 

4: Configure components

Add a .gitignore

In the widget folder, add a new file and call it .gitignore and add the following to it.

node_modules
.DS_Store
dist
dist-report
*.local
package-lock.json

This will tell git that we do not want to push these files up in to our repository.


Configure TypeScript

All the code that we write will be done in TypeScript. TypeScript is a superset of JavaScript that provides strong typing for JavaScript, it helps prevent issues that can occur with JavaScript and ultimately aids in producing better code.

Within the widget folder, create a new file called tsconfig.json.

Once created enter the following in to the file. This will tell the TypeScript complier how we want to compile the TypeScript we write in to JavaScript.

{
    "compilerOptions":
    {
        "outDir": "dist",
        "esModuleInterop": true,
        "lib": [ "ES2020", "DOM" ],
        "module": "ES2020",
        "target": "ES2020",
        "experimentalDecorators": true,
        "importHelpers": true,
        "jsx": "react",
        "jsxFactory": "tsx",
        "allowJs": true,
        "declaration": true,
        "moduleResolution": "Node",
        "noImplicitAny": true,
        "noImplicitReturns": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "preserveConstEnums": true,
        "resolveJsonModule": true,
        "sourceMap": true,
        "strict": true,
        "suppressImplicitAnyIndexErrors": true
    },
    "include": [ "src" ],
    "exclude": ["dist","node_modules"]
}
Configure rollup.js

The purpose of rollup.js is to create the build for us, which will take our TypeScript and compile it in to JavaScript. There are a few configurations we need to provide to ensure we get something that works for us and ensures our tests run correctly.


In the /widget folder add a new file called rollup.config.js


Copy the following code in to the new file.

import typescript from "rollup-plugin-typescript2";
import commonjs from "rollup-plugin-commonjs";
import external from "rollup-plugin-peer-deps-external";
import resolve from "rollup-plugin-node-resolve";
import pkg from "./package.json";
export default {
    input: "src/widget.tsx",
    inlineDynamicImports: true,
    output: [
        {
            file: pkg.main,
            format: "cjs",
            exports: "named",
            sourcemap: true
        },
        {
            file: pkg.module,
            format: "es",
            exports: "named",
            sourcemap: true
        }
    ],
    plugins: [
        external(),
        resolve(),
        typescript({
            rollupCommonJSResolveHack: true,
            exclude: "**/__tests__/**",
            clean: true
        }),
        commonjs({
            include: ["node_modules/**"],
            namedExports: {
                "node_modules/react/react.js": [
                    "Children",
                    "Component",
                    "PropTypes",
                    "createElement"
                ],    
                "node_modules/react-dom/index.js": ["render"]
            }
        })
    ]
};

Your widget folder should now look like this.


There will be an error on the tsconfig.json file, but this because the compiler has nothing to compile. So lets add the files to the src folder to allow us to build our widget.

 

5: Scaffold the widget


The code presented here will let you build out a weather widget that will take the input coordinates from a MapView or SceneView and display the current weather. The function of the widget is not the important lesson. The main lesson is to understand how to create a custom Esri JSAPI widget.


Things to note.

  • For a full explanation of how to create a custom widget, look at the Esri developer documentation.

  • You must subclass the widget, with out this the render method never gets called and nothing will refresh in your widget.

  • The widget must extend the base widget class from @arcgis/core/widgets/Widget.js


  • There are no frameworks used here, we are building a widget without anything like React/Vue/NextJS etc. We use tsx to create the UI, but the TypeScript complier knows how to translate this in to the correct UI for us (no React in involved).

  • The only mandatory method you need to provide is a render() method. See the widget lifecycle for the other methods that are available.

  • Any class you create should extend from Accessor and use marked properties, this so that they can be watched by the Esri framework. See more details here.


  • Its a good idea to create to interfaces for your TypeScript classes to allow for subtyping and inheritance.

  • The goal is to understand how to build a widget in a standard way using a Model-View-ViewModel (MVVM) pattern.

The MVVM pattern means that as much as possible, we remove any logic out of our UI, and instead push that in to the ViewModel which is repsonsible for providing/updating data back to the UI to be displayed. Any data we have is represented by the Model which is our representation of the data. This pattern has the added advantage that if a change to the UI is required, a user/developer can consume just the ViewModel and put their own UI on instead.

With this in mind, download the attached files which contain all the files for the widget itself below.



Unzip the files and copy them in to your project, it should be obvious where they go. When you are done, your project should look like this.



Model folder

The model folder contains a set of TypeScript files which are an implementation of the results we get back from the Weather API. These classes are used to hold the details, by creating the classes and creating the structure we can make use of the typing ability in TypeScript to ensure that we aren't tryng to access properties that don't exist or we are using them in the wrong way, eg trying to perform a mathmatical operation on a string.


View-Model folder

The view-model folder contains our view model, this is core part of code for the application, it makes the calls to the Weather API to get the current weather, parses the results and updates the other properies on the view-model.


widget.tsx

The widget.tsx is our view, this is an extension of the base widget class in the Esri JSAPI. It contains the lifecycle methods and also is responsible for instantiating the view-model.

 

6: Create the demo application


In order to debug our widget, we need an application to test it in. In this step we will create a simple application that we can load our widget in to. Using this, we can use the browser debugging tools to find and fix any issues with our widget.


To create the application navigate to the root of the weather widget folder using a command prompt or terminal window.


Enter the following command

npm create vite@latest demo --template vanilla-ts

Select the vanilla option.

Then select the vanilla-ts option.

You should then something like the below.

Next follow the instructions and enter

cd demo

then

npm install

Now try

npm run dev

You should see something like this.


Open a browser at http://localhost:5173 , you should see the basic application.


We now need to update the demo application to allow us to debug our widget.

Open the index.html page under /demo and update the title to ArcGIS Demo Application.

Update the id of the div to viewDiv.



Open the demo/src folder and delete counter.ts.

Open the style.css file and replace the contents with the following:

@import 'https://js.arcgis.com/4.24/@arcgis/core/assets/esri/themes/light/main.css';
@import 'https://s3-us-west-1.amazonaws.com/patterns.esri.com/files/calcite-web/1.2.5/css/calcite-web.min.css';
#viewDiv {
    padding: 0;
    margin: 0;
    height: 100vh;
    width: 100vw;
}

Next open main.ts and replace the contents with the following. Make sure you add in your API Key from WeatherAPI.com

import './style.css'
import WebScene from "@arcgis/core/WebScene";
import SceneView from "@arcgis/core/views/SceneView";
import Widget from './widget';
const scene = new WebScene({
    portalItem: { 
        id: "7dda92b9f6714eb38f6f02050339d3c9" 
    }
});
const view = new SceneView({
    map: scene,
    container: "viewDiv"
});

view.when(() => {
    const widget = new Widget({
        view,
        weatherAPI: "<ENTER YOUR API KEY HERE>",
        weatherURL: "https://api.weatherapi.com/v1"
    });
    view.ui.add(widget, "top-right");
});

The code above imports our dependancies, creates a new WebScene from an item in ArcGIS Online, the adds it to a new SceneView. When the SceneView is ready we add in our weather widget.


Next we want to pull in our widget files in to the demo application, to make this simpler we can use npm to copy the files across for ius. To do this, open the package.json file in the widget folder and find the scripts section.

Add in the following to the section.

"dev": "rollup -c -w",
"build": "rollup -c",
"postbuild": "copyfiles -a src/**/*.* ../demo"

The dev script creates a watch that will build the files we have any time that we do a commit. The build script will run a build, while the postbuild will run once the build script has run. The postbuild will copy the files from our /src folder in to the demo folder where the demo application can pick them up.


Run the below command to test the build and copy process.

run npm build

The TypeScript complier will complain at this stage about the widget import. To fix this, open the tsconfig.json file in the demo folder and add in the following options to the complier options.

"jsx": "react",
"jsxFactory": "tsx",


Within a console cd back to the demo folder and try:

npm run dev.

You should now be able to open a browser and see a web scene loaded and the widget up in the top right corner.

 

7: Add in testing with Jest


Now we have our weather widget up and running, we want to make sure that we keep it running, and when we write new code, we don't introduce new issues. This is where unit testing can help us.

Jest is a test library we can use to write tests against our code. We will need to configure our project to work with it first.


Open the package.json file and update the test command to the following

"test": "node --experimental-vm-modules node_modules/jest/bin/jest.js"

Add a new file called jest.config.ts to the root of the widget folder and add in the following.




/** @type {import('ts-jest/dist/types').InitialOptionsTsJest} */
export default {
    preset: 'ts-jest',
    moduleFileExtensions: [
        "ts",
        "tsx",
        "js",
        "jsx"
        ],
    testEnvironment: 'node',
    testMatch: ['<rootDir>/**/*.test.ts'],
    testPathIgnorePatterns: ['/node_modules/'],
    collectCoverage: true,
    coverageDirectory: "<rootDir>/dist-report",
    coverageReporters: ["json", "html", "cobertura", "lcov"],
    reporters: [
        "default",
        [
            "jest-junit",
            {
                outputDirectory: "dist-report",
                outputName: "ut-report.xml"
            }
        ]
    ],
    coveragePathIgnorePatterns: ['node_modules', 'dist'],
    globals: { 'ts-jest': { diagnostics: false } },
    transform: {},
};

The above code essentially tells jest that we are using TypeScript, to scan the folders for any files with .test.ts in the file name, and that we want to create a code coverage file and we want to report with JUnit.

Open a command prompt and try running

npm run test

You should see something like the following.

Jest is running, but we have no tests, download the below zip, extract the file and place it in the view-model folder. widgetViewModel.test.zip

Try running npm run test again, you will hopefully see something like the below.

If you check the dist-report folder, you should now see it full of content from our test run.


If you right click on the index.html file and click preview, you can see the code coverage. You can also drill down in to this to see where you may be missing lines of code in your tests.

 

8: Create the build pipeline


Now that we have a project with our widget and with tests to ensure that we can keep our code quality up, we need to be able to share it with other people, this can be either with people in our organisation or publicly.


The first step in achieving this, is creating a build pipeline. This is a pipeline that will be triggered anytime we push our code from our local machine back in to our source control in Azure DevOps.


We can do this all within DevOps which allows us to build up the pipeline using the editor.

Within DevOps open the repository, and click the button Setup Build in the top right of the page.

This will open the pipeline editor, select Starter Pipeline from the options presented.

You will now see a starter pipeline with some hello world code.

Remove all code from line 13 down.

On the right hand side you will see the Show assistant button, click that as it will help us insert code snippets for us.

Place the cursor on line 13 and then within the assitant window, search for the npm task and click on it.


In the dialog that shows, keep install in the command window, update the below section to widget. This tells that command that folder in our source code that we want to run the command in is the widget folder.

Click Add at the bottom of the dialog to add this to our pipeline.

The npm task will install the dependancies we need to build the widget within the DevOps pipeline.


The next step we want to do, is run our tests, so that each time we check in the code, we are running our tests to ensure that they are still passing.

Again, search in the assistant window for command line. and click on the Command line task to configure it.



In the Script section, add in the following code npm run test, click the advanced section and add in widget to the working directory.



The above will run our tests and produce the outputs. We now want to do something with those outputs, such as publishing them to our pipeline so we can keeps track of our code quailty.

Again in the asssitant window, search for publish test.



Select the Publish Test Results


Configure the task as follows.



This will publish the results of the test run to Azure DevOps.

We also want to report our test line coverage as well. To do that we need another publish task. In the assistant search for coverage and select the Publish code coverage results task.



Configure the task as follows. This will get the coverage file created by our test run and publish to Azure Devops.



We specify where that file is within our folder structure. The file and type was configured as part of our jest.config.ts file.



Now we have completed all our tests, the next thing that we want to do is build and pack our widget in preparation for publishing it to npm.


Add in two more command line tasks, the first one is to run our build script.

npm run build.

The second one should use:

npm pack 

This is a special command from npm which will pack our build output so it can be published to npm. When creating the tasks, ensure the working directory is set to widget. Your resulting code should look like this.



The pack command creates an archive file of our build for us, we now want to make that available to the DevOps release pipelines, to do this we need to first copy the files to a staging directory, and then publish the artifcate.

In the task assistant, search for copy files and select the basic Copy files task (second in the list below).



Configure the task with the following settings. The contents, tells the task to looks for any files with the .tgz extension, and the target folder we have specified is a special directory created as part of our build pipeline.



The final task we need to do, is publish this archive from the artifact to Azure Pipelines where we can use it in release pipelines. Search for publish build and select the Publish build artifacts task.



You can accept the defaults and add the task.



The completed yml should look like the below.



# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master

pool:
  vmImage: ubuntu-latest

steps:

- task: Npm@1
  displayName: Install NPM
  inputs:
    command: 'install'
    workingDir: 'widget'
    verbose: true

- task: CmdLine@2
  displayName: Run tests
  inputs:
    workingDirectory: 'widget'
    script: 'npm run test'

- task: PublishCodeCoverageResults@1
  inputs:
    codeCoverageTool: 'Cobertura'
    summaryFileLocation: 'widget/dist-report/cobertura-coverage.xml'

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'ut-report.xml'
    searchFolder: 'widget/dist-report'
    testRunTitle: 'Widget Tests'

- task: CmdLine@2
  displayName: Build Widget
  inputs:
    workingDirectory: 'widget'
    script: 'npm run build'

- task: CmdLine@2
  displayName: Pack Widget
  inputs:
    workingDirectory: 'widget'
    script: 'npm pack'

- task: CopyFiles@2
  inputs:
    SourceFolder: 'widget'
    Contents: '*.tgz'
    TargetFolder: '$(Build.ArtifactStagingDirectory)'
   
- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)'
    ArtifactName: 'drop'
    publishLocation: 'Container'


We are now done. Anytime we check the code in to the repository, the above will run, complete our tests and create a build for us. The next step is to take the artifact that we have created and publish through a release pipeline to npm.

 

9: Create the release pipeline


The final step in our custom widget development is to publish the output to npm. In order to do this, you will need an npm account. In the example below, we are publishing to a public npm package, you can also publish to a private package with the same steps. You do not necessarily need to do this, however, one of the goals of this process is to demonstrate the workflow that is most useful for people in organisations creating code that they may not neccessarily want to share with everyone.


Note, before publishing to npm you will have to bump your version number in the package.json, you cannot simply overwrite an existing version.

For more details on creating an npm account see here . For details on creating an organisation, see the details here .

If you have the above, the next item you will need is an auth token to allow you to read and write. We need this token to authenticate when we publish the item to npm. Be careful with these tokens as they should not be shared publicly.

Follow the details here to create a token. Once you have the token we will place it in a variable library where we can keep it secret and make it available to any pipeline we create in DevOps. To do that, go to the Pipelines section and select Library.

Once the page opens, create a new variable group, by clicking on the below button.

Give the group a meaningful name, and then move down to the Variables section and add a new variable called npm_token. Paste your npm token you created previously, and then click the lock button to hide the details.

The token is now stored securely within DevOps, no-one can read it, but we can make it available to all our pipelines.

Now we have our variable, lets go and create our Release pipeline. Under the Pipelines section, select Releases, click New -> New release pipeline.

Select the Empty job template.

Name the stage vLatest.

We need to add in our outputs from our build pipeline to this, so select the Add artifact in the view.

Configure the settings as follows (using your details).

We would like the release to be triggered as soon as a new artifact becomes available, so to do this, click on the lightning bolt next to our artifact.

Enable the Continuous deployment trigger and configure as below. Now when ever we complete a build in our master branch, the release pipeline will be triggered..

At the moment, our Stage does nothing, so we will want to configure that next. First we need to tell the release pipeline about our npm token we created. To do that, click on the Variables tab at the top of the screen.

Then select Variable groups in the lower section and select Link variable group..

Select the variable group you created earlier, you can scope the group to be across the whole release, or to a specific stage. Scoping to a stage is useful when you have multiple stages in your pipeline and you want to configure different settings for each, for example pushing to a test, pre-production and production environment. With the variable group now linked, we can use the npm_token in our pipeline. Go back to the pipeline and we will configure our publish stage. Click to edit the vLatest stage..

Click the + button to add a new task to the Agent job. The first step we need to do is extract the files from our artifact. To do this we will add and configure an Extract files task.

Configure the task as follows, pay attention to update the Archive file pattern to tgz.

Next add a new Bash task.

Configure the task as follows.



The code does the following

  1. Changes the working directory to where our package was extracted to.

  2. Next we tell npm that we need to authenticate, and for this package, use this token. Note the token is paramertised, pulling in our npm token from our variable library.

  3. Finally we publish the package with the latest tag, this is so that it will be the default version that gets installed by npm.

We now have a release pipeline configured that will publish up to npm anytime we check the code in to the master branch. However, we may want to seek approval before we do that. We can use a pre-deployment approval to send an email to approvers, who will then have the ability to approve the release, which will then trigger the release.

Click the lightning bolt on the left side of the vLatest stage.

Enable the Pre-deployment approvals Configure the approvals, by searching for users who can approve. You can set various rules as to how many approvals are required and who can approve.

Save the pipeline, and you are done. Now users who are on the list will recieve an email asking them to approve the release.

Clicking on the release details will take the user to pipeline results, where the results of tests and code coverage can be viewed.



 

Conclusion

We now have widget and workflow that will allow us to develop, test and publish our widget to npm.


We can add to this by using the out of the box tools in DevOps to track and manage enhancements, bugs and new features.


In the next post, I will show how we can wrap the widget we have created in to an Experience Builder application, and build and distribute this via DevOps.


1,254 views0 comments

Recent Posts

See All

Comments


Post: Blog2_Post
bottom of page