Sept - Nov 2021: DE Africa Sandbox Large Instance Trial
The Digital Earth Africa Sandbox is a cloud-based computational platform that operates through a JupyterLab environment. It provides a limited, but free compute resource for technical users and data scientists to explore DE Africa data and products. It enables access to remote-sensing data and analysis tools for ad-hoc report generation and rapid development of new algorithms.
From September to November 2021, Sandbox users will be able to access a more powerful Sandbox instance. This is in addition to the “Default” Sandbox environment that has been accessible since the inception of DE Africa. This article explains why and how to use and access the more powerful version of the Sandbox during this trial.
After you log in to the Digital Earth Africa Sandbox, the Server page will show two options:
Default environment: 2 Cores, 16 GB Memory
Large environment (trial): 4 Cores, 32 GB Memory
Users can select the type of instance they want to start. The instance type can be stopped and changed at any time. Frequently asked questions are addressed in this article.
For Digital Earth Africa to continue providing free compute resources to users, some good practices should be followed. The sandbox is an environment designed for deriving insights from satellite data over modest sized study areas, along with providing a space for prototyping and exploring methods that later may be scaled using methods beyond the capabilities of a Sandbox. We are offering this trial to understand whether increased resources will better meet the needs of our users. A decision to continue this trial will depend on user feedback and cost implications to DE Africa. Therefore, we recommend using the most appropriate instance for your analysis to help us understand the real needs.
The larger compute resources will allow users to conduct the same analysis as on the Default sandbox but over a larger spatial footprint and/or a longer timeframe. In some cases it will also increase the efficiency of your analysis by increasing the number of parallel computations (e.g. for dask-backed analysis workflows).
We strongly recommend starting your analysis by using the Default sandbox so you can prototype, build capacity, or debug your code before you run it finally on the larger instance.
When you are finished with your instance, please shut it down to prevent it consuming resources on your account. Select File > Hub Control Panel. Click Stop My Server and wait for your current instance to power down.
The number of cores represents the number of processors your analysis will have access to to run the code. The number of GB (gigabytes) denotes the amount of memory your analysis can use to temporarily store the inputs and outputs of your calculation. More cores and more memory generally means that you can perform analysis at a larger spatial or temporal scale.
In cases where an analysis relies on parallel computations, such as when a workflow uses dask to break the workflow into chunks, yes this larger sandbox instance will increase the speed of your analysis.
Wait for your server to finish loading. Make sure to save any open files.
Select File > Hub Control Panel. Click Stop My Server and wait for your current instance to power down. Then click Start My Server and follow the prompts to select the instance type you want to use.
Yes. Files you have edited or stored on the Digital Earth Africa Sandbox are linked to your account, not the type of instance. You will be able to access your files from either instance. Make sure to save any files before changing instance type.
One of the advantages of having more cores (CPUs) is using packages like Dask for parallelisation - faster and more efficient processing. If you are new to Dask, we recommend logging in to the Sandbox on the Default instance and navigating to Beginners_guide > 08_Parallel_processing_with_dask. A simple exercise can be followed on the Default instance by exploring the file Frequently_used_code > Generating_geomedian_composites. Once you are familiar with Dask concepts, you can use it to scale up analyses on the Large instance.
No, you will have to close one instance type before you can start the other.
You can contact the DE Africa helpdesk at email@example.com for technical support. Or go to https://helpdesk.digitalearthafrica.org/portal/en/home and benefit from the wisdom of the community of users together with the Knowledge Base and FAQ.