Introduction
5G usecases, digital transformation, COVID-19 pandemic - all of this is causing an explosion of data usage. It’s needed to bring computation at the edge to resolve latency issues and enable artificial intelligence with machine learning to optimize data processing. According to several studies e.g [1] edge computing will grow at a compound annual growth rate (CAGR) of 21.6% between 2022 and 2028 to hit an estimated $156 billions by 2030. There are multiple factors behind that growth, one of them is the advancing of 5G, which is enabling the establishment of new telco AI & IoT usecases. IoT is perceived as a key enabler for digital transformation and it's improving enterprise efficiencies. The Tietoevry mission is to help our customers and partners with such digital transformation.
In Barcelona MWC2023 we demonstrated our edge computing usecase, which is showing Tietoevry capabilities in cloud edge computing domain. Together with Intel and Advantech, we brought our Scalable Edge computing platform usecase [ 2 ] into 2023 using newer version of all involved HW & SW components. Our usecase is demonstrating containerized Automatic Pedestrian Alert System (APAS) application, but in general any other edge usecase can be considered for such scalable edge platform solution. APAS application is designed to alert drivers about pedestrians potentially crossing the roads, which is especially crucial in difficult driving conditions or multi-lane support. This usecase was successfully presented in City of Tampere in 2019 [3].
APAS application was onboarded on Intel® Smart Edge , as it exposes Intel hardware features to the Kubernetes based containerized Edge environment and enables easy deployment and optimized orchestration of various usecases starting from media analytics through Content Delivery Network (CDN) up to 5G access and core network functions.
Platform
HW Platform used in this case is based on Advantech SKY-8132S [4] , which is a compact 1U edge server based on 3rd Gen Intel® Xeon® Scalable processors. For the scaling scenario we are using also Advantech VEGA-3500 [5] , which is 11th Gen Intel® Core™ i7 processor-based UHD video accelerator card.
Similar benchmarking was done during 2021 on top of following HW & SW stack.
Benchmarking
We have used the same key performance indicators as in 2021. Our goal is to demonstrate how many different camera streams can be processed on single low/mid cost edge platform and how the capabilities of such edge platforms are evolving over time. We have also used the same 3fps input camera streams and OpenVINO™ toolkit pretrained object detection [6] with SSD (Single Shot MultiBox Detector) deep learning model and FP16 precision. Focus areas are zebra crossings and crossroads are reframed to 512x512 pixels for inference processing. The idea is to keep the test environment as close to the original benchmarking, which was done in 2021 but use newer HW & SW stack to visualize the progress.
Based on that criteria we have performed a range of tests and focused on 3 main scenarios:
- Scenario1: to indicate how many parallel camera streams we can run on SKY-8132S and compare it with our previous results from SKY-8101
- Scenario2: to indicate how many parallel camera streams we can run on VEGA-3500, which is PCIe UHD card and compare it it older but VPU specialized VEGA-340 used during 2021 benchmarking
- Scenario3: to indicate how many parallel camera streams we can run on whole platform using combined computing power of SKY-8132S CPU and VEGA-3500 card and compare this with previously tested Scalable edge platform from 2021 benchmarking
We have collected following data, which is very good and in some extent better than what we expected. Average inference is on acceptable level and it indicates that so many camera streams can be processed. However there were some peaks detected, where inference took much longer than what is acceptable. It occured on VEGA-3500, where we hit also memory limitation (VEGA-3500 has own RAM memory 2x 16GB) and there was some disc swapping happening. We believe that this can be further optimized in APAS code for instance by having a dedicated container for OpenVINO™ processing instead of having multiple dedicated container for each camera stream. Our team is already now working on how to optimize this further.
| Minimal inference | Average inference | Maximal inference | Load Level | Comment | |
|---|---|---|---|---|---|
| Scenario1 | 1.85 ms | 106.270 ms | 1098.104 ms | 50 streams | Host CPU only |
| Scenario2 | 77 ms | 40.967 ms | 4700.260 ms | 15+15 streams | VEGA-3500 only |
| Scenario3 | 78 ms | 73.621 ms | 4700.274 ms | 50+15+15 | Both host CPU and VEGA-3500 |
| System component/Capacity | Minimum configuration to serve 50 camera streams (host only) | Minimum configuration to serve 30 camera streams (VEGA-3500 only) | Minimum configuration to serve 80 camera streams (host&VEGA-3500) |
|---|---|---|---|
 | Advantech SKY-8132S Edge Server | ||
 | Intel® Xeon® Gold 6338N CPU @2.20GHz | N/A | Intel® Xeon® Gold 6338N CPU @2.20GHz |
 | N/A | Advantech VEGA-3500 PCIe x4 with 2x Intel® i5-1145G7E @ 2.60GHz | Advantech VEGA-3500 PCIe x4 with 2x Intel® i5-1145G7E @ 2.60GHz |
 | min 60GB RAM | min 2x16GB RAM (whole memory of VEGA-3500) | min 60+32GB RAM (whole memory of VEGA-3500) |
 | 480GB, SSD | min 92GB, SSD | min 92GB, SSD |
 | Intel Corporation I210 Gigabit Network Connection driver compatible card | ||
Summary
We successfully demonstrated significant improvements when comparing 2nd Gen Intel® Xeon® with it’s successor from 3rd Gen Intel® Xeon® family. A newer VEGA-3500 card is also bringing significant boost to the scaling capabilities of such edge platform. VEGA-3500 provides a independent computing unit with dedicated RAM, CPU and OS. So benchmarking executed on VEGA-3500 did not impact results on underlying host tests, which was not the case during 2021 tests with VEGA-340. VEGA-3500 provides even better scaling options as several cards can be inserted into SKY-8000 Advantech platforms.
Edge computing is an important area for Tietoevry, so further improvements on this usecase is planned, as well as other exciting usecases and research work. So stay tuned.
- https://www.globenewswire.com/news-release/2022/09/27/2523687/0/en/Edge-Computing-Market-Forecast-to-2028-COVID-19-Impact-and-Global-Analysis-By-Component-Organization-Size-Application-and-Industry.html
- https://www.tietoevry.com/siteassets/files/pds/wps/tietoevry_scalable_edge_reference_platform.pdf
- https://www.tietoevry.com/en/newsroom/all-news-and-releases/press-releases/2019/11/city-of-tampere-and-tieto-develop-ai-iot-test-solution-for-pedestrian-traffic-safety/
- https://www.advantech.com/en-eu/products/8211d13b-39cb-452f-b24b-14e37cab219b/sky-8132s/mod_3ea6ffc6-6294-4f70-897b-070b82b37c7d
- https://www.advantech.com/en-eu/products/7002552e-4890-4c95-ace5-29ebdad7f992/vega-3500/mod_48d781d6-0aab-4552-81b5-f982f0e105a3
- https://docs.openvino.ai/2020.4/omz_models_intel_person_detection_0102_description_person_detection_0102.html#example