Blog posts tagged "spark"

The purpose of this guide is to highlight the key differences between Apache Spark 3 and Spark 4, and provide advice on how to plan a migration. Let’s get started. The biggest changes Let’s talk about the biggest changes between Apache Spark 3.x and Spark 4. Scala 2.12 no more First up, there’s no support

Here at Canonical we are excited to announce that we have shipped the first release of our solution for enterprise-ready data lakehouses, built on the combination of Apache Spark and Apache Kyuubi. Using our Charmed Apache Kyuubi in integration with Spark, you can deliver a robust, production-level, and open source data lakehouse . Our Ap

Apache Spark has always been very well known for distributing computation among multiple nodes using the assistance of partitions, and CPU cores have always performed processing within a single partition.  What’s less widely known is that it is possible to accelerate Spark with GPUs. Harnessing this power in the right situation brings imm

Everyone agrees security matters – yet when it comes to big data analytics with Apache Spark, it’s not just another checkbox. Spark’s open source Java architecture introduces special security concerns that, if neglected, can quietly reveal sensitive information and interrupt vital functions. Unlike standard software, Spark design permits

I always find the Olympics to be an unusual experience. I’m hardly an athletics fanatic, yet I can’t help but get swept up in the spirit of the competition. When the Olympics took place in Paris last summer, I suddenly began rooting for my country in sports I barely knew existed. I would spend random

Apache Spark is a popular framework for developing distributed, parallel data processing applications. Our solution for Apache Spark on Kubernetes has made significant progress in the past year since we launched, adding support for Apache Iceberg, a new GPU accelerated image using the NVIDIA Spark-RAPIDS plugin, and support for the Volcan

Move over Hadoop, it’s time for Spark on Kubernetes Apache Spark, a framework for parallel distributed data processing, has become a popular choice for building streaming applications, data lake houses and big data extract-transform-load data processing (ETL). It is horizontally scalable, fault-tolerant, and performs well at high scale. H

It’s all about AI these days, so I decided to try and answer the important question: can you make a Spark cluster run AI agents that play a game of Doom, in a multiplayer LAN party? Although I’m no data scientist, I was able to get this to work and I’ll show you how so

We’re super excited to announce that we have shipped the first release of our solution for big data – Charmed Spark. Charmed Spark packages a supported distribution of Apache Spark and optimises it for deployment to Kubernetes, which is where most of the industry is moving these days. Reimagining how to work with big data

Innovate at speed with AI. Stay secure and compliant with Ubuntu Pro Date: 16-20 October 2023 Location: Dubai, UAE Booth: Booth B31, Hall 26, DevSlam Canonical is excited to attend GITEX Global 2023, the largest event in the Middle East. Generative AI, predictive analytics and multi-cloud environments are at the heart of a technological r

I’ve read and watched more than a few articles about ChatGPT in the last couple of months. It seems the large language model AI hype machine just can’t stop.  As somebody with a passion for music production, some of the more interesting things I’ve seen included a guy using ChatGPT to build a virtual effect

We’ve all read the headlines about spectacular data breaches and other security incidents, and the impact that they have had on the victim organisations. And in some ways there’s no place more vulnerable to attack than a big data environment like a data lake.

Juju makes it easy to setup and monitor a Spark cluster with a few commands. In this guide we will setup a new cluster and deploy a Spark job using this tool. According to the official definition, Juju is a service modelling tool that allows people to model, configure and deploy applications in the cloud.

With the amount of industry excitement around virtualized cloud, it is easy to downplay the importance of directly utilizing bare metal infrastructure. And yet bare-metal is the one thing every experienced operator knows they can rely on when deploying mission-critical services, representing guaranteed resource, predictable performance, a

Juju makes it easy to model and deploy Spark and other analytics solution bundles IBM has extended its range of Ubuntu supported systems with today’s announcement of the Power Systems LC models. Canonical, along with fellow OpenPOWER Foundation members – Mellanox, NVIDIA, Tyan and Wistron, collaborated with IBM on the development of these

What do you get when you combine Juju with Spark & Apache Zeppelin, Raspberry Pi with Snappy, Bluetooth Low Energy, DeviceHive and a TI SensorTag? Check out the video from Dataart.