Utilizing Tech

As we've discussed all season on Utilizing Edge, innovation is coming from all directions, including hardware, software, and applications. This special crossover episode of the On-Premise IT and Utilizing Tech podcasts features Edge Field Day delegates Brian Knudtson, Ned Bellavance, and Jody Lemoine discussing their perspectives about edge innovation with Stephen Foskett. The primary drivers at the edge are integration, efficiency, and connectivity, as well as the unique needs of the applications there. Starting with hardware, customers are headed in two directions, with more enterprise availability features deployed in some locations and less-capable hardware in others, both in terms of compute and networking. At the software level, most edge infrastructure is hyper-converged, meaning that multiple layers of the stack are integrated in software and managed as one. Although intended as an application platform, Kubernetes is being deployed as a packaging abstraction and distribution solution at the edge.

Host:

Edge is leading the hardware industry into a new era of innovation. In this episode of Utilizing Tech, Stephen Foskett, and co-hosts, Allyson Klein and Alistair Cooke, sit down to dissect this. In the wake of Intel’s discontinuation of its NUC product line, a question that is in everyone’s mind is, what’s next. Intel, and many behemoths like it, have a proud legacy of knowing how to break a stalemate and preserve the churn, and even if that means stepping up and pulling the plug on an old product. It oxygenates the marketplace, welcoming new solutions and keeping the wheel of innovation moving. In the context of the emerging paradigm of edge, this change will likely propel the market towards a new breed of powerful, low-cost, pocket-size hardware that delivers breakthrough energy-efficiency and compute performance with little infrastructure.

Hosts:

Personal devices are increasingly being used across enterprise IT, from smartphones to wearable devices, and these are becoming the true edge. This episode of Utilizing Edge brings Field Day delegates Mark Houtz and Jim Czuprynski together with Stephen Foskett to discuss the personal side of the edge. Mobile Device Management (MDM) has been used to manage smartphones and similar technology is used for personal computers, but it seems inevitable that there will be a mixing of business and private data. The mix and match of personal devices at the edge is sure to be a topic of future focus.

Host:

Although blockchain technology has been tainted by scammers, the core idea of distributed consensus is relevant for certain edge applications. This episode of Utilizing Edge focuses on practical application of blockchain technology in edge and IoT, featuring Jason Benedicic, Alastair Cooke, and Stephen Foskett. Two key concepts in blockchain are the distributed ledger and consensus approach. There is a lot of work being done to apply blockchain in medical, finance, IoT, and proof of provenance. Although many applications can rely on authority or quorum, larger and more heterogeneous applications might benefit from consensus instead. The other aspect of blockchain, chain of custody and immutability of data, is potentially relevant in preventing supply chain attacks and dealing with transient devices. It's important to remember that many of the things that have put people off blockchain, from financialization to public exposure of transactions, are not necessarily required in all blockchains. Guest:

Computing and sensors have been deployed at the edge for decades, and this is certainly true in the agriculture sector. This episode of Utilizing Edge features Edge Field Day delegates Ben Young and Alastair Cooke discussing with Stephen Foskett the evolution of connectivity and orchestration at the edge. Governmental policies can have a huge impact on the deployment of technology, from connectivity to environmental regulations, as has the emergence of inexpensive sensors, batteries, and processors thanks to the mobile phone industry, along with solar power. Cloud-inspired automation and orchestration is another key enabler of technology in agriculture and at the edge, with zero-touch provisioning, management, and central maintenance. Containerized applications, caching, and cloud-derived control planes are very popular at the edge, though not exactly the same way as in the cloud.
Hosts:

VMware Explore 2023 featured a broad set of products and technologies, all of which could be included under the banner of edge. This episode of Utilizing Edge considers this dynamic and varied world with Roy Chua and Brian Knudtson, along with host Stephen Foskett. Virtualization, networking, storage, and orchestration are all key edge technologies, and VMware and its partners are deeply involved in bringing these technologies to the edge. VMware announced Edge Cloud Orchestrator, new security capabilities, integrated networking, client integration, and the exciting potential of private 5G networks. Dive into this episode to unravel the complexities and promising future of edge computing.

Host:

Industrial control and operational technology systems face the same forces as IT applications and are rapidly being integrated into comprehensive edge infrastructure. This episode of Utilizing Edge brings Andy Foster of IOTech to discuss standardization of industrial IoT with Allyson Klein and Stephen Foskett. There are many more similarities between industrial control systems and information technology systems than many people realize. Both are impacted by standards in commodity hardware, networking and communications protocols, and cloud integration. And OT and IT are colliding at the edge, with a massive opportunity to leverage data.

Hosts:

Media and entertainment is one industry generating and moving vast amounts of data at the edge. This episode of Utilizing Edge brings Jimmy Fusil of Tsecond into a discussion with Alastair Cooke and Stephen Foskett about the unique data challenges in media and entertainment. There is a great need for local processing, secure data transport, and efficient storage utilization in managing the ever-increasing volume of data generated during film and TV production. The podcast explores the continued relevance of physical media, like tape, for long-term archiving, while also emphasizing the security measures necessary to protect creative privacy and prevent potential plot leaks in the industry. Fusil also discusses Tsecond’s groundbreaking product, Bryck, a portable NVMe storage device capable of holding up to a petabyte of data with a remarkable 40GBps data transfer speed. The discussion highlights how Bryck’s high-performance storage facilitates on-device data processing, ensuring quick backups and maintaining data integrity. It’s critical to consider edge storage solutions like Bryck in many data-intensive sectors beyond media and entertainment as well. Hosts:

Edge computing comes in many forms and brings many challenges, including bandwidth limitations, network reliability issues, and limited space. This episode of Utilizing Edge brings Brian Chambers, Alastair Cooke, and Stephen Foskett together to discuss the state of the edge in 2023. Industries like retail, multi-tenant environments, and industrial IoT find practical applications, but defining the edge remains an ongoing exploration. Implementation varies, from repurposing existing technologies to adopting modern approaches like containers and function as a service. The debate between virtual machines and containers continues, driven by organizational comfort. Despite constraints, edge environments offer greater control and accountability. The future promises more innovation and adoption, cementing edge computing’s significance in the tech landscape.

Hosts:

Although we use Intel's NUC as a shorthand for the type of hardware deployed at the far edge, this recently-cancelled platform isn't all there is. This episode of Utilizing Edge looks beyond the NUC, to platforms from Lenovo, Nvidia, and more, with Julian Chesterfield of Sunlight, Andrew Green, and Stephen Foskett. ARM-based solutions, many using the Nvidia Jetson platform, are particularly interesting given their low cost and power consumption and strong GPUs for edge AI. A hyperconverged stack runs all of the components required for high availability, including storage and networking, in software spanning all of the nodes in a cluster, and this is commonly deployed on low-cost devices at the far edge. The trend to deploying applications at the edge is driven both by new hardware and software capabilities and the changing expectation of consumers and businesses.

Hosts:

Datacenter IT is used to having tight control over infrastructure and applications, but this is challenging to maintain at the edge. This episode of Utilizing Edge features Pierluca Chiodelli of Dell Technology discussing the modern edge application platform with Allyson Klein and Stephen Foskett. A typical edge environment features many different platforms, devices, and connections that must be deployed, managed, and controlled remotely. When looking at the modern edge, Chiodelli recognizes the different personas and needs and constructs a plan to achieve the required outcome at this location. Modern applications need specialized hardware and connectivity that must be supported, deployed, and managed.

Hosts:

Between the so-called last mile and first mile lies the middle mile, the realm of colocation and network service providers. This episode of Utilizing Tech features Roy Chua and Allyson Klein, discussing the middle mile with Stephen Foskett. This middle area includes content delivery services like Varnish and Akamai, as well as companies like Cloudflare that are delivering content and compute there. The middle network includes providers like Equinix, Digital Realty, and Megaport, which provide connectivity to the cloud and service providers, the hyperscalers themselves, and some interesting networking startups like Packet Fabric and Graphiant. We must also consider observability, with companies like cPacket and Kentik as well as companies like Cisco and Juniper Networks.

When it comes to edge computing, money is not limitless. Joining us for this episode of Utilizing Edge is Carlo Daffara of NodeWeaver, who discusses the unique economic challenges of edge with Alastair Cooke and Stephen Foskett. Cost is always a factor for technology decisions, but every decision is multiplied when designing edge infrastructure with hundreds or thousands of nodes. Total Cost of Ownership is a critical consideration, especially operations and deployment on-site at remote locations, and the duration of deployment must also be taken into consideration. Part of the solution is designing a very compact and flexible system, but the system must also work with nearly any configuration, from virtual machines to Kubernetes. Another issue is the fact that technology will change over time and the system must be adaptable to different hardware platforms. It is critical to consider not just the cost of hardware but also the cost of maintenance and long-term operation.

Hosts:

One of the main differentiators for edge computing is developing a scalable architecture that works everywhere, from deployment to support to updates. This episode of Utilizing Edge welcomes Dave Demlow of Scale Computing discussing the need for scalable architecture at the edge. Scale Computing discussed Zero-Touch Provisioning and Disposable Units of Compute at their Edge Field Day presentation, and we kick off the discussion with these concepts. We also consider the undifferentiated heavy lifting of cloud infrastructure and the tools for infrastructure as code and patch management in this different environment. Ultimately the differentiator is scale, and the key challenge for designing infrastructure for the edge is making sure it can be deployed and supported at hundreds or thousands of sites.

Hosts:

There is a long-standing gulf between developers and operations, let alone infrastructure, and this is made worse by the scale and limitations of edge computing. This episode of Utilizing Edge features Carl Moberg of Avassa discussing the application-first mindset of developers with Brian Chambers and Stephen Foskett. As we've been discussing, it's critical to standardize infrastructure to make them supportable at the edge, yet we also must make platforms that are attractive to application owners.

Hosts:

Edge has many different stakeholders, applications, and needs, and this is especially true in distributed retail environments. This episode of Utilizing Edge features Simon Gamble of Mako Networks talking with Brian Chambers and Stephen Foskett about the complexities of technology at the retail edge. The key according to Gamble is segmentation of brands, franchisees, and technical applications. In some cases a single location might even include multiple separate companies or tenants under the same roof. Video, sensors, IoT, and location-based services are coming to retail locations as well, and some of these leverage outside service providers as well. Although sharing infrastructure is desirable, segmentation and security is key. Retail edge environments are increasingly complicated, but there are many ways to consolidate, converge, and standardize to make them practical to implement.

Hosts:

Edge environments were historically very specialized, but virtualization and cloud technology is enabling companies to deploy commodity platforms at the edge. This episode of Utilizing Edge features Raghu Vatte of ZEDEDA discussing this commoditization with Alastair Cooke and Stephen Foskett. Although the transition is still getting started, standard compute platforms are rapidly being exploited at edge locations, from warehouses to retail to industrial. Even is some specialized hardware is still needed, a unified platform can increasingly absorb a majority of applications at the edge. Another factor contributing to commoditization is the standardization of application requirements, with most now virtualized or containerized with standards developing for I/O and shared hardware resources.
Hosts:

Although the technology is roughly similar to datacenter or cloud, the unique challenges of edge computing require new approaches to storage, networking, orchestration, deployment, and more. We were kicking off a new season of Utilizing Tech focused on edge computing, featuring Alastair Cooke and Brian Chambers as co-hosts along with Stephen Foskett. One of the key differences for edge computing is the strictly constrained resources there as well as the massive scale. What we now call edge has existed for decades but the proprietary hardware previously used has been largely replaced by commodity and internet-connected systems that inherit technologies like virtualization, containerization, and hyperconvergence. Businesses and consumers expect more interactivity and capability in retail, restaurants, and hospitality environments, and the IoT revolution is strongly affecting manufacturing, military, and industrial settings. But what is the edge really, and how do we deliver services there? That's the question we hope to answer in this season of Utilizing Edge.

Hosts:

We're wrapping up this season of Utilizing Tech by asking the key question: Why would a system architect choose to utilize CXL in their designs? This episode of Utilizing CXL features Stephen Foskett, Nathan Bennett, and Craig Rodgers discussing the practical prospects and benefits of CXL.

Perhaps no one can tell the story of Compute Express Link (CXL) better than Jim Pappas, who was involved in the development of nearly every related technology, from PCI to UCIe. This episode wraps up the season of Utilizing Tech with Stephen Foskett and Craig Rodgers discussing the evolution of CXL with Jim Pappas, Director of Technology Initiatives at Intel and Chairman of the CXL Consortium. No matter how good the technology is, it needs widespread industry support, backwards and forwards compatibility, and open cooperation, and that's what made technologies like PCI, PCI Express, USB, and now CXL successful.  

Moving memory and other resources off the system bus to CXL is exciting, but how do we ensure that these systems will be reliable, available, and serviceable? This episode of Utilizing Tech features Mark Orthodoxou, VP of Strategic Marketing for Datacenter Products at Rambus discussing with Stephen Foskett and Craig Rodgers the technology and standards required for mass adoption of CXL-attached memory. Rambus brings decades of experience and a breadth of technology to the deployment of memory in high-performance and highly-available systems. As a CXL Consortium member, Rambus is bringing this experience to CXL, enabling the technology across the ecosystem. Memory expansion with CXL is being deployed today, and memory pooling over CXL fabrics is coming, but it is disaggregation and rack-scale architecture that will ultimately be the result of the adoption of CXL.

CXL grew out of a long history of Intel platform technology and promises to revolutionize this platform. This episode of Utilizing CXL features Allyson Klein of TechArena, who shares her vision for the future of disaggregated servers with Craig Rodgers and Stephen Foskett. The foundation for CXL was laid with coherent interconnects and fabrics in previous decades, and the concept of tiered memory draws heavily on the history of Intel and Micron 3D XPoint and Optane Persistent Memory. As was the case for virtualization, CXL has an initial value proposition and a revolutionary future. CXL opens up virtually unlimited memory, pools of processors and offload engines, and unprecedented flexibility. Ultimately CXL will allow us to deliver compute resources that are truly balanced and matched to the needs of each workload. We also discussed UCIe, which promises to enable mix and match chiplets within future processors in a way that is compatible with CXL.

Memory expansion is the first application for CXL, and memory pooling is coming next, but this technology will eventually support storage, Ethernet, and more. This episode of Utilizing CXL brings Ronen Hyatt, CEO of UnifabriX, to discuss memory and more over CXL with Nathan Bennett and Stephen Foskett. UnifabriX demonstrated high-performance compute benchmarks using their Smart Memory Node at Super Computing 22 and claims to be the first to show a CXL 3.0 fabric. But the company is also promising NVMe storage and Ethernet connectivity over the CXL fabric. This enables each server to have the right type and capacity of connectivity, from basic Ethernet to DPU, with dynamic reconfiguration. The fabric can also contain an NVMe storage target that combines DRAM and flash and can be dynamically allocated.

Tiered memory will have different performance, so operating systems will need to incorporate techniques to adapt to pages with different characteristics. This episode of Utilizing CXL features Hasan Al Maruf, part of a team that developed transparent page placement for Linux. He began his work enabling transparent page placement for InfiniBand-connected peers before applying the concept to NUMA nodes and now CXL memory.

Bringing CXL to market requires a wide variety of components, and Marvell is a key supplier to datacenter and cloud. This episode of Utilizing CXL features Shalesh Thusoo of Marvell, who discusses the many solutions they are creating to bring CXL to market. Current products are directly connecting memory to CPUs via CXL but soon we will see products enabling the sharing of memory between hosts to enable new applications.  

From Marvell: This podcast contains forward-looking statements within the meaning of the federal securities laws that involve risks and uncertainties. Forward-looking statements include, without limitation, any statement that may predict, forecast, indicate or imply future events or achievements. Actual events or results may differ materially from those contemplated in this podcast. Forward-looking statements speak only as of the date they are made. Listeners are cautioned not to put undue reliance on forward-looking statements, and no person assumes any obligation to update or revise any such forward-looking statements, whether as a result of new information, future events or otherwise.

The CXL Consortium is an open industry standard group responsible for technical specifications and standards. This episode of Utilizing CXL brings Siamak Tavallaei, CXL president, to discuss the emergence of Compute Express Link. The goal of CXL is to disaggregate servers, moving memory away from the CPU and enabling users to compose servers more flexibly. We have already disaggregated storage from compute, and breaking the fixed memory interconnect will have a similar effect. But we also need the system to be manageable, with software to configure and orchestrate resources. The CXL Consortium is facilitating the development of all this in association with nearly every company in the IT industry.

Although the emergence of CXL in server CPUs is big news, the inclusion of this technology in ARM processor IP is just as important. In this episode of Utilizing CXL, Eddie Ramirez of ARM joins Craig Rodgers and Stephen Foskett to discuss CXL in the ARM-powered ecosystem. ARM develops processor IP that is used in CPUs as well as supporting processors throughout the datacenter. We begin with a discussion of CXL 1.1, which brings memory expansion to ARM CPUs. But ARM is also delivering CXL 2.0 which would allow memory pooling to increase the utilization of memory, and thus overall system efficiency. The next step is true heterogeneous compute, with accelerators like GPU and DPU sharing memory with CPUs in a flexible fabric that can leverage CXL.  

With both AMD and Intel supporting CXL in their newly-launched server platforms, it's worth taking a moment to consider what has been delivered to date. This special episode of Utilizing CXL brings hosts Stephen Foskett, Craig Rodgers, and Nathan Bennett together to look back at 6 months of CXL announcements now that real products are shipping. We were pleased to see the support delivered by Intel and AMD, but also from Astera Labs, Samsung, SK Hynix, and MemVerge, as well as promising developments from ARM, Elastics Cloud, IntelliProp, Marvell, and many others.

CXL is being rolled out in production and more reliability, scalability, and security features are being added all the time. This episode of Utilizing Tech focuses on enterprise-grade CXL with John Spiers, CEO of IntelliProp, talking about the ongoing evolution of CXL. IntelliProp is bringing a CXL fabric solution to market the enables memory expansion outside the server over a fabric. CXL 3.0 introduces memory fabrics, but it will take more development to bring features like high availability, routing, peer-to-peer, failover, and re-routing while preserving cache coherency and enabling management, and IntelliProp is working to bring all this to the spec.  

As we discussed in our last episode with Dan Ernst from Microsoft Azure, CXL delivers acceptable memory latency, and John expects that this will continue with fabrics to some extent. IntelliProp will profile memory and enable tiered memory that matches application needs. The company is also working to enable advanced features, including sharing memory between systems, in association with the CXL consortium.

Cloud architects have long wished for more flexibility in how memory is provisioned in servers, and CXL is finally delivering on a decade of promises. In this episode of Utilizing Tech, Stephen Foskett and Craig Rodgers talk to Dan Ernst of Microsoft, who is deeply involved in bringing CXL-attached memory to fruition within Azure. Dan was previously involved in the Gen-Z effort, and feels that CXL picks up the baton and brings valuable real-world benefits to cloud server architecture. Memory has a bigger impact on overall IT platforms than many are aware, and is already the costliest component in large servers. Per Amdahl's Law, it makes sense to look for cost savings in memory. And this is doubly the case because DIMMs only come in certain sizes so memory is usually over-specified. CXL memory modules cost a bit more than a DIMM, but this is offset by efficient right-sizing, and CXL is already cheaper than 3D stacked DIMMs.

As 2023 arrives, so do two server platforms that support CXL, along with associated chipsets, software, and memory expansion cards. This special episode of Utilizing Tech features the three hosts of this season, Stephen Foskett, Craig Rodgers, and Nathan Bennett, discussing the prospects for CXL in 2023. AMD recently introduced Genoa Epyc, featuring CXL and PCIe 5, and it is widely expected that Intel will introduce Sapphire Rapids Xeon very soon with similar support. We expect these new server platforms to be adopted quickly by hyperscalers and to reach the enterprise datacenter throughout the year. We wonder what CXL memory expansion might bring, from other types of DRAM to persistent memory and possible even Optane. The prospect for shared and pooled memory is perhaps a little further off, but we have already heard that this capability might come to CXL 1.1 via device-specific features. What could go wrong? Enterprise server vendors might not embrace composability for various reasons, and this could derail CXL in the datacenter. Another concern is security, especially for shared memory and devices. The big differentiator will be software that enables systems to take advantage of features from memory expansion to pooling to disaggregation to composability.

The key to building an ideal server is balance, and this is why CXL is important, since it helps overcome the inflexible nature of modern server architectures. In this episode of Utilizing CXL, Stephen Foskett and Craig Rodgers talk with Ahmad Danesh of Astera Labs about the company's CXL-based memory expansion technology. Although there are many CXL memory expansion chips coming to market, the industry is keen on interoperability testing to make sure everything works as expected. These products are differentiated based on their reliability, performance, and security, including trusted hardware and encryption. This is especially important with AMD Genoa having recently been launched and Intel Sapphire Rapids coming very soon. Security is very important as memory moves further from the CPU and is shared and pooled among multiple servers. Ahmad also suggests that CXL memory can perform as well as local memory, so specialized memory tiering software might not always be needed. But this technology also allows other types of memory to be used, including non-DDR5 DRAM and potentially future non-DRAM memory.

CXL couldn't be utilized until there was a server platform that supported it, so we're very excited to see AMD launch their next-generation Epyc server platform, code-name Genoa. In this episode of Utilizing CXL, Stephen Foskett and Craig Rodgers talk with Mahesh Wagh of AMD and the CXL Consortium about this important release. The first step to bring CXL to market is to prove it is functional and performs well with a mainstream platform like AMD 4th-generation Epyc. AMD is bringing CXL to market as a tiered memory solution that performs similar to as memory on a remote NUMA socket without any special configuration or software. But AMD also supports other memory technologies, including hierarchical memory with software, security, pooling, and even memory sharing with specialized software. Although Epyc is said to only support CXL 1.1, later spec devices will be backwards-compatible and the platform also supports some CXL 2.0 features including global flush for persistent memory, firmware-first error handling, and device-specific capabilities.

Today's AI and ML systems use proprietary interconnects, which limits the choices available to customers. CXL technology promises to enable greater interoperability, and this is the focus for Xconn Technology. In this episode of Utilizing CXL, Gerry Fan of Xconn joins Stephen Foskett and Craig Rodgers to discuss the ways that CXL can improve machine learning processing. The CXL Consortium is working with nearly every company in the IT industry to bring this promise to life, but we need hardware and software to enable memory pooling, device sharing, and more. The initial CXL products enable right-sizing memory, regardless of the specific architectural details of the CPU chosen. The next addition will be disaggregated and pooled memory using CXL switches, and this is coming to market in the next year or so. This will enable massive pools of memory on-demand for intensive applications. Xconn promises to make memory pooling available to CXL 1.1 hosts as well, and is working on a fabric manager to enable this.

Experts call CXL the lifeblood of composable datacenter infrastructure, and for good reason. It has unlocked tremendous possibilities and is reshaping the server architecture for good. Over 190 companies including some of the biggest names in the industry are involved in it, and as new versions of CXL are rolling out, the technology is clearly taking steps towards maturity. But the market needs more CXL-based technologies to kick-start its evolution.  

In this episode of Utilizing CXL, hosts Stephen Foskett and Craig Rodgers join Founder and CEO of Elastics.cloud, George Apostol to talk about this transition and the need for CXL solutions, and what Elastics.cloud is bringing to market.  

Although x86 servers have some configuration and expansion options, they are increasingly monolithic, putting customers at the whim of the system vendor. CXL promises to change this, allowing customers to configure, and reconfigure, servers according to their needs. On this episode of Utilizing CXL, Ryan Baxter of Micron Technology joins Stephen Foskett and Nathan Bennett to talk about the options that CXL brings to the table. One of the first benefits of CXL technology is memory expansion, allowing servers to be created with exactly the right amount of memory instead of over-populating memory to fill channels and meet needs. But it will soon allow truly modular servers with exactly the right combination of CPU, memory, storage, and I/O.

Data throughput has grown in leaps and bounds with the advent of AI. But as COVID-era digital transformation left the existing systems stressed out, CXL arrived at the heels of that. The newest memory solution that has got everybody talking, CXL is full of promises for AI computing. With the release of v3.0, CXL has started to gather more steam. More companies are now dipping their toes in CXL water bringing to the market their own brand of CXL products making the technology reachable for enterprises.  

In this episode of Utilizing CXL Stephen Foskett and Craig Rodgers sit down with guest Yue Li, Co-Founder and CTO at MemVerge and hold an illuminating discussion on the current CXL product market and things MemVerge is doing on the software side of things.  

The first CXL products have emerged, with Samsung delivering memory and storage expanders and MemVerge supporting big memory with their software. Stephen Foskett discusses these products with Julie Choi of Samsung, Steve Scargall of MemVerge, Shalesh Thusoo of Marvell, and George Apostol of Elastics.cloud to discuss current and emerging CXL products. This special episode of Utilizing CXL was recorded live at CXL Forum in New York, with the entire industry watching. Once memory expansion is delivered, where do we go next? Marvell is working to support the new protocol in chipsets, and Elastics Cloud developing CXL fabric switches. Everyone is ready for Intel and AMD to release their next-generation server chips, which natively support CXL, and the CXL Consortium is already working on the next release!

Links

The emerging CXL standard is making waves, promising big memory, new system architecture, and maybe even rack-scale computing. Utilizing Tech is switching focus from AI and ML to CXL for Season 4, and that means new companies, new technology and new hosts! Join Stephen Foskett as he introduces Utilizing CXL along with co-hosts Nathan Bennett and Craig Rodgers. Look for new episodes of Utilizing CXL every Monday!  

Links

Frederic Van Haren and Stephen Foskett look back on all the subjects covered during Season 3 of Utilizing AI. The podcast covered many topics, from religious and ethical implications of AI to the technology that enables machine learning, but one topic that stands out is data science. If data is the key to AI, then the collection, management, organization, and sharing of data is a critical element of making AI projects possible. We also continue our “three questions” tradition by bringing in open-ended questions from Rich Harang of Duo Security, Sunil Samel of Akridata, Adi Gelvan of Speedb, Bin Fan of Alluxio, Professor Katina Michael, and David Kanter of MLCommons.

Three Questions:

Stephen's Question: Can you think of an application for ML that has not yet been rolled out but will make a major impact in the future?

Frederic's Question: What market is going to benefit the most from AI technology in the next 12 months

Rich Harang Senior Technical Lead, Duo Security: In an alternate timeline where we didn't develop automatic-differentiation and put it on top of GUPs do this entire deep learning hardware family that we depend on now never got invented. What would the dominat AI/ ML technology be and what would have been different? 

Sunil Samel, VP of Pusiness Development, Akriadata: How will new technologies like AI help marginalized members of the communities. Folks like senior citizens, minorities, pepole with disabilities, veterans trying to reenter civilian life?

Adi Gelvan, CEO and Co-Founder of Speedb: What do you think the risks of AI are and what is your recommended solution?

Bin Fan, Founding Member, Alluxio: Im wondering if AI can help with a humanitarian crisis happening in the future?

Katina Michael, Professor, School for the Future of Innovation in Society, Arizona State University: If AI was to self replicate what would be the first thing it would do?

David Kanter, Executive Director of MLCommons: what s a problem in the AI world where you are held back by the lack of good publicly available data?

Hosts:

How fast is your machine learning infrastructure, and how do you measure it? That's the topic of this episode, featuring David Kanter of MLCommons, Frederic Van Haren, and Stephen Foskett. MLCommons is focused on making machine learning better for everyone through metrics, datasets, and enablement. The goal for MLPerf is to come up with a fair and representative benchmark to allow the makers of ML systems to demonstrate the performance of their solutions. They focus on real data from a reference ML model that defines correctness, review the performance of a solution, and post the results. MLPerf started with training then added inferencing, which is the focus for users of ML. We must also consider factors like cost and power use when evaluating a system, and a reliable bench

Links:

The quality of an AI application depends on the quality of the data that feeds it. Sunil Samel joins Frederic Van Haren and Stephen Foskett to discuss DataOps and the importance of data quality. When we consider data-centric AI, we must consider all aspects of the data pipeline, from storing, transporting, and understanding to controlling access and cost. We must look at the data needed to train our models, think about the desired outcomes, and consider the sources and pipeline needed to get that result. We must also decide how to define quality: Do we need a variety of data sources? Should we reject some data? How does the modality of the data type change this definition? Is there bias in what is included and excluded? Data pipelines are usually simple, ingesting and storing data from the source, slicing and preparing it, and presenting it for processing. But DataOps recognizes that the data pipeline can get very complicated and requires understanding of all these steps as well as adaptation from development to production.

Three Questions:

• Frederic: Do you think we should expect another AI winter?
• Stephen: When will we see a full self-driving car that can drive anywhere, any time?
• Mike O'Malley, Seneca Global: Can you give an example where an AI algorithm went terribly wrong and gave a result that clearly wasn’t correct?

Gests and Hosts

Machine learning is unlike any other enterprise application, demanding massive datasets  from distributed sources. In this episode, Bin Fan of Alluxio discusses the unique challenges of distributed heterogeneous data to support ML workloads with Frederic Van Haren and Stephen Foskett. The systems supporting AI training are unique, with GPUs and other AI accelerators distributed across multiple machines, each accessing the same massive set of small files. Conventional storage solutions are not equipped to serve parallel access to such a large number of small files, and they often become a bottleneck to performance in machine learning training. Another issue is moving data across silos, storage systems and protocols, which is impossible with most solutions.

Three Questions:

• Frederic: What areas are blocking us today to further improve and accelerate AI?
• Stephen: How big can ML models get? Will today's hundred-billion parameter model look small tomorrow or have we reached the limit?
• Sara E. Berger: With all of the AI that we have in our day-to-day, where should be the limitations? Where should we have it, where shouldn't we have it, where should be the boundaries?

Gests and Hosts