ericjmorey

As the first alpha version of COSMIC Epoch 1, it is incomplete. You’ll most certainly find bugs. Testing and bug reports are welcome and appreciated. New feature requests will be considered for Epoch 2, COSMIC’s second release.

COSMIC Epoch 1 (alpha 1) on the Pop!_OS 24.04 LTS alpha ISO files are available

Try COSMIC on other Linux distributions

Fedora - See instructions

NixOS - See instructions

Arch - See instructions

openSUSE - Coming soon

Serpent OS - See instructions

Redox OS - includes some COSMIC Components - See Progress

https://system76.com/cosmic

Book Preface:

Welcome to Apache Iceberg: The Definitive Guide! We’re delighted you have embarked on this learning journey with us. In this preface, we provide an overview of this book, why we wrote it, and how you can make the most of it.

About This Book

In these pages, you’ll learn what Apache Iceberg is, why it exists, how it works, and how to harness its power. Designed for data engineers, architects, scientists, and analysts working with large datasets across various use cases from BI dashboards to AI/ML, this book explores the core concepts, inner workings, and practical applications of Apache Iceberg. By the time you reach the end, you will have grasped the essentials and possess the practical knowledge to implement Apache Iceberg effectively in your data projects. Whether you are a newcomer or an experienced practitioner, Apache Iceberg: The Definitive Guide will be your trusted companion on this enlightening journey into Apache Iceberg.

Why We Wrote This Book

As we observed the rapid growth and adoption of the Apache Iceberg ecosystem, it became evident that a growing knowledge gap needed to be addressed. Initially, we began by sharing insights through a series of blog posts on the Dremio platform to provide valuable information to the burgeoning Iceberg community. However, it soon became clear that a comprehensive and centralized resource was essential to meet the increasing demand for a definitive Iceberg reference. This realization was the driving force behind the creation of Apache Iceberg: The Definitive Guide. Our goal is to provide readers with a single authoritative source that bridges the knowledge gap and empowers individuals and organizations to make the most of Apache Iceberg’s capabilities in their data-related endeavors.

What You Will Find Inside

In the following chapters, you will learn what Apache Iceberg is and how it works, how you can take advantage of the format with a variety of tools, and best practices to manage the quality and governance of the data in Apache Iceberg tables. Here is a summary of each chapter’s content:

• Chapter 1, “Introduction to Apache Iceberg”
  Exploration of the historical context of data lakehouses and the essential concepts underlying Apache Iceberg.
• Chapter 2, “The Architecture of Apache Iceberg”
  Deep dive into the intricate design of Apache Iceberg, examining how its various components function together.
• Chapter 3, “Lifecycle of Write and Read Queries”
  Examination of the step-by-step process involved in Apache Iceberg transactions, highlighting updates, reads, and time-travel queries.
• Chapter 4, “Optimizing the Performance of Iceberg Tables”
  Discussions on maintaining optimized performance in Apache Iceberg tables through techniques such as compaction and sorting.
• Chapter 5, “Iceberg Catalogs”
  In-depth explanation of the role of Apache Iceberg catalogs, exploring the different catalog options available.
• Chapter 6, “Apache Spark”
  Practical sessions using Apache Spark to manage and interact with Apache Iceberg tables.
• Chapter 7, “Dremio’s SQL Query Engine”
  Exploration of the Dremio lakehouse platform, focusing on DDL, DML, and table optimization for Apache Iceberg tables.
• Chapter 8, “AWS Glue”
  Demonstration of the use of AWS Glue Catalog and AWS Glue Studio for working with Apache Iceberg tables.
• Chapter 9, “Apache Flink”
  Practical exercises in using Apache Flink for streaming data processing with Apache Iceberg tables.
• Chapter 10, “Apache Iceberg in Production”
  Insights into managing data quality in production, using metadata tables for table health monitoring and employing table and catalog versioning for various operational needs.
• Chapter 11, “Streaming with Apache Iceberg”
  Use of tools such as Apache Spark, Flink, and AWS Glue for streaming data processing into Iceberg tables.
• Chapter 12, “Governance and Security”
  Exploration of the application of governance and security at various levels in Apache Iceberg tables, such as storage, semantic layers, and catalogs.
• Chapter 13, “Migrating to Apache Iceberg”
  Guidelines on transforming existing datasets from different file types and databases into Apache Iceberg tables.
• Chapter 14, “Real-World Use Cases of Apache Iceberg”
  A look at real-world applications of Apache Iceberg, including business intelligence dashboards and implementing change data capture

Direct link to PDF

Dremio bills itself as a "Unified Analytics Platform for a Self-Service Lakehouse". The authors of the book work for Dremio and may have ownership interest in Dremio.

What issues or frustrations have you encountered in trying to use and set up Neovim in Windows 11?

I'm currently writing up my experience with installing, setting up, and using Neovim in Windows and would like to hear from others that have tried the same. What was annoying, difficult, or impossible in your experience?

Many devs dream of one day writing their own operating system. Ideally in their favorite language: Rust. For many of us, this dream remains just that: a dream.

Jeremy Soller from System76, however, didn't just contribute kernel code for Pop!_OS, but also started his own operating system, RedoxOS, which is completely written in Rust. One might get the impression that he likes to tinker with low-level code!

In this episode of Rust in Production, Jeremy talks about his journey. From getting hired as a kernel developer at Denver-based company System76 after looking at the job ad for 1 month and finally applying, to being the maintainer of not one but two operating systems, additional system tools, and the Rust-based Cosmic desktop. We'll talk about why it's hard to write correct C code even for exceptional developers like Jeremy and why Rust is so great for refactoring and sharing code across different levels of abstraction.

Listen to Rust in Production Podcast S02 E07

Many devs dream of one day writing their own operating system. Ideally in their favorite language: Rust. For many of us, this dream remains just that: a dream.

Jeremy Soller from System76, however, didn't just contribute kernel code for Pop!_OS, but also started his own operating system, RedoxOS, which is completely written in Rust. One might get the impression that he likes to tinker with low-level code!

In this episode of Rust in Production, Jeremy talks about his journey. From getting hired as a kernel developer at Denver-based company System76 after looking at the job ad for 1 month and finally applying, to being the maintainer of not one but two operating systems, additional system tools, and the Rust-based Cosmic desktop. We'll talk about why it's hard to write correct C code even for exceptional developers like Jeremy and why Rust is so great for refactoring and sharing code across different levels of abstraction.

Listen to Rust in Production Podcast S02 E07

Many devs dream of one day writing their own operating system. Ideally in their favorite language: Rust. For many of us, this dream remains just that: a dream.

Jeremy Soller from System76, however, didn't just contribute kernel code for Pop!_OS, but also started his own operating system, RedoxOS, which is completely written in Rust. One might get the impression that he likes to tinker with low-level code!

In this episode of Rust in Production, Jeremy talks about his journey. From getting hired as a kernel developer at Denver-based company System76 after looking at the job ad for 1 month and finally applying, to being the maintainer of not one but two operating systems, additional system tools, and the Rust-based Cosmic desktop. We'll talk about why it's hard to write correct C code even for exceptional developers like Jeremy and why Rust is so great for refactoring and sharing code across different levels of abstraction.

Listen to Rust in Production Podcast S02 E07

Based on answers to the following question:

Which development environments did you use regularly over the past year, and which do you want to work with over the next year? Please check all that apply.

Neovim is the most admired code editor in the 2024 Stacked Overflow Developer Survey

Source: https://survey.stackoverflow.co/2024/technology#admired-and-desired-new-collab-tools-desire-admire

It's broader than a Neovim specific mapping, I've changed the system keyboard mapping of <Caps Lock> to <Esc> and <F9> to <Caps Lock>.

I think mapping <Caps Lock> to <Esc> isn't uncommon for Neovim users. But I like having <Caps Lock> available for non Neovim purposes.

I asked some LLM chatbots to give me some silly ideas to try. Below are a few of my favorite responses.

From Perplexity.ai

Six Degrees of Wikipedia: Creating a program that finds the shortest path between two random Wikipedia articles using graph traversal algorithms. This applies graph theory concepts to explore connections in a large knowledge base.

Emoji Encryption: Using hash tables and cryptographic algorithms to create an encryption system that converts text to emojis. This could be an interesting way to explore cryptography concepts in a fun, visual way.

From Gemini.google.com

Procrastination Station: This website creates increasingly elaborate and ridiculous tasks to distract you from what you actually need to do. Dishes? Nah, fold your socks into origami cranes!

Dramatic Password Validator: Forget boring error messages. This program rejects weak passwords with Shakespearean insults or movie villain monologues.

From Chatgpt.com

3. Time Travel Email Service: Build a data structure that allows you to send emails to yourself in the past, with time complexity considerations that are totally ignored because it’s time travel.

4. Mood-Driven Random Number Generator: Implement an algorithm that generates random numbers based on the mood of the user, using sentiment analysis on real-time facial expressions.

Why did you decide to work with Odin?

What did you build with Odin?

Where does Odin shine? What could use some polish?

Karl Zylinski writes:

This article is an introduction to the Odin Programming Language. It is aimed at people who know a bit of programming, but have never touched Odin. It is not a reference guide, rather I try to keep things informal and talk about what I think are important aspects of the language. There will be some notes on differences to C/C++, as Odin in many ways tries to be better C. If you enjoy this article and want to support me, then you can do so by becoming a patron.

In the recent years most of my programming has been done using the Odin Programming Language, sometimes referred to as Odinlang. Since a year back I create my own video games using Odin. You can find my game CAT & ONION on Steam, it is completely created in Odin.

...

While Odin is a great language, it is currently quite lacking in documentation. There is lots of documentation, but it is scattered and incomplete. I tried to address that by writing this article, which I hope gives a holistic introduction to the language.

Read Introduction to the Odin Programming Language

July 18, 2024 Narek Galstyan writes:

We were naturally curious when we saw Pinecone's blog post comparing Postgres and Pinecone.

In their post on Postgres, Pinecone recognizes that Postgres is easy to start with as a vector database, since most developers are familiar with it. However, they argue that Postgres falls short in terms of quality. They describe issues with index size predictability, index creation resource intensity, metadata filtering performance, and cost.

This is a response to Pinecone's blog post, where we show that Postgres outperforms Pinecone in the same benchmarks with a few additional tweaks. We show that with just 20 lines of additional code, Postgres with the pgvector or lantern extension outperforms Pinecone by reaching 90% recall (compared to Pinecone's 60%) with under 200ms p95 latency.

Read Postgres vs. Pinecone