ericjmorey

cross-posted from: https://programming.dev/post/16349359

July 2, 2024

Sylvain Kerkour writes:

Rust adoption is stagnating not because it's missing some feature pushed by programming language theory enthusiasts, but because of a lack of focus on solving the practical problems that developers are facing every day.

... no company outside of AWS is making SDKs for Rust ... it has no official HTTP library.

As a result of Rust's lack of official packages, even its core infrastructure components need to import hundreds of third-party crates.

• cargo imports over 400 crates.

• crates.io has over 500 transitive dependencies.

...the offical libsignal (from the Signal messaging app) uses 500 third-party packages.

... what is really inside these packages. It has been found last month that among the 999 most popular packages on crates.io, the content of around 20% of these doesn't even match the content of their Git repository.

...how I would do it (there may be better ways):

A stdx (for std eXtended) under the rust-lang organization containing the most-needed packages. ... to make it secure: all packages in stdx can only import packages from std or stdx. No third-party imports. No supply-chain risks.

[stdx packages to include, among others]:

gzip, hex, http, json, net, rand

Read Rust has a HUGE supply chain security problem

Submitter's note:

I find the author's writing style immature, sensationalist, and tiresome, but they raise a number of what appear to be solid points, some of which are highlighted above.

July 2, 2024

Sylvain Kerkour writes:

Rust adoption is stagnating not because it's missing some feature pushed by programming language theory enthusiasts, but because of a lack of focus on solving the practical problems that developers are facing every day.

... no company outside of AWS is making SDKs for Rust ... it has no official HTTP library.

As a result of Rust's lack of official packages, even its core infrastructure components need to import hundreds of third-party crates.

• cargo imports over 400 crates.

• crates.io has over 500 transitive dependencies.

...the offical libsignal (from the Signal messaging app) uses 500 third-party packages.

... what is really inside these packages. It has been found last month that among the 999 most popular packages on crates.io, the content of around 20% of these doesn't even match the content of their Git repository.

...how I would do it (there may be better ways):

A stdx (for std eXtended) under the rust-lang organization containing the most-needed packages. ... to make it secure: all packages in stdx can only import packages from std or stdx. No third-party imports. No supply-chain risks.

[stdx packages to include, among others]:

gzip, hex, http, json, net, rand

Read Rust has a HUGE supply chain security problem

Submitter's note:

I find the author's writing style immature, sensationalist, and tiresome, but they raise a number of what appear to be solid points, some of which are highlighted above.

Jul 1, 2024

Aman Salykov writes:

This blog post is the result of my attempt to implement high-performance matrix multiplication on CPU while keeping the code simple, portable and scalable. The implementation follows the BLIS design, works for arbitrary matrix sizes, and, when fine-tuned for an AMD Ryzen 7700 (8 cores), outperforms NumPy (=OpenBLAS), achieving over 1 TFLOPS of peak performance across a wide range of matrix sizes.

By efficiently parallelizing the code with just 3 lines of OpenMP directives, it’s both scalable and easy to understand. The implementation hasn’t been tested on other CPUs, so I would appreciate feedback on its performance on your hardware. Although the code is portable and targets Intel Core and AMD Zen CPUs with FMA3 and AVX instructions (i.e., all modern Intel Core and AMD Zen CPUs), please don’t expect peak performance without fine-tuning the hyperparameters, such as the number of threads, kernel, and block sizes, unless you are running it on a Ryzen 7700(X). Additionally, on some Intel CPUs, the OpenBLAS implementation might be notably faster due to AVX-512 instructions, which were intentionally omitted here to support a broader range of processors. Throughout this tutorial, we’ll implement matrix multiplication from scratch, learning how to optimize and parallelize C code using matrix multiplication as an example.

Read Beating NumPy's matrix multiplication in 150 lines of C code

Video description:

We reproduce the GPT-2 (124M) from scratch.

This video covers the whole process:

First we build the GPT-2 network, then we optimize its training to be really fast, then we set up the training run following the GPT-2 and GPT-3 paper and their hyperparameters, then we hit run, and come back the next morning to see our results, and enjoy some amusing model generations.

Keep in mind that in some places this video builds on the knowledge from earlier videos in the Zero to Hero Playlist (see my channel). You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.

The presenter compares some of the functionality to Leap.nvim

Trey Hunner writes:

This article is primarily meant to act as a Python time complexity cheat sheet for those who already understand what time complexity is and how the time complexity of an operation might affect your code. For a more thorough explanation of time complexity see Ned Batchelder's article/talk on this subject.

Read Python Big O: the time complexities of different data structures in Python

In the command-line window the command line can be edited just like editing text in any window. It is a special kind of window, because you cannot leave it in a normal way.

There are two ways to open the command-line window:

1. From Command-line mode, use the key specified with the 'cedit' option (default CTRL-F).

2. From Normal mode, use the "q:", "q/" or "q?" command.

• This starts editing an Ex command-line ("q:") or search string ("q/" or "q?"). Note that this is not possible while recording is in progress (the "q" stops recording then).

When the window opens it is filled with the command-line history. The last line contains the command as typed so far. The left column will show a character that indicates the type of command-line being edited

Pedro Domingos summarizes 12 key lessons that machine learning researchers and practitioners have learned. These include pitfalls to avoid, important issues to focus on, and answers to common questions.

• 1. Learning = Representation + Evaluation + Optimization
• 2. It's Generalization that Counts
• 3. Data Alone Is Not Enough
• 4. Overfitting Has Many Faces
• 5. Intuition Fails in High Dimensions
• 6. Theoretical Guarantees Are Not What They Seem
• 7. Feature Engineering Is The Key
• 8. More Data Beats a Cleverer Algorithm
• 9. Learn Many Models, Not Just One
• 10. Simplicity Does Not Imply Accuracy
• 11. Representable Does Not Imply Learnable
• 12. Correlation Does Not Imply Causation