This article will describe how to download an image from a (docker) container registry.

Manual Download of Container Images with wget and curl

Intro

Remember the good `'ol days when you could just download software by visiting a website and click "download"?

Even apt and yum repositories were just simple HTTP servers that you could just curl (or wget) from. Using the package manager was, of course, more secure and convenient -- but you could always just download packages manually, if you wanted.

But have you ever tried to curl an image from a container registry, such as docker? Well friends, I have tried. And I have the scars to prove it.

It was a remarkably complex process that took me weeks to figure-out. Lucky you, this article will break it down.

Examples

Specifically, we'll look at how to download files from two OCI registries.

1. Docker Hub
2. GitHub Packages

Terms

First, here's some terminology used by OCI

1. OCI - Open Container Initiative
2. blob - A "blob" in the OCI spec just means a file
3. manifest - A "manifest" in the OCI spec means a list of files

Prerequisites

This guide was written in 2024, and it uses the following software and versions:

1. debian 12 (bookworm)
2. curl 7.88.1
3. OCI Distribution Spec v1.1.0 (which, unintuitively, uses the '/v2/' endpoint)

Of course, you'll need 'curl' installed. And, to parse json, 'jq' too.

sudo apt-get install curl jq

What is OCI?

OCI stands for Open Container Initiative.

OCI was originally formed in June 2015 for Docker and CoreOS. Today it's a wider, general-purpose (and annoyingly complex) way that many projects host files (that are extremely non-trivial to download).

One does not simply download a file from an OCI-complianet container registry. You must:

1. Generate an authentication token for the API
2. Make an API call to the registry, requesting to download a JSON "Manifest"
3. Parse the JSON Manifest to figure out the hash of the file that you want
4. Determine the download URL from the hash
5. Download the file (which might actually be many distinct file "layers")

One does not simply download from a container registry

In order to figure out how to make an API call to the registry, you must first read (and understand) the OCI specs here.

• https://opencontainers.org/release-notices/overview/

OCI APIs

OCI maintains three distinct specifications:

1. image spec
2. runtime spec
3. distribution spec

OCI "Distribution Spec" API

To figure out how to download a file from a container registry, we're interested in the "distribution spec". At the time of writing, the latest "distribution spec" can be downloaded here:

• https://github.com/opencontainers/distribution-spec/releases/tag/v1.1.0
• https://github.com/opencontainers/distribution-spec/releases/download/v1.1.0/oci-distribution-spec-v1.1.0.pdf

The above PDF file defines a set of API endpoints that we can use to query, parse, and then figure out how to download a file from a container registry. The table from the above PDF is copied below:

ID	Method	API Endpoint	Success	Failure
end-1	GET	/v2/	200	404/401
end-2	GET / HEAD	/v2/<name>/blobs/<digest>	200	404
end-3	GET / HEAD	/v2/<name>/manifests/<reference>	200	404
end-4a	POST	/v2/<name>/blobs/uploads/	202	404
end-4b	POST	/v2/<name>/blobs/uploads/?digest=<digest>	201/202	404/400
end-5	PATCH	/v2/<name>/blobs/uploads/<reference>	202	404/416
end-6	PUT	/v2/<name>/blobs/uploads/<reference>?digest=<digest>	201	404/400
end-7	PUT	/v2/<name>/manifests/<reference>	201	404
end-8a	GET	/v2/<name>/tags/list	200	404
end-8b	GET	/v2/<name>/tags/list?n=<integer>&last=<integer>	200	404
end-9	DELETE	/v2/<name>/manifests/<reference>	202	404/400/405
end-10	DELETE	/v2/<name>/blobs/<digest>	202	404/405
end-11	POST	/v2/<name>/blobs/uploads/?mount=<digest>&from=<other_name>	201	404
end-12a	GET	/v2/<name>/referrers/<digest>	200	404/400
end-12b	GET	/v2/<name>/referrers/<digest>?artifactType=<artifactType>	200	404/400
end-13	GET	/v2/<name>/blobs/uploads/<reference>	204	404

In OCI, files are (cryptically) called "blobs". In order to figure out the file that we want to download, we must first reference the list of files (called a "manifest").

The above table shows us how we can download a list of files (manifest) and then download the actual file (blob).

Examples

Let's look at how to download files from a couple different OCI registries:

1. Docker Hub
2. GitHub Packages

Docker Hub

To see the full example of downloading images from docker hub, click here

GitHub Packages

To see the full example of downloading files from GitHub Packages, click here.

Why?

I wrote this article because many, many folks have inquired about how to manually download files from OCI registries on the Internet, but their simple queries are usually returned with a barrage of useless counter-questions: why the heck would you want to do that!?!

The answer is varied.

Some people need to get files onto a restricted environment. Either their org doesn't grant them permission to install software on the machine, or the system has firewall-restricted internet access -- or doesn't have internet access at all.

3TOFU

Personally, the reason that I wanted to be able to download files from an OCI registry was for 3TOFU.

Verifying Unsigned Releases with 3TOFU

Unfortunaetly, most apps using OCI registries are extremely insecure. Docker, for example, will happily download malicious images. By default, it doesn't do any authenticity verifications on the payloads it downloaded. Even if you manually enable DCT, there's loads of pending issues with it.

Likewise, the macOS package manager brew has this same problem: it will happily download and install malicious code, because it doesn't use cryptography to verify the authenticity of anything that it downloads. This introduces watering hole vulnerabilities when developers use brew to install dependencies in their CI pipelines.

My solution to this? 3TOFU. And that requires me to be able to download the file (for verification) on three distinct linux VMs using curl or wget.

⚠ NOTE: 3TOFU is an approach to harm reduction.

It is not wise to download and run binaries or code whose authenticity you cannot verify using a cryptographic signature from a key stored offline. However, sometimes we cannot avoid it. If you're going to proceed with running untrusted code, then following a 3TOFU procedure may reduce your risk, but it's better to avoid running unauthenticated code if at all possible.

Registry (ab)use

Container registries were created in 2013 to provide a clever & complex solution to a problem: how to package and serve multiple versions of simplified sources to various consumers spanning multiple operating systems and architectures -- while also packaging them into small, discrete "layers".

However, if your project is just serving simple files, then the only thing gained by uploading them to a complex system like a container registry is headaches. Why do developers do this?

In the case of brew, their free hosing provider (JFrog's Bintray) shutdown in 2021. Brew was already hosting their code on GitHub, so I guess someone looked at "GitHub Packages" and figured it was a good (read: free) replacement.

Many developers using Container Registries don't need the complexity, but -- well -- they're just using it as a free place for their FOSS project to store some files, man.

https://news.itsfoss.com/crowdstrike-windows-linux/

#infosec

Frage zur Nextcloud APP am Smartphone (Lineage/Android/F-Driod)
Wie kann man Dateien aus dem Nextcloud-Pfad des Smartphones in die Dateistruktur des Smartphones verschieben? Und das so komfortabel, dass auch Menschen mit wenig Lust auf technische Dinge damit zurechtkommen?

Dass ich Dateien mittels einer Dateimanager-APP von SD-Karte > Android > Media > com.nextcloud.client > nextcloud > user@mycloud.com > MeinPfad manuell nach SD-Karte > MyData verschieben kann, ist jedem halbwegs technisch Interessiertem vermutlich klar. Aber auf Dauer sicherlich schon nervig genug. Und für einen normalen, vielleicht schon älteren Handynutzer, ist das in den meisten Fällen wohl viel zu viel.

Innerhalb der Netxcloud-App habe ich zwar Optionen zum Verschieben gefunden, aber die beziehen sich, wie es aussieht, nur auf den Nextcloud-Space. Nach außerhalb ins Dateisystem des Smartphones kann ich keine Dateien kopieren.

Weiß da jemand von euch etwas? Tipps?

@linux #linux #smartphone #nextcloud #nextcloudapp #Android

Do you have an AMD aura GPU? Do you also use Linux? There's this this driver that needs to be tested.

It allows you to control the lighting of the GPU using programs like openRGB.

I wrote that PR that should make it work for more GPUs, but I only have an RX 480 so I can only test that one. It would be useful to try it on a Vega gpu.

If you have an rDNA 1/2/3 GPU, it most likely won't work, but without the card there's nothing I can do.

On a side note, if you are interested in maintaining the driver it would be great.

Welche Fediverse-Plattform für die ganz Unerfahrenen? Ich bin von einem neugierigen älteren Herren (~70) gefragt worden, ob ich ihm die ersten Schritte ins Fediverse zeigen kann. Mit dem Computer kann er im Grunde umgehen, allerdings eher auf der Word-und-Surf-Ebene. Social-Media ist für ihn noch Fremdland.

Welche der Plattformen (friendica, mastodon, peertube, pixelfeed, hubzuilla, und wie sie alle heißen ...) sind für Menschen ohne fediverse Erfahrungen geeignet und gut zu erfassen? Habt ihr Tipps / Erfahrungen?

#fediverse #FragDasFediverse #Friendica #peertube #Pixelfeed #streams #calckey #firefish #IceShrimp @linux #Hubzilla #social #socialmedia #SocialMedia-Forschung #Mastodon #twitter #rentner #Usabiity

Detailed issue

Basically Kwin and other programs (simple xdg-desktop-portal or even gimp) crash and they bork the entire screen with no recovery other than rebooting. When the program that crash is Kwin it's particularly bad because it happens at login.

Take note of the quote in the article...

---

OP/bug finder here with some clarifying information. It's a common misconception that this issue can only be abused if you use HTTP boot. That is not the case at all, otherwise it wouldn't be Critical. This bug can be abused locally (privileged malware can overwrite the EFI partition), from an adjacent network if PXE boot is enabled (w/ MiTM), or remotely if HTTP boot is used (w/ MiTM).

More details on these scenarios:

1. A remote attacker with no privileges in a man-in-the-middle (MitM) position could leverage the issue against a victim machine that uses HTTP boot. No direct access to the victim machine is required.

2. A remote attacker with privileges and code execution on the victim machine could leverage the issue to bypass Secure Boot, even if the victim does not already use HTTP boot (as long as firmware has HTTP support). How? Several ways:

• An attacker can edit the boot order variable to specify a controlled attacker server.

• An attacker can chain shim->GRUB2->shim (via HTTP). For this technique, an attacker would overwrite the boot loader in the EFI partition to a legitimate shim and GRUB2 image. The attacker would create a grub.cfg that chainloads a new shim via HTTP. This is possible because GRUB2's device syntax allows you to specify any supported device, including HTTP (if available).

3. An adjacent attacker with no privileges in a man-in-the-middle (MitM) position could leverage the issue against a victim machine that uses PXE boot. PXE is separate from HTTP boot, but similar to the local vector, an attacker can chain together shim (via PXE)->GRUB2 (via PXE)->shim (via HTTP).

cross-posted from: https://floss.social/users/kde/statuses/111732458987994100

Plasma 6 Release Candidate 1 has landed.

We are less than 50 days away from the final version of #Plasma6.

Along with Frameworks 6 and KDE Gear 24.02, the Megarelaease on the 28th of February will be one of the biggest and more complex upgrades in KDE's history.

One more RC will be released on the 31st of January and then it will be (hopefully) clear sailing until the final release.

https://kde.org/announcements/megarelease/6/rc1/

@kde@lemmy.kde.social

Hello everyone,

I just joined the Linux community on Lemmy.

Love Gnu/ Linux. Would love to create my own custom Linux servers and machines one day, and I'm already on that path.

(Maybe only a quarter of the way though. 😁)

I don't know what else to say, other than that the goals and admirations and aspirations of Gnu/ Linux fall in line with my personal endeavours in creating a collaborative and supportive environment that drives creativity, freedom, and independence.

This initiative comes from the beautiful but encouraging and supportive community that I had the chance to speak to and talk to (in matrix), which focused on the social values of providing free and opensource resources to people around the world - which is unfortunately neglected by many, if not exploited and abused, and not mentioned, around the world (or at least the society that I have been in contact with in my life, including in the academic aspect and the organisational aspect).

Of course, I also listened to Linus and Richard stallman respectively (I do not endorse any of RMS's political, religious, or ideological beliefs).

The building blocks upon which helped create Gnu/ Linux what it is today. The ankle point that holds many organisations and companies around the world. The hard working and faithful people that strived in creating values in all of their forms to support such initiatives.

And it reminds me of a beautiful qoute, said by [childhood] friend, Mr. Fred Rogers:

"L'essentiel est invisible pour les yeux".

"What is essential is invisible to the eyes".

Originally written by good sir story writer: Antoine de Saint-Exupéry. (Recommend reading chapter 10).

My nickname here (or username) does not reflect my real name, but I do find joy in nicknames too. You could call me Engelbert, for casually-endorsing-social-conversations' - sake'.

And I am here to learn, and exchange knowledge and experiences with others too ! Help and give advice, and seek out challenging issues the community may be facing.

Talk to me about anything. Religion, culture, history, politics, some good philosophy, but mostly Gnu/ Linux, Rust, design philosophy !

(If you feel that the subject is too sensitive, we may have a private council together 😀).

My goal is, as a gesture of goodwill and thanksgiving, to give back to society and give back to people who helped shape and create this society the way it is today, all things regarding Gnu/ Linux and the opensource objectives.

I have read more than 5 books in rust, and I'm still nowhere near perfect. I had more than 50+ projects in rust (small to medium, nothing crazy), had experience in C, C++, and C#.

On final note, I would like to mention these beautiful words by imam Ali: "Hide the good you do, and make known the good done to you !".

---

Thank you for having me !

Sir Engelbert.

(

"Curl 8.4.0 will hit at around 0600 UTC (0800 CEST, 0700 BST, 0200 EST, 2300 PDT) on October 11 and deal with CVE-2023-38545, which affects both libcurl and the curl tool, and CVE-2023-38546, which only affects libcurl...."

"Remote code execution requiring no authentication fixed. 2 other RCEs remain unpatched...."

"On September 28, 2023, the Snap Store team was notified of a potential security incident. A number of snap users reported several recently published and potentially malicious snaps...."

Like choose your-own-ending novels and BASH? What would you do if you suddenly had ssh access to someone else's server? This is a really fun corner of the net. Excellent resource/trainer/time-waster. When I talk to people from the DIY linux/selfhosting/FOSS communities, many folks haven't heard of it.

What other cool stuff do you know about? Think it sucks and want to tell me why?

Note: I am not at all associated with this project - just think it's a valuable stop on the good 'ole internet.

At the moment there is slim pickings for other communities to join if this instance is chosen by a user to be their home server.

Presumably other instances have to approve linux.community federating with them before their subs can show up? Is it that this instance is new, or is there some other reason. u/nkukard can you chime in?

EDIT: A-ha! I was searching poorly! Once a member of a instance subscribes, it pulls it into that instance for easier discovery. Neat!

A comprehensive mapping of old subreddits to new communities.