I was recently approached by a couple of local flippers.

The first offered me three sealed AMD Radeon Pro Vega II modules for $915 USD each.

The second offered me four sealed AMD Radeon Pro Vega II Duo modules for $1,200 USD each.

Neither offer includes the Infinity Fabric Link Bridge, of course. I also came to learn that, unlike the W6800X Duo and W6900X, the Vega II did not ship with the Infinity Fabric Link Bridge in the box. Only the Infinity Fabric Link Jumper shipped with the Duo model.

Still, I have to admit, I am impressed that sealed Vega II and Vega II Duo modules are still showing up on the market in 2026.

At first, the prices felt like a catch. But then I had to stop and seriously think about whether it was actually worth the hassle to invest in them for my use case.

I took a ride with my AI of choice and discussed it. The conclusion was pretty practical:

For a local AI workload, especially if the goal is raw usefulness, software support, and flexibility, it suggested going with four NVIDIA RTX 3090s instead of four Vega II Duos.

And honestly, that makes sense, even though the NVIDIA RTX 3090 is almost as old as the Vega II GPUs.

But at the same time, I am having so much fun with this whole “local AI on a discontinued Mac Pro” journey that part of me is still tempted.

There is something really interesting about pushing the Mac Pro 2019 / MacPro7,1 as far as it can go, even if the more logical path is to move to standard PC GPUs with better AI support.

So now I am genuinely curious:

What would you do?

Would you buy the sealed Vega II / Vega II Duo modules and keep exploring the Mac Pro 2019 rabbit hole?

Would you skip the Apple MPX route entirely and put the money toward a multi-GPU RTX 3090 setup instead?

Or would you shoot for a Mac Studio for the same budget?

The logical answer seems obvious, but the fun answer is not always the logical one.

---

Disclaimer: I wrote this post myself. I also used AI as a tool to help clean up the wording and formatting.

Why AI? And why go local?

Many of us have used some form of AI by now, and most of us have seen the power and convenience it can provide.

Back in 2015, I wrote a simple Visual Basic program in Visual Studio. It took me almost a full week to complete, including online searches, trial and error, and asking questions on Stack Overflow.

In 2025, with the help of ChatGPT, I completed a piece of iOS software in just a few hours, despite having never coded for iOS before.

In the right hands, AI is a very powerful tool.

I want access to that tool regardless of the circumstances I am in. I do not want my access to depend entirely on subscriptions, cloud availability, internet access, changing policies, rate limits, or someone else deciding what I can and cannot run.

That is where local AI comes in.

---

The Hardware

I was fortunate enough to receive a Mac Pro 2019 back in 2020.

In 2023, while searching for SSDs for it, I stumbled across a seller offering brand new, sealed AMD Radeon Pro W6800X Duo and W6900X MPX modules for about 75% of today’s used market price.

I bought one mainly for the Thunderbolt ports. I also bought three more with other goals in mind.

In 2024, with the release of Llama 3, and later with ROCm becoming a more serious path for my hardware, I committed to using these GPUs for local AI. The problem was that my Mac Pro was still my main desktop, and I could not repurpose it entirely for AI work.

So I ended up investing about $3,000 USD into two additional machines, including shipping and tax:

• Two Mac Pro 2019 / MacPro7,1 towers
• 16-core Xeon CPUs
• 96 GB RAM each
• SSDs that I later upgraded to 8 TB
• Standard Radeon Pro 580X MPX GPUs

I then installed the MPX GPUs into each machine.

The first machine is **LinuxAI-64**, with dual AMD Radeon Pro W6900X GPUs.

The second machine is **LinuxAI-128**, with dual AMD Radeon Pro W6800X Duo GPUs.

---

The Software

Because ROCm mainly supports Ubuntu and RHEL-based distributions, I chose Ubuntu Server 22.04 LTS.

I chose Ubuntu because I had prior experience with it. I chose the Server variant to minimize non-AI GPU load. I chose 22.04 LTS because it was the latest ROCm-supported Ubuntu version at the time.

Then came the frameworks. I was mainly considering:

• Ollama
• llama.cpp
• vLLM

I initially wanted to go with vLLM, but I ended up using Ollama because vLLM does not support my hardware.

Then came the next layers:

• Web UI / GUI
• Document support
• RAG
• Agentic frameworks
• Multi-agent workflows
• OpenClaw, Hermes, and other possible stacks

And then reality hit:

That is a lot of work for a local AI setup that may still be only half as capable as cloud AI today.

I still went for it though.

---

What Is the Goal?

I love the idea of multi-agent workloads.

In practical terms, I want AI agents that can help act as:

• A secretary
• A tutor for my kids
• A chief of operations

And honestly, I want to see how far I can push the idea of building something like a 20–30 person company using local AI agents as the supporting workforce, running on my own hardware, and using only the investment I have already made.

If it works, I get what I wanted.

If it does not, I gain the experience.

Either way, I learn.

---

The Path

I need a community around me.

Community helps me work better, stay motivated, and increase my productivity.

Online, that means communities like:

• MacPro
• Linux communities
• T2 Linux
• MacLLM
• ROCm
• LocalLLM
• LocalLLaMA
• vLLM
• OpenClaw
• And now, r/MacPro2019LocalAI

There are many others too, both on Reddit and elsewhere.

Locally, I am based in Riyadh, the capital of Saudi Arabia. I have a very small local circle interested in local AI, but I would love to find more people nearby who are in the same boat: same hardware, same goals, same interest in pushing local AI forward.

Someone like that would help motivate me, challenge me, and push me to improve.

---

What Next?

For the purpose of growing this community, and documenting the work I have put into these machines, I plan to release a series of guides over the next few weeks.

The goal is to document success after success as I work toward my final local AI setup.

The general idea is simple:

The more that is documented, the easier it becomes for others to follow the same path, join the discussion, brainstorm solutions, and hopefully even pass me and innovate beyond what I have done.

That would benefit the community as a whole, and it would also help me on my own journey.

I am interested to know what you think is important to cover.

I am also interested in any solution, framework, stack, or workflow that you think is worth testing.

Consider this a brainstorming session.

I have already written some starter guides, and I plan to update and fine-tune them first. After that, I want to go deeper into frameworks, especially vLLM. I also plan to work on OpenClaw and document my experience with it.

I am a big believer in copy-paste instructions. I format and rebuild systems from scratch often, and having clean, repeatable instructions makes the recovery period much faster.

Let me know what you think, what I should look into, and what topics would be most useful as I work toward the final goal.

---

TL;DR

• I want to build a local AI community around me.
• I plan to write several local AI guides during the coming weeks.
• I want your help brainstorming topics, tools, frameworks, and problems worth solving and documenting.

---

^(Disclaimer: I wrote this post myself. I also used AI as a tool to help clean up the wording and formatting.)

>"For some, Linux fails to boot, for some it's okish, for some it's good"

u/AdityaGarg8 said that to me back in November 2024, when he was kind enough to help guide me through installing Ubuntu on my Mac Pro 2019.

After a lot of testing, I think that quote perfectly describes the current state of Linux on the Mac Pro 2019, especially when using Apple’s MPX AMD GPUs with the Infinity Fabric Link jumper or bridge installed.

What seems to be happening?

From my testing, the main issue appears to involve the Infinity Fabric Link jumper/bridge.

On newer kernels, especially kernel 6.8 and later, some GPUs with the Infinity Fabric Link installed do not initialize correctly. In my case, this has shown up as amdgpu initialization failures and psp -22 errors.

On kernel 5.15.0, the GPUs initialize more successfully, but I still see errors, especially SDMA-related errors. So I would describe 5.15.0 as partial support, not full support.

So far, my practical summary is:

• Kernel 5.15.0: GPUs can initialize, but support appears incomplete.
• Kernel 6.8: GPUs may fail to initialize when Infinity Fabric Link is installed.
• Later kernels, including 6.17 and 7.0: in my testing, one GPU may initialize correctly, while the remaining GPUs fail with psp -22.

This is not meant to be a final technical diagnosis. It is a report of what I and others are seeing on real Mac Pro 2019 hardware.

Does Infinity Fabric Link matter?

For local AI, the most important factors are usually:

• GPU compute
• VRAM capacity
• Memory bandwidth
• Inter-GPU bandwidth

On multi-GPU setups, VRAM is not automatically pooled into one shared memory space. Each GPU has its own VRAM, and when a workload is split across multiple GPUs, the GPUs need to communicate with each other.

Without a direct GPU-to-GPU interconnect, the normal path is usually something like:

GPU0 -> CPU / PCIe -> GPU1

That means traffic has to go through the PCIe path, with the CPU/platform sitting in the middle.

The AMD MPX GPUs in the Mac Pro 2019 are based on PCIe 4.0-capable GPUs, but the Mac Pro 2019 platform itself provides PCIe 3.0 bandwidth. A PCIe 3.0 x16 link has a theoretical maximum of about 15.75 GB/s per direction.

This is where Infinity Fabric Link becomes interesting.

Why Infinity Fabric Link could matter

With proper support, Infinity Fabric Link should allow direct GPU-to-GPU communication:

GPU0 -> GPU1

That removes the normal CPU/PCIe middle step for supported GPU-to-GPU traffic.

Apple rates the Infinity Fabric Link connection at up to 84 GB/s in each direction. That is more than five times the theoretical one-direction bandwidth of PCIe 3.0 x16.

In theory, that could be a major advantage for multi-GPU workloads, especially workloads where GPUs need to exchange data frequently.

For local AI, this could matter most in cases like:

• tensor-parallel inference
• large models split across multiple GPUs
• concurrent inference with many users
• workloads where inter-GPU communication becomes a bottleneck

But does it actually work on Linux?

My current answer is:

Not reliably, at least not on the W6800X Duo and W6900X in my testing.

Some users have reported better results with Vega II / Vega II Duo, and it is possible that older MPX GPUs behave differently. But with the W6800X Duo and W6900X, I do not currently see clean, reliable Infinity Fabric Link behavior under Linux.

To be clear, I am not saying Linux has no AMD GPU support. The GPUs themselves can work under Linux. The issue appears to be specifically around the Infinity Fabric Link Jumper/Bridge with the MPX GPU implementation; firmware/PSP initialization and how the AMDGPU driver handles this hardware combination.

What am I testing now?

Personally, I am experimenting with:

• Ubuntu Server 22.04 LTS
• Kernel 5.15.0
• W6800X Duo and W6900X MPX GPUs
• Infinity Fabric Link jumper/bridge installed

The goal is to see how far this partial support can go, whether the link actually becomes active, and whether there is any measurable bandwidth advantage when it does.

I am also watching newer stacks such as:

• Ubuntu Server 24.04 LTS / kernel 6.17
• Ubuntu Server 26.04 LTS / kernel 7.0

Hopefully, proper support or a workaround appears for these newer kernels.

Community tracking / bug report

There is already activity on the DRM AMD GitLab here:

https://gitlab.freedesktop.org/drm/amd/-/work_items/3793

If you have a Mac Pro 2019 with MPX GPUs, especially Vega II, Vega II Duo, W6800X, W6800X Duo, or W6900X, please consider sharing your results there.

Useful information would include:

• Mac Pro 2019 configuration
• GPU model or models
• Whether the Infinity Fabric Link jumper/bridge is installed
• Linux distro
• Kernel version
• ROCm version, if applicable
• Whether the GPUs initialize
• Relevant dmesg / journalctl errors
• Whether removing the jumper/bridge changes behavior

What can you do to help?

Share your experience.

What hardware do you have?
What OS and kernel are you using?
Does the system boot?
Do all GPUs initialize?
Does removing the Infinity Fabric Link jumper or bridge change anything?
Have you found a kernel version where it works better?

Hopefully, with more of us testing, reporting, and giving this issue attention, we can help establish better Linux support for these powerful MPX GPUs on the Mac Pro 2019.

Disclaimer: I wrote this post myself, but used AI to help clean up the wording and formatting.

Resources:

• DRM AMD Gitlab
• Coelacanth's Dream: Research
• nbritton's Lab (Vega II Implimentation)
• Enable XGMI Support for Sienna Cichlid
• Enable XGMI support for Arcturus

Hi !

I have an opportunity to get a 2019 Mac Pro with a 32GB Vega2 board. I know LM Studio isn't supported on MacOS for Intel, so what other way to turn this machine into a local AI server would you recommend ?

Windows isn't an option, Linux could be possible, though I'd like to keep it running Mac OS if possible. Main objective is to run MCP servers alongside on this dedicated machine.

What kind of speed could I get from the Vega II chip ? (similar to Mi50 32GB from what I could find)

Thanks !

Hey everyone! I’m u/Faisal_Biyari, the founding moderator of r/MacPro2019LocalAI.

This is our new home for all things related to using the amazing, but now discontinued, Mac Pro 2019 / MacPro7,1 for local AI.

Whether you are running macOS, Windows, or any Linux distro, and whether you are using Ollama, vLLM, llama.cpp, LM Studio, OpenClaw, or the awesomely named Oobabooga, this community is here for one purpose:

To help each other get the most out of this powerful hardware for local AI workloads.

This subreddit is especially focused on the Mac Pro 2019’s unique hardware, including MPX GPUs with 32 GB of VRAM, Duo modules with up to 64 GB, Infinity Fabric Link Bridge experimentation, ROCm, local LLMs, image generation, voice AI, video generation, multimodal models, and all AI workloads.

A Brief Introduction

I started this subreddit because I have personally gone through the struggle of making local AI work on the Mac Pro 2019.

I have run into many of the same roadblocks others are likely facing:

• macOS support limitations
• AMD GPU support challenges
• ROCm installation and compatibility issues
• PyTorch, Triton, and framework confusion
• Ollama, vLLM, llama.cpp, LM Studio, LangChain, Hermes Agent, Oobabooga, and other tooling choices
• User interface decisions
• Hardware limitations
• Infinity Fabric Link Bridge experimentation
• Deprecated MPX GPU support

The struggle is real, and I understand it.

Fortunately, I have managed to get local AI working on this hardware. I have installed Linux, first Ubuntu and later Proxmox, installed ROCm, used Ollama, worked on vLLM, experimented with OpenClaw, and continued exploring the Infinity Fabric Link Bridge.

I have also shared guides in the past to help people install Linux on the MacPro7,1, set up ROCm, and reach a working local AI setup. Those guides focused mostly on getting started, but there is much more to explore.

The reality is that MPX GPUs are losing support across many tools and platforms, and because this use case is so niche, AI tools and assistants often do not provide useful guidance.

What helped me the most were other Mac Pro 2019 users working toward the same goal. Their motivation, ideas, troubleshooting, and even general technical knowledge helped me understand the bigger picture and keep moving forward.

That is why I created this subreddit: to centralize our experiences, guides, lessons learned, experiments, successes, and failures in one place instead of forcing everyone to search through hundreds of websites and dozens of subreddits.

My Hardware

I currently work with two Mac Pro 2019 machines:

LinuxAI-64

Mac Pro 2019 / MacPro7,1
3.2 GHz 16-core Intel Xeon W
96 GB DDR4 RAM
Two AMD Radeon Pro W6900X GPUs, 32 GB each
64 GB total VRAM
8 TB Apple SSD
100GbE Mellanox ConnectX-5 Ex NIC

System Firmware: 2069.0.0.0.0
iBridge Firmware: 22.16.10353.0.0
OS Loader / iBoot: 860.140.1~8

LinuxAI-128

Mac Pro 2019 / MacPro7,1
3.2 GHz 16-core Intel Xeon W
96 GB DDR4 RAM
Two AMD Radeon Pro W6800X Duo MPX modules, 32 GB each GPU
128 GB total VRAM
8 TB Apple SSD
100GbE Mellanox ConnectX-5 Ex NIC

System Firmware: 2069.0.0.0.0
iBridge Firmware: 22.16.10353.0.0
OS Loader / iBoot: 860.140.1~8

What to Post

Post anything you think the community would find interesting, helpful, or inspiring.

Examples include:

• Your Mac Pro 2019 local AI setup
• Hardware specs and GPU configuration
• Linux, macOS, Windows, Proxmox, or dual-boot experiences for local AI workloads
• ROCm installation notes
• Ollama, vLLM, llama.cpp, LM Studio, OpenClaw, Hermes Agent, Oobabooga, or other framework experiences
• Benchmarks and performance results
• Model compatibility reports
• Text, image, voice, video, or multimodal AI workflows
• Troubleshooting questions
• Guides, scripts, and installation notes
• Cooling, power, PCIe, storage, or networking setups to support local AI workloads
• Infinity Fabric Link Bridge experiments
• Things that worked, and things that definitely did not

Introduce Yourself

Please introduce yourself in the comments below.

When you do, I kindly ask that you include your hardware details, such as:

• Mac Pro 2019 CPU
• RAM
• GPU / MPX module configuration
• Total VRAM
• System & iBridge Firmwares, and OS Loader / iBoot, if known
• Operating system
• AI frameworks, agents, models, or tools you are using
• What you hope to run locally
• Any challenges you are currently facing

Even if you are just getting started, your experience may help someone else.

Community Vibe

We are here to be friendly, constructive, and helpful.

This is a niche community, and many of us are simply trying to keep powerful hardware useful long after official support has started to fade. Let’s build a space where people feel comfortable asking questions, sharing experiments, posting failures, and helping each other move forward.

How to Get Started

Introduce yourself in the comments below.

Post something today, even if it is just a simple question or a photo of your setup.

If you have guides, notes, scripts, benchmarks, or lessons learned, please share them.

If you know someone who owns a Mac Pro 2019 and is interested in local AI, invite them to join.

Interested in helping out? I am always open to hearing from people who may want to help moderate or contribute to the community.

Thanks for being part of the very first wave. Together, let’s make r/MacPro2019LocalAI an amazing resource for everyone trying to run local AI on the Mac Pro 2019.

I know these are both older GPUs at this point and that the W6800X generally outperforms the Vega II, but I'm wondering if there are any use cases where a Vega II is better (maybe due to the higher memory bandwidth?)

I ended up with both of these GPUs and am planning to keep one alongside my RX 6900 XT.

I previously made guide for using the Mac Pro 2019 (MacPro7,1) with MPX GPUs for Local LLM/AI. While I was not fully satisfied with the outcome, I continued to push forward to find better ways to take advantage of my hardware, improving utilization and efficiency.

Previous Guide: https://www.reddit.com/r/macpro/comments/1q9xeov/guide_mac_pro_2019_macpro71_w_linux_local_llmai/

____________________________________________________

I am still developing and working on it further. Full testing has not been done, and this may be unstable. However, my main priority while doing this has always bee Protecting my Investment in these GPUs. As such, the first thing I work on is making sure Fans are always set to Maximum, within an acceptably cold room.

I will share my experience and the steps I built for myself to repeat this. This is based on my preference and my personal needs. Modify as you see fit for your scenario. This guide assumes some general knowledge relating to command line; AI is your friend otherwise.

Proceed at your own Risk: I am just fumbling through, and documenting what worked for me.

Important note: If I did not already have the GPUs on hand, I would not have done any of the below, or invested in Apple devices for local AI/LLMs.

Credit where Credit is Due:

• A HUGE Thank You to the T2 Linux Community!! & a special Thank You!! to u/AdityaGarg8 for tolerating me and helping guide me.
• NetworkChuck, for inspiring me to work on Local AI, and his awesome attitude.
• ChatGPT, who's been working closely with me to stop using it and move on to more private AIs. Much Love 😘
• AMD, for ROCm, and the plethora of documentation. It's always the right time to try and improve.
• Meta, for making a big deal over going Open Source and seemingly paving the way for others to follow suit.
• Everyone that worked on the references below.

Thank You All

Hardware: I now had two machines with similar specs (only difference are the GPUs) First machine, LinuxAI-128:

• Xeon W 3.2 Ghz 16-core CPU
• 96 GB 2933 Mhz DDR4 RAM
• 8 TB SSD
• Dual AMD Radeon PRO W6800X Duo (Total VRAM: 128 GB)
• 100GbE NIC PCIe Card, Mellanox ConnectX-5

Second machine, LinuxAI-64:

• Xeon W 3.2 Ghz 16-core CPU
• 96 GB 2933 Mhz DDR4 RAM
• 8 TB SSD
• Dual AMD Radeon PRO W6900X (Total VRAM: 64 GB)
• 100GbE NIC PCIe Card, Mellanox ConnectX-5

Goals: Continue to use the servers for LLM/AI, but also conveniently run other solutions to take advantage of the hardware, and make it acceptable to invest further based on need.

Summary:

• Rebuild utilizing Industry Standard, in the Home Lab
  • Setup Proxmox
  • Achieve Fan control on Proxmox (To protect GPU investment)
  • Setup Ubuntu in VM and Pass through GPUs
  • Verify no issues with virtualization
  • Verify no (significant) degradation in efficiency

Decisions:

• Maintain the same previous decisions; Ubuntu, ROCm, etc.
• Update to the latest supported software; 24.04.3, 7.1.1, etc.
• Segregate utilization by using Virtual Machines (VMs).
• Due to my experience with Proxmox, that will be my hypervisor of choice.
• To free GPU resources, the machines will be headless, in CLI.
• Due to the (well documented) heat issues with the AMP Radeon PRO W6800X Duo, I need to have the fans continuously on, on maximum. (I prefer having to replace the fans in a few years over having to replace any hardware, such as the GPUs - ^(cc: Mac Pro 2013))
• To benefit from the 100 Gbps connection, and to avoid the loud fan noise, the machines will be in my dataroom; home lab area.
• Avoid virtualization, and docker, due to perceived (no scientific data) reduction in tokens/s.

0. Prepare the Hardware

1. If you have an Infinity Fabric Link (Bridge or Jumper) attached to your GPU, it must be removed. Although it theoretically will improve GPU function, as of this writing, it will add no value to local LLM/AI, and will only give you a headache.
2. Modify Mac Boot Security Settings:
   1. Boot into macOS Recovery Mode (Cmd + R at startup).
   2. Open Startup Security Utility and:
   3. Disable Secure Boot.
   4. Enable Allow booting from external or removable media.

1. Download and Prepare Proxmox Installation

1. Download Proxmox Virtual Environment 9.1-1 ISO: Proxmox Official Site
2. Create a bootable USB using your preferred method. Possible Options:
   1. Etcher
   2. iodd Device (My preferred method)
   3. Rufus

2. Install Proxmox Virtual Environment

1. Boot from USB and start installation.
   1. Connect the USB & boot the mac while holding alt (option)
   2. Select Proxmox Installation (Typically on the far right. Possibly called "EFI Boot")
2. Follow installation steps
   1. Take note of the IP Address used during setup.
   2. Default username is root
3. Finish installation and reboot into Proxmox.

3. Setup Proxmox

This guide uses Terminal/Command Line where possible to make it easier and copy/paste friendly for efficiency (especially given that I need to duplicate this on more than just one machine). Using ssh from the comfort of your desktop or laptop would be ideal, in my personal opinion.

The following will:

• Modify Grub to support Mac Pro and Virtualization
• Personal Preference: Fix locale issues when ssh-ing from macOS
• Remove Enterprise repo & add no-subscription repo
• Upgrade Proxmox
• Upgrade Kernel
• Install Kernel Headers, dkms, and required files to support T2 Macs (Part 01)
• Setup T2 Mac repo
• Install T2 Mac packages to support T2 Macs (Part 02)
• Personal Preference: Set fans to Always Maximum Air Flow
• Download Ubuntu Server 24.04.3 LTS ISO
• Create a VM 1001, attach Ubuntu ISO to it, Detect GPUs, attach GPUs to VM, with the following specs:
  • CPU: 4
  • RAM: 8 GB
  • Storage: 512 GB (My personal setup is higher, due to size of LLMs)
  • All AMD GPUs
  • Other miscillanous settings

​

# Setup Proxmox Virtual Environment 9.1 on Mac Pro 2019 (MacPro7,1)

# Modify Grub to support Mac hardware and pass through
GRUB_LINE='GRUB_CMDLINE_LINUX_DEFAULT="loglevel=7 log_buf_len=16M iommu=pt intel_iommu=on pcie_ports=compat vfio_iommu_type1.allow_unsafe_interrupts=1 pcie_acs_override=downstream,multifunction"'; \
(grep -Eq '^[[:space:]]*#?[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT=' /etc/default/grub && \
sed -i -E "s|^[[:space:]]*#?[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT=.*|$GRUB_LINE|" /etc/default/grub || \
printf '\n%s\n' "$GRUB_LINE" >> /etc/default/grub) && \
update-grub && \
grep -nE '^[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT=' /etc/default/grub

# Personal Preference: Fix locale, time, time zone, and errors from ssh into the server
locale-gen en_US.UTF-8 && tee /etc/default/locale >/dev/null <<'EOF'
LANG="en_US.UTF-8"
LC_ALL="en_US.UTF-8"
LC_CTYPE="en_US.UTF-8"
EOF
. /etc/default/locale && export LANG LC_ALL LC_CTYPE && locale && timedatectl

# Remove Enterprise repo
rm -f /etc/apt/sources.list.d/pve-enterprise.sources
rm -f /etc/apt/sources.list.d/ceph.sources

# Add No-Subscription Repo
cat > /etc/apt/sources.list.d/proxmox.sources <<'EOF'
Types: deb
URIs: http://download.proxmox.com/debian/pve
Suites: trixie
Components: pve-no-subscription
Signed-By: /usr/share/keyrings/proxmox-archive-keyring.gpg
EOF

# Upgrade Proxmox
apt update && apt full-upgrade -y

# Install required files for T2 Mac setup
apt install -y proxmox-default-kernel proxmox-default-headers dkms build-essential
reboot

# Setup T2 Mac repos
curl -s --compressed "https://adityagarg8.github.io/t2-ubuntu-repo/KEY.gpg" | gpg --dearmor | tee /etc/apt/trusted.gpg.d/t2-ubuntu-repo.gpg >/dev/null
curl -s --compressed -o /etc/apt/sources.list.d/t2.list "https://adityagarg8.github.io/t2-ubuntu-repo/t2.list"
apt update

CODENAME=trixie
echo "deb [signed-by=/etc/apt/trusted.gpg.d/t2-ubuntu-repo.gpg] https://github.com/AdityaGarg8/t2-ubuntu-repo/releases/download/${CODENAME} ./" | tee -a /etc/apt/sources.list.d/t2.list
apt update

# Install T2 Mac Files, to achieve fan control
apt install applesmc-t2 apple-bce t2fanrd -y
systemctl enable --now t2fanrd
systemctl restart t2fanrd
reboot

# Personal Preference: Set always_full_speed=true on all 4 fans, to protect GPU investment
sudo bash -c 'cfg=/etc/t2fand.conf; tmp=$(mktemp); countfile=$(mktemp); awk '"'"'function flush_missing(){if(in_fan && !saw_afs){print "always_full_speed=true"; changed++} saw_afs=0} BEGIN{in_fan=0;saw_afs=0;changed=0;total_fans=0} /^\[Fan[0-9]+\][[:space:]]*$/{flush_missing(); in_fan=1; total_fans++; print; next} /^\[[^]]+\][[:space:]]*$/{flush_missing(); in_fan=0; print; next} /^[[:space:]]*always_full_speed[[:space:]]*=/{saw_afs=1; line=$0; val=line; sub(/^[^=]*=/,"",val); gsub(/[[:space:]]/,"",val); if(tolower(val)!="true"){sub(/=.*/,"=true",line); changed++} print line; next} {print} END{flush_missing(); printf "%d %d\n", changed, total_fans > "/dev/stderr"}'"'"' "$cfg" > "$tmp" 2> "$countfile"; read changed total < "$countfile"; mv "$tmp" "$cfg"; rm -f "$countfile"; echo "Fans changed to always_full_speed=true: $changed (Fan sections detected: $total)";'
systemctl restart t2fanrd

# Download Ubuntu Server 24.04 LTS
cd /var/lib/vz/template/iso
wget -O ubuntu-24.04.3-live-server-amd64.iso https://releases.ubuntu.com/noble/ubuntu-24.04.3-live-server-amd64.iso
cd ~

# Auto Detect GPUs and Create VM for Ubuntu with GPUs passed through
VMID=1001; \
NAME="UbuntuAI"; \
ISO="local:iso/ubuntu-24.04.3-live-server-amd64.iso"; \
DISK="local-lvm:512"; \
BRIDGE="vmbr0"; \
NETMODEL="e1000e"; \
MACHINE="q35"; \
OSTYPE="l26"; \
CORES=4; \
MEMORY=8192; \
BIOS="seabios"; \
SCSIHw="virtio-scsi-single"; \
EXTRA="pcie=1,rombar=0"; \
mapfile -t GPUS < <(lspci -Dn | awk '$2 ~ /^030[02]:/ && $3 ~ /^1002:/ {print $1}' | sort -V); \
((${#GPUS[@]})) || { echo "ERROR: No AMD GPUs found (vendor 1002, class 0300/0302)."; exit 1; }; \
HOSTPCI_ARGS=(); \
for i in "${!GPUS[@]}"; do \
  addr="${GPUS[$i]%.*}"; \
  HOSTPCI_ARGS+=( "--hostpci${i}" "${addr},${EXTRA}" ); \
done; \
echo "GPUs detected:"; \
printf '  %s\n' "${GPUS[@]}"; \
echo; \
echo "Running:"; \
printf '%q ' qm create "$VMID" --name "$NAME" --ostype "$OSTYPE" --machine "$MACHINE" --cores "$CORES" --memory "$MEMORY" --balloon 0 --bios "$BIOS" --start 0 --net0 "${NETMODEL},bridge=${BRIDGE}" --scsihw "$SCSIHw" --scsi0 "$DISK" --ide2 "${ISO},media=cdrom" "${HOSTPCI_ARGS[@]}" --boot "order=scsi0;ide2"; \
echo; \
qm create "$VMID" --name "$NAME" --ostype "$OSTYPE" --machine "$MACHINE" --cores "$CORES" --memory "$MEMORY" --balloon 0 --bios "$BIOS" --start 0 --net0 "${NETMODEL},bridge=${BRIDGE}" --scsihw "$SCSIHw" --scsi0 "$DISK" --ide2 "${ISO},media=cdrom" "${HOSTPCI_ARGS[@]}" --boot "order=scsi0;ide2"

# Launch Ubuntu and set it up accordingly

# Done Proxmox setup

3. Setup Ubuntu

Access proxmox's GUI, start the VM, and setup ubuntu. I did the following during setup:

• Take note of the IP Address (To SSH into the VM later on)
• Change the mirror address from http to https
• Unchecked "Set up this disk as an LVM group"
• Setup username and password
• Enabled Ubuntu Pro (Free Account)
• Checked "Install OpenSSH server"

Once installation completes, reboot and either ssh into the VM using the IP address you noted during installation earlier, or continue on the proxmox GUI.

ssh ubuntu-user-name@IP-Address

4. Install AMDGPU, ROCm, and everything else

All of the following will need to be done on Terminal. I personally opted to ssh into Linux, so I can easily copy/paste into it.

# Setup Ubuntu Server 24.04 LTS and ROCm 7.1.1 in a Virtual Machine on Proxmox Virtual Environment 9.1

# Personal Preference: replace http Repos to https in ubuntu repo
sudo sed -i 's|http://|https://|g' /etc/apt/sources.list.d/ubuntu.sources


# Personal Preference: Modify grub to prepare for debugging
sudo bash -c 'GRUB_LINE='\''GRUB_CMDLINE_LINUX_DEFAULT="loglevel=7 log_buf_len=16M"'\''; \
(grep -Eq "^[[:space:]]*#?[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT=" /etc/default/grub && \
sed -i -E "s|^[[:space:]]*#?[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT=.*|$GRUB_LINE|" /etc/default/grub || \
printf "\n%s\n" "$GRUB_LINE" >> /etc/default/grub) && \
update-grub && \
grep -nE "^[[:space:]]*GRUB_CMDLINE_LINUX_DEFAULT=" /etc/default/grub'


# Update kernel
sudo apt update
sudo apt install linux-generic-hwe-24.04 -y
sudo reboot


# Update & Upgrade
sudo apt update && sudo apt upgrade -y


# Download & Install ROCm 7.1.1
wget https://repo.radeon.com/rocm/installer/rocm-runfile-installer/rocm-rel-7.1.1/ubuntu/24.04/rocm-installer_1.2.4.70101-25-38~24.04.run
chmod +x rocm-installer_1.2.4.70101-25-38~24.04.run
./rocm-installer_1.2.4.70101-25-38~24.04.run deps=install rocm amdgpu force amdgpu-start gpu-access=all postrocm


# Verify AMDGPU & ROCm Installation, outputting CPU & GPU Information
update-alternatives --list rocm
dkms status
rocminfo
clinfo
rocm-smi
sudo amd-smi


# Reboot to clean up
sudo reboot


# Personal Preference: to avoid errors and simplify life
sudo apt install 2to3 python-is-python3 -y


# Add ~/.local/bin to path, so the new installed commands can work easily
grep -qxF 'export PATH="$HOME/.local/bin:$PATH"' ~/.bashrc || echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc


# Installing PyTorch
sudo apt install python3-pip -y
pip3 install --upgrade pip wheel --break-system-packages
python3 -m pip install "numpy==1.26.4" --break-system-packages
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/torch-2.9.1%2Brocm7.1.1.lw.git351ff442-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/torchvision-0.24.0%2Brocm7.1.1.gitb919bd0c-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/triton-3.5.1%2Brocm7.1.1.gita272dfa8-cp312-cp312-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/torchaudio-2.9.0%2Brocm7.1.1.gite3c6ee2b-cp312-cp312-linux_x86_64.whl
pip3 uninstall torch torchvision triton torchaudio --break-system-packages -y
pip3 install torch-2.9.1+rocm7.1.1.lw.git351ff442-cp312-cp312-linux_x86_64.whl torchvision-0.24.0+rocm7.1.1.gitb919bd0c-cp312-cp312-linux_x86_64.whl torchaudio-2.9.0+rocm7.1.1.gite3c6ee2b-cp312-cp312-linux_x86_64.whl triton-3.5.1+rocm7.1.1.gita272dfa8-cp312-cp312-linux_x86_64.whl --break-system-packages


# Verify PyTorch Installation, you want to see "Success" & "True", and then GPU information output, and finally a list of all the GPUs
python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'
python3 -c 'import torch; print(torch.cuda.is_available())'
python3 -c "import torch; print(f'device name [0]:', torch.cuda.get_device_name(0))"
python3 -m torch.utils.collect_env
python3 -c "import torch; n=torch.cuda.device_count(); print('is_available:', torch.cuda.is_available()); print('device_count:', n); print('\n'.join([f'[{i}] {torch.cuda.get_device_name(i)}' for i in range(n)]))"


# Add AMD-ROCm Repo (ROCm 7.1.1 for Ubuntu 24.04 "noble")
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://repo.radeon.com/rocm/rocm.gpg.key | gpg --dearmor | sudo tee /etc/apt/keyrings/rocm.gpg > /dev/null
sudo chmod 644 /etc/apt/keyrings/rocm.gpg
echo "deb [arch=amd64 signed-by=/etc/apt/keyrings/rocm.gpg] https://repo.radeon.com/rocm/apt/7.1.1 noble main" | sudo tee /etc/apt/sources.list.d/rocm.list
sudo apt update


# Install MIGraphX
sudo apt install migraphx half -y


# Verify Installation
/opt/rocm-7.1.1/bin/migraphx-driver perf --test
dpkg -l | grep migraphx
dpkg -l | grep half


# Install ONNX Runtime
pip3 uninstall onnxruntime-rocm --break-system-packages
pip3 install onnxruntime-rocm -f https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/ --break-system-packages


# Verify installation
python3 -c "import onnxruntime as ort; print(ort.get_available_providers())"


# Install TensorFlow for ROCm
pip install tf-keras --no-deps --break-system-packages
pip3 uninstall tensorflow-rocm --break-system-packages
pip3 install https://repo.radeon.com/rocm/manylinux/rocm-rel-7.1.1/tensorflow_rocm-2.20.0.dev0%2Bselfbuilt-cp312-cp312-manylinux_2_28_x86_64.whl --break-system-packages


# Verify TensorFlow Installation:
python3 -c 'import tensorflow' 2> /dev/null && echo 'Success' || echo 'Failure'


# Clean up installation
sudo apt update && sudo apt upgrade -y
sudo apt autoremove -y
sudo reboot

# Done

11. Ollama Installation:

Step 1: Installation

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Download LLM(s)

# Models smaller than 60 GB:
ollama pull gpt-oss:20b
ollama pull gpt-oss:120b # Takes up 94% & 97% of 2 GPUs, respectively.
ollama pull deepseek-r1:70b
ollama pull llama3.3
ollama pull llama3.2-vision:90b
ollama pull mxbai-embed-large:335m
ollama pull nomic-embed-text
ollama pull llava:34b
ollama pull qwen2:72b
ollama pull qwen2.5:72b
ollama pull qwen3-vl:32b
ollama pull codellama:70b
ollama pull qwen2.5-coder:32b
ollama pull granite-code:34b
ollama pull aya-expanse:32b
ollama pull deepseek-r1:1.5b
ollama pull deepseek-r1:7b
ollama pull deepseek-r1:8b
ollama pull deepseek-r1:14b
ollama pull deepseek-r1:32b
ollama pull nemotron-3-nano:30b
ollama pull olmo-3.1:32b
ollama pull devstral-small-2:24b

# Models smaller than 128 GB:
ollama pull mistral-large
ollama pull mixtral:8x22b
ollama pull dolphin-mixtral:8x22b
ollama pull devstral-2:123b

Step 3: Run the LLM

ollama pull gpt-oss:20b --verbose

Step 4: Profit 😁😁😁

The End ???

Sources:

>https://amdgpu-install.readthedocs.io/en/latest/index.html
https://rocm.docs.amd.com/en/latest/
https://rocm.docs.amd.com/projects/radeon/en/latest/index.html
https://rocm.docs.amd.com/projects/install-on-linux/en/latest/index.html
https://rocm.docs.amd.com/projects/radeon-ryzen/en/latest/index.html
https://repo.radeon.com/
https://t2linux.org/
https://ollama.com/download

I'm the furthest thing from an expert, and probably don't understand or know what I'm doing. If you can optimize this, please do. I'll take any help I can get, and spread it where I can.

Notes:

• GPT-OSS is extremely efficient, compared to other LLMs, in this setup. Load time is under 30 seconds. Tokens are over 40 per second.
• I gave Ubuntu low resources, ignoring recommendation of RAM matching VRAM, to give these resources to other VMs.
• I noted improved performance of LLMs in the VM over bare-metal. It was a pleasant surprise.
• Loading Deepseek-r1 took over 3-5 minutes in the VM, compared to under 30 seconds in the bare-metal scenario.
• Loading Llama3.3 took over 2-5 minutes in the VM, compared to under 30 seconds in the bare-metal scenario.

tl;dr

Ubuntu VM, in Proxmox, on MacPro7,1: Nice

GPU pass through works

AI/LLM Working on GPU

gpt-oss:20b: 70-80 tokens/s

gpt-oss120b: 44-53 tokens/s

deepseek-r1:70b: 7-8 tokens/s

llama3.3: 7-8 tokens/s

Deepseek-r1:1.5b: 171-189 tokens/s

AMD Radeon PRO W6900X more token/s than AMD Radeon PRO W6800X Duo

Good Luck, & Have Fun!!

P.S.

If an experienced person can reach out to me to look into parallelism between two machines, utilizing the Mellanox ConnectX-5 with Remote Direct Memory Access (RDMA), I would greatly appreciate it.

Original Post was removed due to shadow banning. Posting again for reference and benefit of the community.

Original: https://www.reddit.com/r/macpro/comments/1ij3k4s/guide_mac_pro_2019_macpro71_w_linux_local_llmai/

____________________________________________________

Trying to take advantage of the MPX GPUs available to me for the purpose of Local AI/LLM, I started a journey to install Linux on my Mac Pro 2019 ( MacPro7,1 ), ROCm, and figure out the complicated web of Local AI/LLMs. I will share my experience and the steps I built for myself to repeat this. This is based on my preference and my personal needs. Modify as you see fit for your scenario. This guide assumes some general knowledge relating to command line; AI is your friend otherwise.

Proceed at your own Risk: I am just fumbling through, and documenting what worked for me.

Quick Back Story: I've had a Mac Pro 2019 since 2020, for multiple use cases. In early 2023, I found an unbelievable deal for SSDs & GPUs for it, and ended up with several, including 2 of the AMD Radeon Pro W6900X & 2 of the AMD Radeon Pro W6800X Duo. With the release of ROCm (or an update?) mid-2024, I decided to take advantage of these GPUs for Local AI/LLM utilization, but I was not about to do it on my main machine. 🤷🏻‍♂️ After of a month or two of searching for good/affordable deals on Mac Pros 2019, I picked up a couple above-minimum spec'd Mac Pro 2019 machines.

If I did not already have the GPUs on hand, I would not have done any of the below, or invested in Apple devices for local AI/LLMs.

Credit where Credit is Due:

• A HUGE Thank You to the T2 Linux Community!! & a special Thank You!! to u/AdityaGarg8 for tolerating me and helping guide me.
• NetworkChuck, for inspiring me to work on Local AI, and his awesome attitude.
• ChatGPT, who's been working closely with me to stop using it and move on to more private AIs. Much Love 😘
• AMD, for ROCm, and the plethora of documentation. It's always the right time to try and improve.
• Meta, for making a big deal over going Open Source and seemingly paving the way for others to follow suit.
• u/Juanlumg, for motivating me to get this done 😅
• Everyone that worked on the references below.

Thank You All

Hardware: I now had two machines with similar specs (only difference are the GPUs) First machine, LinuxAI-128:

• Xeon W 3.2 Ghz 16-core CPU
• 96 GB 2933 Mhz DDR4 RAM
• 8 TB SSD
• Dual AMD Radeon PRO W6800X Duo (Total VRAM: 128 GB)
• 100GbE NIC PCIe Card, Mellanox ConnectX-5

Second machine, LinuxAI-64:

• Xeon W 3.2 Ghz 16-core CPU
• 96 GB 2933 Mhz DDR4 RAM
• 8 TB SSD
• Dual AMD Radeon PRO W6900X (Total VRAM: 64 GB)
• 100GbE NIC PCIe Card, Mellanox ConnectX-5

Goals: The goal was to utilize the GPUs for a local AI, to remember all my history some how, and help me with my daily work as a personal assistant. (Including be a teacher to my kids... Some How)

Original Goals:

• Setup local AI/LLM to "type-chat"
  • Setup ROCm
• Allow for voice communication
  • Setup TTS
  • Setup Whisper
• Setup secure remote access
  • TwinGate
  • Cloud Flare secure tunnel?
• Allow access across my home via voice
• Setup IoT control across my home
  • Setup Home Assistant

Developed Goals as I progress:

• Setup Memory across chats
  • LangChain
  • Memoir+ ?
• Allow for reading documents
• Allow for document generation
• Use both machine's GPUs simultaneously (Benefit from larger models, up to 192 GB VRAM)
• Improve tokens/s & optimize

Decisions:

• I needed to use Linux for ROCm support.
• Due to my experience with Ubuntu, that will be my Linux of choice.
• Due to ROCm limited support, I will be using Ubuntu LTS 22.04.
• To benefit from the machine hardware/resources, I will be using Ubuntu Server LTS 22.04.
• To free GPU resources, the machines will be headless, in CLI.
• Due to the (well documented) heat issues with the AMP Radeon PRO W6800X Duo, I need to have the fans continuously on, on maximum. (I prefer having to replace the fans in a few years over having to replace any hardware, such as the GPUs - ^(cc: Mac Pro 2013))
• To benefit from the 100 Gbps connection, and to avoid the loud fan noise, the machines will be in my dataroom, homelab area.
• Avoid virtualization, and docker, due to perceived (no scientific data) reduction in tokens/s.

0. Prepare the Hardware

1. If you have an Infinity Fabric Link (Bridge or Jumper) attached to your GPU, it must be removed. Although it theoretically will improve GPU function, as of this writing, it is not supported on Linux.
2. Modify Mac Boot Security Settings:
   1. Boot into macOS Recovery Mode (Cmd + R at startup).
   2. Open Startup Security Utility and:
   3. Disable Secure Boot.
   4. Enable Allow booting from external or removable media.
3. Shrink macOS partition (if keeping macOS):
   1. Use Boot Camp Assistant or Disk Utility to reduce macOS to 50 GB (or your preferred size).
   2. Create a new partition

1. Download and Prepare Ubuntu Installation

1. Download Ubuntu Server LTS 22.04 ISO: Ubuntu Official Site
2. Create a bootable USB using your preferred method. Possible Options:
   1. Etcher
   2. iodd Device (My preferred method)
   3. Rufus

2. Install Ubuntu 22.04

1. Boot from USB and start installation.
   1. Connect the USB & boot the mac while holding alt (option)
   2. Select Ubuntu Installation (Typically on the far right. Possibly called "EFI Boot")
2. Follow installation steps
3. For Installation location:
   1. Select Custom Installation
   2. Choose free space left after macOS.
   3. Format it as ext4 and mount as / (root).
   4. Boot should be mounted automatically. If not, please make some room for it.
4. Finish installation and reboot into Ubuntu.

3. Install AMDGPU, ROCm, and everything else

All of the following will need to be done on Terminal. I personally opted to ssh into Linux, so I can easily copy/paste into it from the comfort of my main PC.

# Update & Upgrade
sudo apt update && sudo apt upgrade -y

# Improve Boot Time by disabling cloud-init & Network Wait
sudo apt remove --purge cloud-init -y
sudo systemctl disable systemd-networkd-wait-online.service
sudo systemctl mask systemd-networkd-wait-online.service

# Modify grub to comply with ROCm and T2-Linux Documentation as well as prepare for debugging
# Replace GRUB_CMDLINE_LINUX_DEFAULT="" with the one below
# GRUB_CMDLINE_LINUX_DEFAULT="loglevel=7 log_buf_len=16M iommu=pt intel_iommu=on pcie_ports=compat"
sudo nano /etc/default/grub
sudo update-grub

# Update kernel
sudo apt install linux-generic-hwe-22.04 -y
sudo reboot

# Install T2-Linux repo and files for improved function
curl -s --compressed "https://adityagarg8.github.io/t2-ubuntu-repo/KEY.gpg" | gpg --dearmor | sudo tee /etc/apt/trusted.gpg.d/t2-ubuntu-repo.gpg >/dev/null
sudo curl -s --compressed -o /etc/apt/sources.list.d/t2.list "https://adityagarg8.github.io/t2-ubuntu-repo/t2.list"
CODENAME=jammy
echo "deb [signed-by=/etc/apt/trusted.gpg.d/t2-ubuntu-repo.gpg] https://github.com/AdityaGarg8/t2-ubuntu-repo/releases/download/${CODENAME} ./" | sudo tee -a /etc/apt/sources.list.d/t2.list
sudo apt update
sudo apt install applesmc-t2 apple-bce t2fanrd -y
sudo reboot

# Edit fan file as needed
sudo nano /etc/t2fand.conf
sudo systemctl restart t2fanrd

# Prepare Prerequisites for AMDGPU & ROCm (Kernel, groups, and new user groups, i386 support):
sudo apt install "linux-headers-$(uname -r)" "linux-modules-extra-$(uname -r)" -y
sudo usermod -a -G render,video $LOGNAME
echo 'ADD_EXTRA_GROUPS=1' | sudo tee -a /etc/adduser.conf
echo 'EXTRA_GROUPS=video' | sudo tee -a /etc/adduser.conf
echo 'EXTRA_GROUPS=render' | sudo tee -a /etc/adduser.conf
sudo dpkg --add-architecture i386
sudo reboot

# Update & Upgrade
sudo apt update && sudo apt upgrade -y

# Download all AMDGPU 6.2.3 & ROCm files
# Folder 01
mkdir ~/downloads/
mkdir ~/downloads/rocm-6.2.3/
mkdir ~/downloads/rocm-6.2.3/1
cd ~/downloads/rocm-6.2.3/1
wget https://repo.radeon.com/amdgpu-install/6.2.3/ubuntu/jammy/amdgpu-install_6.2.60203-1_all.deb

# Folder 02
mkdir ~/downloads/rocm-6.2.3/2
cd ~/downloads/rocm-6.2.3/2
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/amdgpu-pro-oglp_24.20-2044449.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-amdgpu1_2.4.120.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-radeon1_2.4.120.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm2-amdgpu_2.4.120.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libegl1-amdgpu-mesa_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libegl1-amdgpu-mesa-drivers_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libegl1-amdgpu-pro-oglp_24.20-2044449.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgbm1-amdgpu_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgl1-amdgpu-mesa-dri_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgl1-amdgpu-mesa-glx_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgl1-amdgpu-pro-oglp-dri_24.20-2044449.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgl1-amdgpu-pro-oglp-glx_24.20-2044449.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libglapi-amdgpu-mesa_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgles1-amdgpu-pro-oglp_24.20-2044449.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgles2-amdgpu-pro-oglp_24.20-2044449.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/libllvm18.1-amdgpu_18.1.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-dev_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-drm2_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-glx2_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-wayland2_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-x11-2_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva2-amdgpu_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libvdpau-amdgpu/libvdpau-amdgpu-dev_6.2-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libvdpau-amdgpu/libvdpau1-amdgpu_6.2-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-client0_1.22.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-cursor0_1.22.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-dev_1.22.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-egl-backend-dev_1.22.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-egl1_1.22.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-server0_1.22.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libxatracker2-amdgpu_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-va-drivers_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-vdpau-drivers_24.2.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/va-amdgpu-driver-all_2.16.0.60203-2044426.22.04_i386.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/v/vulkan-amdgpu-pro/vulkan-amdgpu-pro_24.20-2044449.22.04_i386.deb

# Folder 03
mkdir ~/downloads/rocm-6.2.3/3
cd ~/downloads/rocm-6.2.3/3
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu/amdgpu_6.2.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu/amdgpu-lib_6.2.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu/amdgpu-lib32_6.2.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/g/gst-omx-amdgpu/gst-omx-amdgpu_1.0.0.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/llvm-amdgpu_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/llvm-amdgpu-18.1_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/llvm-amdgpu-18.1-dev_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/llvm-amdgpu-18.1-runtime_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/llvm-amdgpu-dev_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/llvm-amdgpu-runtime_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/l/llvm-amdgpu/libllvm18.1-amdgpu_18.1.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-amdgpu1_2.4.120.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-dev_2.4.120.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-radeon1_2.4.120.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-static_2.4.120.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm-amdgpu-utils_2.4.120.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu/libdrm2-amdgpu_2.4.120.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-dev_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-drm2_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-glx2_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-wayland2_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva-amdgpu-x11-2_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/libva2-amdgpu_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libva-amdgpu/va-amdgpu-driver-all_2.16.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libvdpau-amdgpu/libvdpau-amdgpu-dev_6.2-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libvdpau-amdgpu/libvdpau1-amdgpu_6.2-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libegl1-amdgpu-mesa_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libegl1-amdgpu-mesa-dev_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libegl1-amdgpu-mesa-drivers_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libegl1-amdgpu-pro-oglp_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgbm-amdgpu-dev_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgbm1-amdgpu_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libglapi-amdgpu-mesa_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgl1-amdgpu-mesa-dev_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgl1-amdgpu-mesa-dri_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libgl1-amdgpu-mesa-glx_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgl1-amdgpu-pro-oglp-dri_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgl1-amdgpu-pro-oglp-ext_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgl1-amdgpu-pro-oglp-gbm_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgl1-amdgpu-pro-oglp-glx_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/s/smi-lib-amdgpu/smi-lib-amdgpu_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/s/smi-lib-amdgpu/smi-lib-amdgpu-dev_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/v/vulkan-amdgpu/vulkan-amdgpu_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-bin_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-client0_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-cursor0_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-dev_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-egl-backend-dev_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-egl1_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-server0_1.22.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/x/xserver-xorg-amdgpu-video-amdgpu/xserver-xorg-amdgpu-video-amdgpu_22.0.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/a/amdgpu-pro/amdgpu-pro_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/a/amdgpu-pro/amdgpu-pro-lib32_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/a/amf-amdgpu-pro/amf-amdgpu-pro_1.4.35-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/liba/libamdenc-amdgpu-pro/libamdenc-amdgpu-pro_1.0-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/amdgpu-pro-oglp_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgles1-amdgpu-pro-oglp_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/o/oglp-amdgpu-pro/libgles2-amdgpu-pro-oglp_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/v/vulkan-amdgpu-pro/vulkan-amdgpu-pro_24.20-2044449.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-common-dev_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-multimedia_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-omx-drivers_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-va-drivers_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/mesa-amdgpu-vdpau-drivers_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libxatracker-amdgpu-dev_24.2.0.60203-2044426.22.04_amd64.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/m/mesa-amdgpu/libxatracker2-amdgpu_24.2.0.60203-2044426.22.04_amd64.deb


# Folder 04
mkdir ~/downloads/rocm-6.2.3/4
cd ~/downloads/rocm-6.2.3/4
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu-core/amdgpu-core_6.2.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu-dkms/amdgpu-dkms_6.8.5.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu-dkms/amdgpu-dkms-firmware_6.8.5.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu-dkms/amdgpu-dkms-headers_6.8.5.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu-doc/amdgpu-doc_6.2-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/a/amdgpu-install/amdgpu-install_6.2.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/proprietary/a/amdgpu-pro-core/amdgpu-pro-core_24.20-2044449.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-amdgpu/libwayland-amdgpu-doc_1.22.0.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/w/wayland-protocols-amdgpu/wayland-protocols-amdgpu_1.34.60203-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libv/libvdpau-amdgpu/libvdpau-amdgpu-doc_6.2-2044426.22.04_all.deb
wget https://repo.radeon.com/amdgpu/6.2.3/ubuntu/pool/main/libd/libdrm-amdgpu-common/libdrm-amdgpu-common_1.0.0.60203-2044426.22.04_all.deb

# Move Back to User Folder
cd ~/

# Install first AMDGPU file followed by AMDGPU script for ROCm and Everything AMD has to offer
sudo apt-get install ~/downloads/rocm-6.2.3/1/*.deb -y
amdgpu-install --usecase=dkms,graphics,multimedia,workstation,rocm,rocmdev,rocmdevtools,amf,lrt,opencl,openclsdk,hip,hiplibsdk,openmpsdk,mllib,mlsdk,asan -y --accept-eula --opencl=rocr --opengl=mesa --vulkan=amdvlk,pro

# Install remaining AMDGPU files for full coverage
sudo apt-get install ~/downloads/rocm-6.2.3/2/*.deb -y
sudo apt-get install ~/downloads/rocm-6.2.3/3/*.deb -y
sudo apt-get install ~/downloads/rocm-6.2.3/4/*.deb -y

# The following command should install Nothing
sudo apt install amdgpu-dkms rocm

# AMDGPU post installation setup
sudo tee --append /etc/ld.so.conf.d/rocm.conf <<EOF
/opt/rocm/lib
/opt/rocm/lib64
EOF
sudo ldconfig
echo 'export PATH="$HOME/.local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"' >> ~/.bashrc
source ~/.bashrc

# Install vulkan-tools & mesa-utils
sudo apt install vulkan-tools mesa-utils -y
sudo reboot

# Verify AMDGPU & ROCm Installation, outputting CPU & GPU Information
update-alternatives --list rocm
module avail
dkms status
rocminfo
clinfo
rocm-smi

# Installing PyTorch
sudo apt install python3.10 -y
sudo update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1 -y
sudo update-alternatives --config python3
sudo apt install python3.10-distutils python3.10-venv -y
pip install --upgrade pip
pip3 install --upgrade pip wheel
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/torch-2.3.0%2Brocm6.2.3-cp310-cp310-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/torchvision-0.18.0%2Brocm6.2.3-cp310-cp310-linux_x86_64.whl
wget https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/pytorch_triton_rocm-2.3.0%2Brocm6.2.3.5a02332983-cp310-cp310-linux_x86_64.whl
pip3 uninstall torch torchvision pytorch-triton-rocm
pip3 install torch-2.3.0+rocm6.2.3-cp310-cp310-linux_x86_64.whl torchvision-0.18.0+rocm6.2.3-cp310-cp310-linux_x86_64.whl pytorch_triton_rocm-2.3.0+rocm6.2.3.5a02332983-cp310-cp310-linux_x86_64.whl
sudo apt install python-is-python3

# Verify PyTorch Installation, you want to see "Success" & "True", and then GPU information output
python3 -c 'import torch' 2> /dev/null && echo 'Success' || echo 'Failure'
python3 -c 'import torch; print(torch.cuda.is_available())'
python3 -c "import torch; print(f'device name [0]:', torch.cuda.get_device_name(0))"
python3 -m torch.utils.collect_env

# Install ONNX Runtime
pip3 uninstall onnxruntime-rocm
pip3 install onnxruntime-rocm -f https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/

# Verify installation
python3 -c "import onnxruntime as ort; print(ort.get_available_providers())"

# Install TensorFlow for ROCm
pip install tf-keras --no-deps
pip3 uninstall tensorflow-rocm
pip3 install https://repo.radeon.com/rocm/manylinux/rocm-rel-6.2.3/tensorflow_rocm-2.16.2-cp310-cp310-manylinux_2_28_x86_64.whl

# Verify TensorFlow Installation:
python3 -c 'import tensorflow' 2> /dev/null && echo 'Success' || echo 'Failure'

# Done

11. Ollama Installation:

Step 1: Installation

curl -fsSL https://ollama.com/install.sh | sh

Step 2: Download LLM(s)

# Models smaller than 60 GB:
ollama pull llama3.3
ollama pull llama3.2-vision:90b
ollama pull mxbai-embed-large:335m
ollama pull nomic-embed-text
ollama pull llava:34b
ollama pull deepseek-r1:70b
ollama pull qwen2:72b
ollama pull qwen2.5:72b
ollama pull qwen3-vl:32b
ollama pull codellama:70b
ollama pull qwen2.5-coder:32b
ollama pull granite-code:34b
ollama pull aya-expanse:32b
ollama pull deepseek-r1:1.5b
ollama pull deepseek-r1:7b
ollama pull deepseek-r1:8b
ollama pull deepseek-r1:14b
ollama pull deepseek-r1:32b

# Models smaller than 128 GB:
ollama pull gpt-oss:120b
ollama pull mistral-large
ollama pull mixtral:8x22b
ollama pull dolphin-mixtral:8x22b

Step 3: Run the LLM

ollama run llama3.3

Step 4: Profit 😁😁😁

The End ???

Sources:

>https://amdgpu-install.readthedocs.io/en/latest/index.html
https://rocm.docs.amd.com/en/latest/
https://rocm.docs.amd.com/projects/radeon/en/latest/index.html
https://rocm.docs.amd.com/projects/install-on-linux/en/latest/index.html
https://repo.radeon.com/
https://t2linux.org/
https://ollama.com/download

I'm the furthest thing from an expert, and probably don't understand or know what I'm doing. If you can optimize this, please do. I'll take any help I can get, and spread it where I can.

tl;dr

Ubuntu on MacPro7,1 Nice

AI/LLM Working on GPU

LLM 14b: 25-28 token/s

LLM 32b: 13-16 token/s

LLM 70b: 5-7 token/s

AMD Radeon PRO W6900X more token/s than AMD Radeon PRO W6800X Duo

Good Luck, & Have Fun!!