{
  "source": {
    "repo": "https://github.com/aien-dev/CUDAplusplus",
    "readme": "cp:README.md",
    "how": "cp@:62ea5a214d46f36e88cb7b892cecebf1f3e8a172/docs/HOW_WE_DID_IT.md",
    "how_branch": "docs/how-we-did-it",
    "how_commit": "62ea5a2",
    "main_commit": "fc90713",
    "omega_doc": "om@:89aad7c99b4fced80f92b656d2a3f350699456f0/docs/OMEGA.md",
    "omega_branch": "docs/what-omega-is",
    "omega_commit": "89aad7c",
    "checked": "2026-10-22",
    "ledger": "L402",
    "visibility": "private",
    "owner_ledger": "L395"
  },
  "claim": "CUDA++ is the GPU half of Omega: everything CUDA gives you, plus the motor.",
  "claim_rest": [
    "It runs on the NVIDIA GPUs people already own, with no NVIDIA user-space software: no CUDA toolkit, no nvcc, no PTX, no libcuda, no cuBLAS. It is its own open source project, under the MIT licence, in the repository aien-dev/CUDAplusplus.",
    "It runs on the NVIDIA GB10 in the DGX Spark today. On 2026-10-21 the 35 billion parameter model AIEN serves on the Spark was running on this code with zero CUDA libraries loaded."
  ],
  "purpose_title": "Less computation, out of the big data centers.",
  "purpose": "The goal is not a faster way to rent a data center. It is to need less of one: the same work with fewer bytes moved, fewer operations and fewer joules per answer, so it runs on a machine a person owns (a DGX Spark, a PC with one graphics card, a laptop). Every result is reported as work per answer, against the machine's own ceiling.",
  "purpose_link": "om@:89aad7c99b4fced80f92b656d2a3f350699456f0/docs/OMEGA.md",
  "private_note": "The repository is private until its licence and safety audit is done (ledger L395), then it is made public. Until then the links into it on this page do not open.",
  "audit_link": "cp:docs/AUDIT.md",
  "hops": [
    {
      "step": "Write a kernel",
      "cuda": "CUDA C++",
      "ours": "The kernel builder, called from C. Omega's own languages reaching the GPU is Omega bar C1, built on this toolchain.",
      "status": "PARTIAL",
      "note": "C calls only",
      "links": [["kernel builder", "cp:src/omega_blackwell_codegen.c"], ["bar C1", "/omega/#c1"]],
      "box": "cp:README.md#cuda-piece-by-piece"
    },
    {
      "step": "Compile to machine code",
      "cuda": "nvcc, NVVM, PTX, ptxas",
      "ours": "Our own sm_121 encoder and register allocator: 59 instruction kinds, launch descriptors.",
      "status": "PASS",
      "note": "every kernel the live server runs is encoded by it",
      "links": [["encoder", "cp:src/omega_blackwell_encoder.c"]],
      "box": "om@:89aad7c99b4fced80f92b656d2a3f350699456f0/evidence/WHAT-OMEGA-IS-20261021/live-serve-no-cuda.txt"
    },
    {
      "step": "Talk to the driver",
      "cuda": "libcuda",
      "ours": "Our own client for the open kernel module's ioctls, written from its MIT headers.",
      "status": "PASS",
      "note": "no libcuda linked",
      "links": [["nvrm client", "cp/:nvrm"], ["open headers", "cp/:third_party/nvidia-open-580.173.02"]],
      "box": "cp:evidence/m16-blackwell-native-path-requalification-receipt.json"
    },
    {
      "step": "Run work",
      "cuda": "libcudart",
      "ours": "Session, command lists and job engine: device, memory, upload, launch.",
      "status": "PARTIAL",
      "note": "one device, one process; no streams or events yet",
      "links": [["command lists", "cp:src/omega_gpu_cmdlist.c"]],
      "box": "cp:README.md#cuda-piece-by-piece"
    },
    {
      "step": "Libraries",
      "cuda": "cuBLAS, cuDNN, CUB",
      "ours": "Matrix multiply (BF16, FP8, 4 bit, ternary), decode and prefill attention, frozen-order reductions.",
      "status": "PARTIAL",
      "note": "shaped for the models we run; no any-shape GEMM yet",
      "links": [["reductions receipt", "cp/:evidence/E1-CLOSURE"]],
      "box": "cp:ROADMAP.md#1-nvidia-software-on-the-spark-today"
    },
    {
      "step": "Feed the math cores",
      "cuda": "nothing: left to you",
      "ours": "The motor, the ++ in CUDA++: one schedule built before anything runs, movement engines that check every weight part, and the stroke, many conversations on one read of the weights.",
      "status": "PASS",
      "note": "live on port 18091; 1.80x to 1.98x the same conversations taken in turn; on branches, not on omega main yet",
      "links": [["the motor on /omega/", "/omega/#motor"], ["design", "om@:89aad7c99b4fced80f92b656d2a3f350699456f0/docs/OMEGA.md#6-cuda-the-motor"]],
      "box": "sc@/:9c62690a252ab1c0b75eb8933c550316b5c50d52/evidence/q36-batch-20261021"
    },
    {
      "step": "Kernel driver",
      "cuda": "nvidia.ko, nvidia-uvm.ko",
      "ours": "AIEN's own GB10 driver. Its simulator gates pass; it has not run on the chip yet.",
      "status": "IN PROGRESS",
      "note": "ledger L332",
      "ledger": "L332",
      "links": [["simulator gates receipt", "ax@:ce74e10c13ff3bd0fa22855efd93c65a0633f4ac/evidence/own-gpu-driver/rung1/gates-20261021T2300Z.txt"], ["L332 on /now/", "/now/#L332"]],
      "box": "ax@:89360c060102e4242187cda6156d696452e38181/docs/GPU_DRIVER.md"
    },
    {
      "step": "Inside the GPU",
      "cuda": "GSP firmware",
      "ours": "Stays NVIDIA's. The chip checks the firmware's signature and boots nothing else. CUDA++ never ships it.",
      "status": "NVIDIA",
      "note": "not replaceable",
      "links": [],
      "box": "cp:ROADMAP.md#1-nvidia-software-on-the-spark-today"
    }
  ],
  "how_lead": "One person and a team of AI coding agents, one DGX Spark, four weeks. NVIDIA publishes the source of its Linux kernel module, but not its compiler, its driver library or the GPU's machine code. So we skipped every closed layer.",
  "steps": [
    {
      "title": "1. The driver client, from the open kernel module's headers",
      "text": "libcuda is closed, but everything it does to the hardware goes through ioctls that the open kernel module defines in its public headers. We copied 67 of those headers unchanged (MIT) and wrote our own client against them: client, device, address space, memory, command ring, compute object, doorbell, completion.",
      "links": [["nvrm client", "cp/:nvrm"], ["the 67 headers", "cp/:third_party/nvidia-open-580.173.02"], ["no libcuda receipt", "cp:evidence/m16-blackwell-native-path-requalification-receipt.json"]],
      "box": "cp@:62ea5a214d46f36e88cb7b892cecebf1f3e8a172/docs/HOW_WE_DID_IT.md#step-1-talk-to-the-kernel-module-without-libcuda"
    },
    {
      "title": "2. The machine code, reverse engineered",
      "text": "NVIDIA does not publish the GPU's instruction encoding. We compiled small probe kernels of our own once with NVIDIA's compiler, read the machine code back with NVIDIA's disassembler as an offline oracle, and compared kernels that differ in one detail to see which bits mean what. We cross-checked against Mesa's open NVIDIA compiler, NAK, and believed nothing until the chip ran it and gave the right answer. The tensor-core arithmetic was measured on the chip over 2,048 cases until one model matched every output bit.",
      "links": [["encoder", "cp:src/omega_blackwell_encoder.c"], ["tensor-core probe", "cp:tests/gpu_tc_probe_test.c"], ["third-party notices", "cp:THIRD_PARTY.md"]],
      "box": "cp@:62ea5a214d46f36e88cb7b892cecebf1f3e8a172/docs/HOW_WE_DID_IT.md#step-2-learn-the-machine-code-the-reverse-engineering"
    },
    {
      "title": "3. Launches in NVK's order",
      "text": "A launch on Blackwell is a short list of commands in the channel ring. We took their order from Mesa's NVK Vulkan driver and the method offsets from the open headers, built our own command lists, and measured them: about 24 to 31 microseconds per extra dependent launch.",
      "links": [["command lists", "cp:src/omega_gpu_cmdlist.c"], ["launch cost receipt", "cp/:evidence/OWN30B-LAUNCH-COST-20261014T0120Z"]],
      "box": "cp@:62ea5a214d46f36e88cb7b892cecebf1f3e8a172/docs/HOW_WE_DID_IT.md#step-3-launch-work-like-the-driver-does"
    },
    {
      "title": "4. Built up, nothing counts until the chip agrees",
      "text": "2026-09-26 vector add on the GB10. 2026-10-02 numerics closed on the chip. 2026-10-13 our own probes measured the tensor cores at 115.14 TFLOPS and memory reads at 195.1 GB/s. 2026-10-21 a 35B model served on this code with zero CUDA libraries in the process.",
      "links": [["numerics receipt", "cp/:evidence/E1-CLOSURE"], ["ceiling receipt", "cp/:evidence/OWN30B-T3-TC-PEAK-BW-5faa0f70-20261013T2236Z"]],
      "box": "cp@:62ea5a214d46f36e88cb7b892cecebf1f3e8a172/docs/HOW_WE_DID_IT.md#step-4-build-up-checking-everything"
    }
  ],
  "did_not": [
    "No NVIDIA source code other than the open kernel module headers, and no NVIDIA binary. No libcuda, cubin, PTX or firmware in the repository or its history.",
    "We did not take apart libcuda, nvcc or any other NVIDIA program. We ran NVIDIA's compiler and disassembler on our own small kernels and read what they printed.",
    "We did not touch the GPU's firmware. It is still NVIDIA's, and the chip checks its signature.",
    "We did not translate CUDA programs to run on other hardware. CUDA++ runs on NVIDIA GPUs, the ones people already paid for."
  ],
  "did_not_box": "cp@:62ea5a214d46f36e88cb7b892cecebf1f3e8a172/docs/HOW_WE_DID_IT.md#what-we-did-not-do",
  "pain_lead": "What makes CUDA hard to use and hard to leave, how CUDA++ fixes each one, and the command that shows it. A row is marked fixed only with its receipt.",
  "pains": [
    {
      "title": "CUDA's pain points and the fix for each",
      "text": "Being written in docs/PAIN_POINTS.md and a README section of CUDA++: each pain point people report, how CUDA++ answers it, and the one command that shows the answer.",
      "status": "IN PROGRESS",
      "ledger": "L400",
      "links": [["L400 on /now/", "/now/#L400"]],
      "box": "s:crates/site-builder/data/cudaplusplus.json"
    }
  ],
  "swap_lead": "Take CUDA off a machine and put CUDA++ in its place, all the way down to the kernel driver, with one command each way.",
  "swaps": [
    {
      "title": "User space: cudapp swap",
      "text": "A drop-in libcuda.so.1 on the CUDA++ runtime, so a CUDA program runs unmodified on CUDA++. One command in, one command back. Written up in docs/SWAP.md.",
      "status": "IN PROGRESS",
      "ledger": "L401",
      "links": [["L401 on /now/", "/now/#L401"]],
      "box": "s:crates/site-builder/data/cudaplusplus.json"
    },
    {
      "title": "Kernel driver: cudapp swap kernel",
      "text": "Packaging, switch and rollback that replace nvidia.ko and nvidia-uvm.ko with AIEN's own GB10 driver. L401 builds the packaging; L332 builds the driver, whose simulator gates pass and which has not run on the chip yet.",
      "status": "IN PROGRESS",
      "ledger": "L401",
      "links": [["L401 on /now/", "/now/#L401"], ["L332 on /now/", "/now/#L332"], ["driver simulator gates receipt", "ax@:ce74e10c13ff3bd0fa22855efd93c65a0633f4ac/evidence/own-gpu-driver/rung1/gates-20261021T2300Z.txt"]],
      "box": "ax@:89360c060102e4242187cda6156d696452e38181/docs/GPU_DRIVER.md"
    }
  ]
}