GitHub has standard privacy controls and policies protecting user data and discussion content from unauthorized access.
Terms of Service
+0.05
Article 1 Article 2
GitHub ToS establish baseline equal treatment of users without discrimination, though enforcement depends on implementation.
Accessibility
+0.15
Article 25 Article 26
Observable accessibility features including keyboard navigation, ARIA support, and responsive design promote equitable access to platform functionality.
Mission
—
GitHub's public mission emphasizes open collaboration and global access to development tools, indirectly supporting knowledge-sharing rights.
Editorial Code
+0.08
Article 19 Article 27
GitHub community guidelines establish standards for respectful discussion and protect user expression within community contexts.
Ownership
-0.05
Article 17
GitHub retains platform control; user-generated content ownership is subject to platform terms, creating conditional rather than absolute intellectual property rights.
Access Model
+0.12
Article 19 Article 27
Public discussion board model enables open participation and knowledge dissemination without gatekeeping, supporting freedom of expression and information access.
Ad/Tracking
-0.08
Article 12
GitHub's feature flags and analytics tracking create potential privacy concerns; behavioral data collection may infringe on privacy of thought.
0.2 tok/s is fine for experimentation, but it is not interactive in any meaningful sense. For many use cases, a well-quantized 8B or 13B that stays resident will simply deliver a better latency-quality tradeoff
Could be neat to see what giving the 8b like 6gb ram instead of 10gb. Something in-between, where you still need NVMe, but not like the 3x ratio of the 70b model on 23GB.
Nice work. PCI-P2P (GPU-Direct (tm)) is such great stuff. Cool to see!
Nice. I've been looking at doing something similar, more on the order of running a 1T model with less than half the available VRAM.
One workup indicated it was theoretically possible to modify a piece of SGLang's routing layer to support JIT predict-ahead expert swaps from Gen5 NVMe storage straight into GPU memory.
I'm hoping that proves true. The setup relies on NVIDIA Dynamo, so NIXL primitives are available to support that.
I feel like we need an entirely new type of silicon for LLMs. Something completely focused on bandwidth and storage probably at the sacrifice of raw computation power.
This is an interesting area for experiments. I suspect that in the longer term model optimization (knowing which bits you can leave out without affecting the functioning of the model) will become the dominant area of research just like it did with compression algorithms because effectively a model is a lossy compression scheme.
And that's good because that increases democratization of AI away from the silos that are being created.
Yeah, GPUdirect should allow you to dma straight to a storage device.
I wonder... what if the m.2 storage was actually DRAM? You probably don't need persistence for spilling a model off the GPU. How would it fare vs just adding more host memory? The m.2 ram would be less flexible, but would keep the system ram free for the CPU.
Really cool. I'm wondering: what background did you need to be able to think of the question that resulted in this project?
I know you said you're involved in some retrogaming and were experimenting, but as someone who works in a world where hardware is pretty heavily abstracted away, even if I got into retrogaming I don't know that I'd consider that there may be a systems improvement lying around. Beyond the creative aspect, it feels like there is some systems and hardware background that helped put the idea together (and I'd be interested to go learn about of that systems/hardware knowledge myself).
Cool project. Can you provide more details about your DKMS patching process for consumer GPUs? This would be fun to try out, but I’d need some more details on that patch process first.
I've often wondered doing this with extreme compression. What if you did extreme compression + decompression on the GPU? Because you're leaving a lot of compute unused.
Cool hack but 0.5 tok/s on 70B when a 7B does 30+ on the same card. NVIDIA's own research says 40-70% of agentic tasks could run on sub-10B models and the quality gap has closed fast.
Really interesting experiment i should have done this before
Do you have numbers on effective throughput vs PCIe theoretical bandwidth?
I’m curious whether this is primarily latency-bound or bandwidth-bound in practice
Can some tell me??
Score Breakdown
+0.25
PreamblePreamble
Medium Practice
Editorial
ND
Structural
+0.25
SETL
ND
Combined
ND
Context Modifier
ND
Repository structure and GitHub platform support international collaboration and development without explicit discrimination. No editorial content directly addressing preamble values.
+0.20
Article 1Freedom, Equality, Brotherhood
Medium Practice
Editorial
ND
Structural
+0.15
SETL
ND
Combined
ND
Context Modifier
ND
GitHub ToS establish equal treatment baseline. Repository itself contains no editorial on equal dignity or rights.
+0.20
Article 2Non-Discrimination
Medium Practice
Editorial
ND
Structural
+0.15
SETL
ND
Combined
ND
Context Modifier
ND
GitHub ToS include non-discrimination provisions. No on-page evidence of discrimination in repository access or participation.
ND
Article 3Life, Liberty, Security
No observable content addressing right to life, liberty, personal security on this repository page.
ND
Article 4No Slavery
No observable content addressing slavery or servitude on this repository page.
ND
Article 5No Torture
No observable content addressing torture or cruel treatment on this repository page.
ND
Article 6Legal Personhood
No observable content addressing right to recognition before law on this repository page.
ND
Article 7Equality Before Law
No observable content addressing equal protection of law on this repository page.
ND
Article 8Right to Remedy
No observable content addressing right to effective remedy on this repository page.
ND
Article 9No Arbitrary Detention
No observable content addressing freedom from arbitrary arrest on this repository page.
ND
Article 10Fair Hearing
No observable content addressing fair trial and impartial hearing on this repository page.
ND
Article 11Presumption of Innocence
No observable content addressing presumption of innocence on this repository page.
+0.07
Article 12Privacy
Medium Practice
Editorial
ND
Structural
+0.05
SETL
ND
Combined
ND
Context Modifier
ND
GitHub privacy protections apply; however, feature flag and analytics tracking present on page create modest privacy concerns. Net effect slightly positive due to privacy controls outweighing tracking.
+0.30
Article 13Freedom of Movement
Medium Practice
Editorial
ND
Structural
+0.30
SETL
ND
Combined
ND
Context Modifier
ND
Repository is publicly accessible without geographic restriction; users can freely move within and across the platform globally for collaboration purposes.
ND
Article 14Asylum
No observable content addressing asylum or refuge rights on this repository page.
ND
Article 15Nationality
No observable content addressing right to nationality on this repository page.
ND
Article 16Marriage & Family
No observable content addressing family rights or marriage on this repository page.
0.00
Article 17Property
Medium Practice
Editorial
ND
Structural
+0.05
SETL
ND
Combined
ND
Context Modifier
ND
Repository code is visible and attributable to author. However, GitHub retains platform control and content ownership is conditional, limiting absolute property rights protection.
ND
Article 18Freedom of Thought
No observable content addressing freedom of thought, conscience, religion on this repository page.
+0.57
Article 19Freedom of Expression
High Advocacy Framing Practice Coverage
Editorial
+0.35
Structural
+0.40
SETL
-0.14
Combined
ND
Context Modifier
ND
Repository demonstrates freedom of expression: open-source code publicly shared, comments and collaboration enabled, no gatekeeping of discussion. GitHub's access model and editorial guidelines support expression. Public repository enables information dissemination without censorship.
+0.20
Article 20Assembly & Association
Medium Practice
Editorial
ND
Structural
+0.20
SETL
ND
Combined
ND
Context Modifier
ND
Repository allows discussions and collaborative participation; no observable prohibition on peaceful assembly or association. GitHub's community features enable collective action around technical projects.
ND
Article 21Political Participation
No observable content addressing political participation or voting on this repository page.
ND
Article 22Social Security
No observable content addressing social security or welfare rights on this repository page.
ND
Article 23Work & Equal Pay
No observable content addressing work or employment rights on this repository page.
ND
Article 24Rest & Leisure
No observable content addressing rest and leisure rights on this repository page.
+0.40
Article 25Standard of Living
Medium Practice
Editorial
ND
Structural
+0.25
SETL
ND
Combined
ND
Context Modifier
ND
GitHub's accessible design (keyboard navigation, ARIA, responsive) supports equitable access to this technical resource. Repository contains developmental tools that may support standard of living when used broadly.
+0.35
Article 26Education
Medium Framing Practice
Editorial
+0.15
Structural
+0.25
SETL
-0.16
Combined
ND
Context Modifier
ND
Repository description and code demonstrate technical education and knowledge sharing. GitHub's public model supports education access. Accessibility features enable learning for diverse users.
+0.57
Article 27Cultural Participation
High Advocacy Framing Practice Coverage
Editorial
+0.40
Structural
+0.35
SETL
+0.14
Combined
ND
Context Modifier
ND
Repository explicitly shares intellectual property (software) openly; code is accessible and reusable. Open-source licensing model (implied by public repository) enables participation in scientific/cultural life. GitHub editorial guidelines and access model support creative expression and benefit-sharing.
+0.15
Article 28Social & International Order
Medium Practice
Editorial
ND
Structural
+0.15
SETL
ND
Combined
ND
Context Modifier
ND
GitHub infrastructure provides foundational support for social and international order enabling UDHR rights. Repository participates in global technical commons. No observable impediment to rights realization.
ND
Article 29Duties to Community
No observable content addressing duties to community or limitations on rights on this repository page.
+0.10
Article 30No Destruction of Rights
Low Practice
Editorial
ND
Structural
+0.10
SETL
ND
Combined
ND
Context Modifier
ND
No activity or content promoting destruction of UDHR rights observed. Repository exists within legal framework protecting rights; no hostile interpretation detected.