Resources
The data, tools, and references behind HuskyBot.
Open academic datasets, modern AI tooling, and the research that informs how HuskyBot is built — all transparent, all credited.
Academic Datasets
Open and freely available scholarly data sources powering HuskyBot’s knowledge.
arXiv
Comprehensive open-access archive of preprint scientific papers across physics, math, computer science, and more. Bulk downloadable via S3.
Semantic Scholar / S2ORC
Allen AI’s open corpus of ~50–100M full-text articles and 300M metadata records, spanning all academic disciplines.
PubMed Central (Open Access Subset)
Free full-text archive of biomedical and life sciences journal literature from the U.S. National Library of Medicine.
OpenAlex
Open catalog of 250M+ scholarly works, authors, institutions, and citation networks. Fully CC0 (public domain).
CORE
Worlds largest collection of open-access research papers — aggregating from thousands of repositories worldwide.
Academic Style Guides
Citation manuals and academic writing style references that inform HuskyBot’s citation generation and tone.
Development Tools
The open-source AI/ML stack we use to fine-tune, serve, and evaluate HuskyBot.
Hugging Face Transformers
Library for working with open-weight LLMs (Mistral, Llama, etc.) and managing fine-tuning workflows.
PEFT / LoRA / QLoRA
Parameter-efficient fine-tuning techniques that let us train large models on modest hardware budgets.
Ollama
Local LLM serving framework for running fine-tuned models on consumer hardware. Ships GGUF compatibility.
llama.cpp
Efficient C++ implementation for running quantized LLMs on CPU and consumer GPUs — the engine behind GGUF.
LangChain / LlamaIndex
Frameworks for building retrieval-augmented generation (RAG) pipelines that ground HuskyBot in academic sources.
Weights & Biases
Experiment tracking and evaluation tooling for monitoring training runs and model behavior.
Reference Materials
Foundational research and legal references informing HuskyBot’s design.
LoRA: Low-Rank Adaptation of LLMs
Hu et al., 2021
Foundational paper on parameter-efficient fine-tuning.
QLoRA: Efficient Finetuning of Quantized LLMs
Dettmers et al., 2023
Quantized LoRA — lets us fine-tune large models on a single consumer GPU.
Retrieval-Augmented Generation for Knowledge-Intensive NLP
Lewis et al., 2020
The RAG architecture HuskyBot uses for grounded, citable answers.
Bartz v. Anthropic
U.S. court ruling, 2024
Recent legal precedent: training on lawfully acquired content is “highly transformative” and likely Fair Use.
Curious how it all comes together?
Read about HuskyBot’s technical approach — fine-tuning strategy, RAG architecture, and the full data pipeline.
Technical Approach