MSA Code Search

Semantic code search powered by MSA-4B (Memory Sparse Attention).

First-of-its-kind: nobody has built a code search tool on MSA yet.

Quick Start

See examples.txt for the full walkthrough.

# Setup
git clone https://github.com/EverMind-AI/MSA.git
conda create -n msa python=3.12 -y && conda activate msa
cd MSA && pip install -r requirements.txt && cd ..
pip install datasets flash-attn==2.7.4.post1 --no-build-isolation

# Download model
huggingface-cli download EverMind-AI/MSA-4B --local-dir MSA/ckpt/MSA-4B

# Compile training data
python scripts/01_compile_training_data.py --output data/msa_15k.json --target-pairs 15000 --skip-git-mining

# Train
cd MSA && python ../scripts/02_train_msa.py --data ../data/msa_15k.json --model-path ckpt/MSA-4B --output-path ckpt/MSA-4B-Code --epochs 3 && cd ..

# Benchmark
python scripts/03_benchmark.py --model-path MSA/ckpt/MSA-4B-Code --benchmark swebench

Architecture

MSA-4B finds files → Memory Interleave reads them → Returns file + snippet + related files.

No vector database. No embeddings. No reranker. One model.

License

Scripts: MIT. MSA model: see EverMind-AI/MSA license.