GitHub - vinnylarouge/jevlike · GitHub

Skip to content

Navigation MenuSign inAppearance settingsPlatformAI CODE CREATIONGitHub CopilotWrite better code with AIGitHub Copilot appDirect agents from issue to mergeMCP RegistryIntegrate external toolsDEVELOPER WORKFLOWSActionsAutomate any workflowCodespacesInstant dev environmentsIssuesPlan and track workCode ReviewManage code changesCode QualityEnforce quality at mergeAPPLICATION SECURITYGitHub Advanced SecurityFind and fix vulnerabilitiesCode securitySecure your code as you buildSecret protectionStop leaks before they startEXPLOREWhy GitHubDocumentationBlogChangelogMarketplaceView all featuresSolutionsBY COMPANY SIZEEnterprisesSmall and medium teamsStartupsNonprofitsBY USE CASEApp ModernizationDevSecOpsDevOpsCI/CDView all use casesBY INDUSTRYHealthcareFinancial servicesManufacturingGovernmentView all industriesView all solutionsResourcesEXPLORE BY TOPICAISoftware DevelopmentDevOpsSecurityView all topicsEXPLORE BY TYPECustomer storiesEvents & webinarsEbooks & reportsBusiness insightsGitHub SkillsSUPPORT & SERVICESDocumentationCustomer supportCommunity forumTrust centerPartnersView all resourcesOpen SourceCOMMUNITYGitHub SponsorsFund open source developersPROGRAMSSecurity LabMaintainer CommunityGitHub StarsArchive ProgramREPOSITORIESTopicsTrendingCollectionsEnterpriseENTERPRISE SOLUTIONSEnterprise platformAI-powered developer platformAVAILABLE ADD-ONSGitHub Advanced SecurityEnterprise-grade security featuresCopilot for BusinessEnterprise-grade AI featuresPremium SupportEnterprise-grade 24/7 supportPricingSearch/Sign inSign upAppearance settings

You signed in with another tab or window. Reload to refresh your session.
You signed out in another tab or window. Reload to refresh your session.
You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

vinnylarouge

/

jevlike

Public

Notifications
You must be signed in to change notification settings

Fork
22

Star
218

Code

Issues
0

Pull requests
0

Actions

Projects

Security and quality
0

Insights

Additional navigation options

Code

Issues

Pull requests

Actions

Projects

Security and quality

Insights

mainBranchesTagsGo to fileCodeOpen more actions menuLatest commit History3 Commits3 CommitsFolders and filesNameNameLast commit messageLast commit datedocsdocs  examplesexamples  jevlikejevlike  scriptsscripts  teststests  .gitignore.gitignore  AGENTS.mdAGENTS.md  LICENSELICENSE  README.mdREADME.md  pyproject.tomlpyproject.toml  View all filesRepository files navigationREADMEMIT licenseMore itemsJevlike
Train a small model that chooses among a changing list of text options.
A Jev-like model takes a piece of text and a list of N text options. It returns one probability for each option. It does this in one pass instead of writing an answer word by word. Jev is TypeSafe's commercial model for this kind of task. TypeSafe has not published its design. This repository is an independent starter model with the same input and output shape.
Demo
The same option-attention head can score controller buttons from image patches. This ten-second film joins two selected five-second windows: live deadly_corridor combat on the seven Doom buttons, then a chess controller walking to and playing moves with five keys. The diagram shows the tensors used for each decision. The Doom window came from the supplied joint checkpoint, which averaged 0.60 kills and -97.50 reward across its ten recorded episodes. The chess window came from the stronger chess-only checkpoint, which scored 4 wins, 46 draws and 0 losses in 50 sampled games against a random mover, but 0 wins, 2 draws and 48 losses against Stockfish level 0. The windows were selected for activity and are not typical-play or competence claims.

Install the game extras and record a fresh 640 by 480 Doom trace from the released joint checkpoint:
uv pip install -e '.[games]'
python examples/doom/play.py examples/checkpoints/joint-imitation.pt --episodes 10 --game-seconds 35.3 --device cpu --capture-resolution 640x480 --output runs/doom.mp4 --trace runs/doom-trace.json
Render the trace in the same visual layout. This writes a silent film because the author-owned soundtrack source is not part of the repository.
(cd examples/film && npm install && npx playwright install chromium)
examples/film/make-film.sh runs/doom-trace.json runs/doom-film.mp4 10
The release includes the Doom example, the chess example, the single-game checkpoints and the shared 12-option checkpoint. Both games import the visual scorer from jevlike.vision; there is no second model copy in either example.
Architecture
Each option becomes a query vector, which is a short list of numbers representing its text. The query assigns attention weights to the context tokens. Those weights make one context vector for that option. A shared dot product turns each option and context pair into one score. A softmax, which converts scores into probabilities that sum to one, runs across the options.

The default encoder learns byte embeddings from scratch. An encoder is the part that turns text into vectors. The optional Hugging Face path uses a frozen pretrained encoder, whose existing weights stay fixed while the small scorer learns.
Data format
Use one JSON object per line:
{"context":"The customer needs a refund.","options":["refund","sales","technical support"],"label":0}
label is the zero-based index of the correct option. Each row may have a different number of options, with a minimum of two.
Quickstart
Run these commands from the repository root. They create local synthetic data, train on it, evaluate the saved model and score one new menu.
uv venv
source .venv/bin/activate
uv pip install -e '.[dev]'

jevlike-data synthetic --output data/synthetic
jevlike-train data/synthetic/train.jsonl \
--validation data/synthetic/validation.jsonl \
--output runs/synthetic.pt
jevlike-eval runs/synthetic.pt data/synthetic/test.jsonl
jevlike-predict runs/synthetic.pt \
--context "Choose the exact badge amber badger. Badge: amber badger." \
--option "azure crane" \
--option "amber badger" \
--option "gold heron"
The evaluation prints top-1 accuracy, which is the fraction of correct first choices. Top-3 accuracy is the fraction with the right answer among the three highest scores. Expected calibration error compares confidence with observed accuracy. The command also prints a shuffled-context control, which pairs each menu with the wrong context. A useful model should beat that control.
Use your own data

Export train, validation and test JSONL files in the format above.
Keep all options that the model will see at prediction time in each row.
Split related records together. For example, keep all records for one customer or one target page in one split. This prevents near-duplicates from leaking into the test set.
Run jevlike-train with your train and validation files.
Run jevlike-eval once on the held-out test file. Held-out means the file was never used for training or model selection.

The default byte encoder truncates context to 192 bytes and each option to 32 bytes. Raise --context-tokens or --option-tokens when your text needs more room. Training supports CPU, Apple MPS for a Mac GPU, and CUDA for an NVIDIA GPU through --device.
Use a frozen pretrained encoder
Install the optional dependency and name any compatible encoder from Hugging Face:
uv pip install -e '.[transformers]'
jevlike-train data/synthetic/train.jsonl \
--validation data/synthetic/validation.jsonl \
--output runs/qwen-head.pt \
--encoder hf \
--hf-model Qwen/Qwen2.5-0.5B \
--rank 256 \
--batch-size 8
The checkpoint stores the trained scorer head and the encoder name. It does not copy the frozen encoder weights. Loading the checkpoint therefore needs access to the same Hugging Face model.
--rank sets the width of the small scorer head. A wider head has more trainable weights and uses more memory.
Wikispeedia example
scripts/get_wikispeedia.sh downloads the public SNAP archives and builds next-click JSONL files. The data stay outside this repository.
scripts/get_wikispeedia.sh
jevlike-train data/wikispeedia/jsonl/train.jsonl \
--validation data/wikispeedia/jsonl/validation.jsonl \
--output runs/wikispeedia.pt
Cite Robert West and Jure Leskovec, Human Wayfinding in Information Networks, WWW 2012. Review the source data terms on the SNAP dataset page.
What to expect
In the experiments that led to this starter, the one-pass scorer reached about 98% accuracy on synthetic menus. On target-disjoint Wikispeedia next-click data, a frozen Qwen2.5-0.5B encoder plus the scorer reached 26%, against about 8% for shuffled and random-encoder controls. A small model trained from scratch on 40,000 clicks reached 29%. At eight options, one pass was about 100 times faster than a small decoder forced to write 400 tokens.
These numbers describe local experiments, not this quickstart run. We did not show equal quality with Jev or reproduce TypeSafe's private training method.
Limitations

This is a research starter, not a copy of Jev.
Accuracy depends on data quality, split quality and the encoder.
The byte encoder is cheap but weak on language meaning.
The pretrained path may download a large model and needs more memory.
One-pass scoring requires the complete option list before prediction.
The speed comparison used a small local decoder rather than a large commercial model.

Licence
Code is released under the MIT License. Downloaded datasets and pretrained models keep their own terms.
AboutNo description, website, or topics provided.ResourcesReadmeMIT licenseActivityStars218 starsWatchers3 watchingForks22 forksReport repositoryReleasesPackagesContributorsLanguages

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms

Privacy

Security

Status

Community

Docs

Contact

Manage cookies

Do not share my personal information

You can’t perform that action at this time.

The jevlike repository presents an independent starter model designed to train a small model capable of selecting among a changing list of text options, drawing inspiration from TypeSafe's commercial Jev model, although the specific design of Jev is not published. The model utilizes a one-pass scoring mechanism, which processes a piece of text and a list of text options simultaneously to return a probability distribution for each option rather than generating an answer word by word. Architecturally, each option is represented as a query vector, a short numerical list encoding its text. This query vector assigns attention weights across the context tokens, which in turn derive a context vector for that specific option. A shared dot product then calculates a score for each option against the context. Finally, a softmax function converts these scores into probabilities that sum to one across all available options.

The system incorporates an encoder component responsible for transforming text into these numerical vectors. The default implementation learns byte embeddings from scratch during training. Alternatively, the implementation allows for the use of a frozen pretrained encoder from the Hugging Face library, where the weights remain fixed while the small scorer head learns the necessary scoring mechanism.

The data format for training involves structuring input as JSON objects, with each line representing a context, a list of options, and a label indicating the zero-based index of the correct option. This data can be used for synthetic data generation, where synthetic examples are created and used to train, validate, and evaluate the model. The quickstart process involves generating synthetic data, training the model, evaluating it, and demonstrating prediction on new contexts. When using external datasets, such as the Wikispeedia next-click data, the process involves downloading the data, constructing JSONL files, and training the model on these splits.

Performance evaluations focus on metrics like top-1 accuracy, which measures the fraction of correct first choices, and top-3 accuracy, which measures the fraction with the correct answer among the three highest-scoring options. The evaluation process also includes a shuffled-context control to measure the model's ability to distinguish relevant contexts. Experimental results indicate that the one-pass scorer achieved approximately ninety-eight percent accuracy on synthetic menus. When tested against target-disjoint Wikispeedia next-click data using a frozen Qwen2.5-0.5B encoder, the model reached twenty-six percent accuracy, compared to approximately eight percent for controls using shuffled contexts or random encoders. Furthermore, a small model trained from scratch on forty thousand clicks achieved twenty-nine percent accuracy. The framework also demonstrates significant speed advantages, noting that one-pass scoring is about one hundred times faster than a small decoder forced to generate four hundred tokens.

Limitations associated with this research starter include the dependency of accuracy on the quality of the training and validation splits, the choice of the encoder, and the quality of the data itself. The byte encoder is computationally inexpensive but lacks semantic understanding of language. When utilizing a pretrained encoder, the model requires access to that large model, which necessitates additional memory. Additionally, the one-pass scoring method requires the complete set of options to be available before making a prediction. The comparison speed also used a smaller local decoder rather than a large commercial model. The code is released under the MIT License, with the datasets and pretrained models retaining their separate terms.