GitHub - kyle-pena-nlp/jevchat: Turns Jev into a chatbot · GitHub

Skip to content

Navigation MenuSign inAppearance settingsPlatformAI CODE CREATIONGitHub CopilotWrite better code with AIGitHub Copilot appDirect agents from issue to mergeMCP RegistryIntegrate external toolsDEVELOPER WORKFLOWSActionsAutomate any workflowCodespacesInstant dev environmentsIssuesPlan and track workCode ReviewManage code changesCode QualityEnforce quality at mergeAPPLICATION SECURITYGitHub Advanced SecurityFind and fix vulnerabilitiesCode securitySecure your code as you buildSecret protectionStop leaks before they startEXPLOREWhy GitHubDocumentationBlogChangelogMarketplaceView all featuresSolutionsBY COMPANY SIZEEnterprisesSmall and medium teamsStartupsNonprofitsBY USE CASEApp ModernizationDevSecOpsDevOpsCI/CDView all use casesBY INDUSTRYHealthcareFinancial servicesManufacturingGovernmentView all industriesView all solutionsResourcesEXPLORE BY TOPICAISoftware DevelopmentDevOpsSecurityView all topicsEXPLORE BY TYPECustomer storiesEvents & webinarsEbooks & reportsBusiness insightsGitHub SkillsSUPPORT & SERVICESDocumentationCustomer supportCommunity forumTrust centerPartnersView all resourcesOpen SourceCOMMUNITYGitHub SponsorsFund open source developersPROGRAMSSecurity LabMaintainer CommunityGitHub StarsArchive ProgramREPOSITORIESTopicsTrendingCollectionsEnterpriseENTERPRISE SOLUTIONSEnterprise platformAI-powered developer platformAVAILABLE ADD-ONSGitHub Advanced SecurityEnterprise-grade security featuresCopilot for BusinessEnterprise-grade AI featuresPremium SupportEnterprise-grade 24/7 supportPricingSearch/Sign inSign upAppearance settings

You signed in with another tab or window. Reload to refresh your session.
You signed out in another tab or window. Reload to refresh your session.
You switched accounts on another tab or window. Reload to refresh your session.

Dismiss alert

kyle-pena-nlp

/

jevchat

Public

Notifications
You must be signed in to change notification settings

Fork
0

Star
1

Code

Issues
0

Pull requests
0

Actions

Projects

Security and quality
0

Insights

Additional navigation options

Code

Issues

Pull requests

Actions

Projects

Security and quality

Insights

mainBranchesTagsGo to fileCodeOpen more actions menuLatest commit History8 Commits8 CommitsFolders and filesNameNameLast commit messageLast commit datedocsdocs  jevchatjevchat  teststests  toolstools  .gitignore.gitignore  README.mdREADME.md  jevchat.tomljevchat.toml  poetry.lockpoetry.lock  pyproject.tomlpyproject.toml  View all filesRepository files navigationREADMEMore itemsjevchat
We know Jev.
jevchat turns that into a chat model. At every step it asks Jev one question:

Given the user's question and the reply written so far, which symbol comes next?

The options are an alphabet plus an option to stop emitting. Jev returns a probability for each
one, and the sampler draws the next symbol from that normalised distribution.
Append, repeat, and stop when STOP is drawn.
There are several alphabets and sampling strategies available.
The idea is for fun, the cost is somewhat impractical, and the results are hilarious.

This was a Claude accelerated experiment. I described the sampling algorithms, strategies, and so on, and it implemented them.
Setup
poetry install
Put your Jev key in .env next to pyproject.toml (git-ignored):
api_key="..."

JEV_API_KEY and TYPESAFE_API_KEY are also accepted. Values in .env win over
exported ones, so editing the file is enough to switch keys.
Use
poetry run jevchat # interactive chat
poetry run jevchat ask "do people need water?"
poetry run jevchat alphabets # what you can sample from
poetry run jevchat bench # compare every mode (table below)
The reply appears as it is sampled, in a panel with a live readout of the
generation rate — symbols/s, characters/s, milliseconds per API call, elapsed
time — and the top few symbols Jev scored at the last step, so you can watch the
distribution the sampler is drawing from.
Ctrl-C cancels. The first press stops generation once the in-flight request
returns and keeps the partial reply; a second press aborts immediately. In chat,
the partial reply stays in the conversation history. ask exits 130 when cancelled.
Chat commands: /help, /alphabet [name], /temp <v>, /stop-bias <v>,
/reset, /stats, /exit.
Modes
Two things are swappable: how the distribution over the next symbol is
obtained (-s/--strategy), and what it is over (-a/--alphabet). Every
combination below is a runnable command.
Strategies
choice asks one question over the whole alphabet. bisect sorts the alphabet
and asks earlier/later yes-no questions until the group is small, then asks one
choice question inside it.
# choice — one question over the whole alphabet (the default)
poetry run jevchat -s choice ask "how many eyes do people have?"

# ...without the re-ordering that cancels Jev's position bias (worst mode)
poetry run jevchat -s choice --no-shuffle-criteria ask "how many eyes do people have?"

# ...averaging 4 re-orderings, sent as 4 parallel questions in one request
poetry run jevchat -s choice --ensemble 4 ask "how many eyes do people have?"

# bisect — earlier/later down to groups of 20, each split asked both ways
poetry run jevchat -s bisect ask "how many eyes do people have?"

# ...cheaper: bigger groups, each split asked once
poetry run jevchat -s bisect --bisect-cutoff 32 --no-bisect-swap ask "how many eyes do people have?"

# buckets — the alphabet split across many questions, each with an OTHER escape.
# The only strategy that can hold more than 255 symbols.
poetry run jevchat -a words1k -s buckets ask "what colour is snow?"
poetry run jevchat -a bpe5k -s buckets --bucket-size 127 ask "what is the capital of france?"

# refine — buckets, then a question over the winners, then a rescored nucleus.
# Twice the probability on the right symbol and ~19x the vocabulary resolved.
poetry run jevchat -a words1k -s refine ask "where do fish live?"
poetry run jevchat -a words1k -s refine --refine-nucleus 6 --refine-rounds 2 ask "…"
Presentations
# hypothesis — options are the resulting texts (the default)
poetry run jevchat -p hypothesis --window 40 ask "what colour is snow?"

# symbol — options are the bare symbols, as the first version of this did
poetry run jevchat -p symbol ask "what colour is snow?"
Beam search
# keep 3 candidate replies alive instead of committing symbol by symbol
poetry run jevchat -b 3 ask "what is the opposite of hot?"
Costs one score per live beam per step. Above width 1, temperature, top_p and
top_k stop applying — beams are ranked by probability, not drawn from.
Alphabets
poetry run jevchat -a lower26 -t 0 ask "what is 2+2?" # a-z and space only
poetry run jevchat -a ascii -t 0 ask "what is 2+2?" # spells anything
poetry run jevchat -a tokens -t 0 ask "do people need water?" # whole words

# these three exceed 255 options, so they need --strategy buckets
poetry run jevchat -a words1k -s buckets -t 0 ask "what colour is grass?"
poetry run jevchat -a bpe2k -s buckets -t 0 ask "where do fish live?"
poetry run jevchat -a bpe5k -s buckets -t 0 ask "what do bees make?"
Combining them
poetry run jevchat -a tokens -s bisect --bisect-cutoff 20 ask "do people need water?"
poetry run jevchat -a ascii -s choice --ensemble 12 -t 0.2 --repetition-penalty 1.0 \
ask "what colour is grass?"
Hypothesis options
There are two ways to ask Jev the same question. Under --presentation symbol the
options are the symbols themselves — 'a', 'i', ' the' — and Jev has to append
the option to the reply in its head before judging it. The instructions used to say
exactly that: "judge grammar and spelling on the concatenation, not on the option
on its own."
Under --presentation hypothesis the options are the resulting texts:
answer_so_far = "The capital of France is Par"

symbol options: 'a' 'i' 's' … STOP
hypothesis options: '…he capital of France is Para'
'…he capital of France is Pari'
'…he capital of France is Pars'
'…he capital of France is Par' <- unchanged: this is STOP

The append is already done, so Jev only ranks finished strings — which is what a
decision model is built for. It is the single largest improvement in the project:
on character alphabets it roughly triples top-1 and doubles the probability mass
landing on the right symbol, for fewer input tokens than symbol options with their
per-option descriptions.
Tests
poetry run pytest
158 tests, all offline — a scripted fake client for the generation loop and an
httpx.MockTransport for the HTTP layer. No API key and no network needed.
jevchat bench is the part that does hit the API.
AboutTurns Jev into a chatbotResourcesReadmeActivityStars1 starWatchers0 watchingForks0 forksReport repositoryReleasesPackagesContributorsLanguages

Footer

© 2026 GitHub, Inc.

Footer navigation

Terms

Privacy

Security

Status

Community

Docs

Contact

Manage cookies

Do not share my personal information

You can’t perform that action at this time.

The project jevchat transforms the Jev model into an interactive chatbot by employing sophisticated sampling algorithms and presentation strategies to explore the model's behavior. The core mechanism involves repeatedly querying Jev based on the current user question and partial reply to determine the next symbol, which is drawn from a probability distribution. This process allows for experimentation with various sampling strategies, alphabet constraints, and presentation formats to observe and manipulate the model's generation process.

The setup requires installing dependencies and configuring an API key, which can be provided via Jev_API_KEY or TYPESAFE_API_KEY, with environment variables taking precedence. The system supports interactive chat commands and specific modes for testing, such as asking for alphabets or benchmarking different methods.

The project defines several advanced sampling strategies that govern how the next symbol is selected. These strategies include choice, which asks a single question over the entire alphabet, and bisect, which sorts the alphabet and asks yes or no questions to narrow down the set before asking a choice question within the resulting group. Bucket strategies divide the alphabet across multiple questions, and the refine strategy further refines the probability distribution through successive questions and nucleus scoring, which can resolve a large vocabulary space.

The system also supports various alphabet constraints, allowing users to restrict the sampling space, such as using lower26 for letters or ascii for any character, or token-based constraints like tokens or bpe5k for word or subword sequences. Combining these constraints with strategies, such as using the bisect strategy with specified cutoffs, demonstrates complex control over the generation distribution.

Presentations are also controlled via two main methods: hypothesis and symbol. The hypothesis method presents the resulting text strings, allowing the model to judge grammar and spelling on the concatenated options. In contrast, the symbol method presents only the individual symbols, requiring the model to internally append the option before judging. The authors note that presenting the full text hypothesis offers a significant improvement, roughly tripling the probability mass landing on the correct symbol for fewer input tokens compared to presenting symbol options with their descriptions.

Beam search is implemented as an alternative generation method, allowing the system to keep several candidate replies alive instead of committing to a single symbol at each step, ranking them by probability rather than drawing exclusively from the distribution. The system also includes command functions for managing the session, such as asking for help, resetting, and displaying statistics. The testing infrastructure includes a scripted client for the generation loop and a mock transport layer to facilitate offline testing of the generation and API interaction.