What is pub-local-jarvis?
pub-local-jarvis is an open-source desktop assistant for Windows that interprets screen content and system playback audio using local multimodal inference. The project presents assistance through a desktop pet, contextual messages, game overlays, and course notes. The intended uses include everyday desktop companionship, game observation, and online learning.
pub-local-jarvis uses the MiniCPM-o 4.5 GGUF model to process text, images, and audio. Screen perception, audio perception, and core inference take place on the user's computer. Ordinary operation does not require uploading these inputs to the cloud, and raw captured material is not retained as long-term memory.
What pub-local-jarvis can do
pub-local-jarvis combines continuous scene awareness with several output formats rather than relying only on manual questions. Contextual assistance can appear beside the desktop pet or over a game, while course mode produces review material. Local memory organizes interpreted activity into timelines and summaries rather than preserving the raw input stream.
- pub-local-jarvis provides brief reminders or comments based on screen content and system audio.
- pub-local-jarvis opens a local-model chat window with Ctrl+M.
- pub-local-jarvis displays game tips and interactions through transparent overlays.
- pub-local-jarvis creates Markdown course notes containing selected screenshots and knowledge summaries.
- pub-local-jarvis organizes activity timelines and daily summaries.
- pub-local-jarvis generates visual schedules when a compatible image-generation API is configured.
- pub-local-jarvis pauses or resumes screen and audio perception when the desktop pet is double-clicked.
How pub-local-jarvis works
pub-local-jarvis can be installed using the executable available from the project's releases page. The installer includes the Python backend and compiled C++ inference runtime, so those development tools do not need separate installation. Model weights are downloaded, resumed if interrupted, and verified automatically during the first launch.
pub-local-jarvis continuously processes screen and system audio inputs, choosing whether to keep observing or produce a brief text response grounded in the current scene. The installer prioritizes its bundled NVIDIA CUDA runtime and adjusts GPU offloading to available video memory. CPU execution provides a fallback when compatible GPU acceleration is unavailable.
Who pub-local-jarvis is for
pub-local-jarvis serves Windows users who want contextual desktop assistance with local processing. The documented workflows address people seeking desktop companionship, players who want observational game commentary, and learners reviewing online courses. MIT-licensed source code and documented build instructions also support developers who want to inspect or build the project.
- pub-local-jarvis supports desktop companionship through contextual reminders and comments.
- pub-local-jarvis supports game companionship without controlling gameplay.
- pub-local-jarvis supports course review through selected screenshots and summarized knowledge points.
- pub-local-jarvis supports daily activity review through local timelines and summaries.
pub-local-jarvis pricing and access
pub-local-jarvis is distributed as an open-source project with source code under the MIT license. The repository provides a release installer and instructions for running from source. Access is therefore through the project download or source code rather than a subscription plan described in the documentation.
pub-local-jarvis strengths and limitations
pub-local-jarvis combines local inference, a bundled desktop installer, and explicit controls over perception. Double-clicking the desktop pet stops further screen and audio acquisition until resumed. Course recording saves selected key frames and organized notes, while optional visual schedule generation sends the daily review and project character reference image to the configured image API.
- pub-local-jarvis requires Windows 10/11, an AVX2-capable x64 processor, and at least 12 GiB of available disk space.
- pub-local-jarvis downloads approximately 6.32 GiB of model files at first launch.
- pub-local-jarvis has noticeably slower perception and text generation during CPU fallback.
- pub-local-jarvis currently provides neither speech playback nor real-time voice conversation.
- pub-local-jarvis observes games and offers tips but does not operate games for the user.
pub-local-jarvis FAQ
What is pub-local-jarvis?
pub-local-jarvis is an open-source Windows desktop assistant that uses a local multimodal model to understand screen content and system audio. Contextual assistance appears through a desktop pet, chat window, game overlays, course notes, and local activity summaries.
Is pub-local-jarvis open source?
pub-local-jarvis is an open-source project whose source code uses the MIT license. The repository provides a Windows installer, source startup instructions, and build documentation. The installer bundles the backend and compiled inference runtime, but model weights download separately at first launch.
Does pub-local-jarvis send screen recordings or audio to the cloud?
pub-local-jarvis processes screen content, system audio, and core inference locally by default, without retaining raw captures as long-term memory. Course mode saves selected screenshots and organized notes. Optional visual schedule generation sends the daily review and project character reference image to a user-configured image API.
What does pub-local-jarvis need to run?
pub-local-jarvis requires Windows 10/11, an AVX2-capable x64 processor, at least 12 GiB of available disk space, and internet access for the initial model download. The installer attempts NVIDIA CUDA acceleration automatically and falls back to slower CPU inference when necessary.
Can pub-local-jarvis talk aloud or control games?
pub-local-jarvis currently focuses on text interaction and does not offer speech playback or real-time voice conversation. Game companionship consists of observing gameplay and providing tips or interactions through transparent overlays, without directly controlling the game or replacing user actions.
Can pub-local-jarvis take notes during online courses?
pub-local-jarvis provides course reminders and generates Markdown notes after a course session. Course notes contain selected key screenshots, summarized knowledge points, and image descriptions, rather than a long-term archive of all captured screen content and audio.
From the community
No reviews yet
Be the first to review this tool. What you write helps the next person choose.




