diff --git a/FAQ.md b/FAQ.md index 1823c7b..194359b 100644 --- a/FAQ.md +++ b/FAQ.md @@ -16,23 +16,24 @@ Yes, you can customize the storage location for NOMAD's content by modifying the Short answer: yes, but we can't do it for you (and we recommend a local drive for best performance). -Long answer: Custom storage paths, mount points, and external drives (like iSCSI or SMB/NFS volumes) **are possible**, but this will be up to your individual configuration on the host before NOMAD starts, and then passed in via the compose.yml as this is a *host-level concern*, not a NOMAD-level concern (see above for details). NOMAD itself can't configure this for you, nor could we support all possible configurations in the install script. +Long answer: Custom storage paths, mount points, and external drives (like iSCSI or SMB/NFS volumes) **are possible**, but this will be up to your individual configuration on the host before NOMAD starts, and then passed in via the compose.yml as this is a _host-level concern_, not a NOMAD-level concern (see above for details). NOMAD itself can't configure this for you, nor could we support all possible configurations in the install script. ## Can I run NOMAD on MAC, WSL2, or a non-Debian-based Distro? **WSL2 on Windows** is community-supported via the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) — covers two install paths (native Docker and Docker Desktop) with all known gotchas documented and empirical performance numbers comparing WSL2 to bare-metal. -**macOS and other non-Debian Linux distros** aren't officially supported. See [Why does NOMAD require a Debian-based OS?](#why-does-nomad-require-a-debian-based-os) for details. +**macOS Apple Silicon** is supported through the macOS installer path. It uses Docker Desktop or Colima for NOMAD's existing services and a native launchd-managed `nomad-mac-ai` worker for MLX/Core ML inference, because those runtimes need the macOS host Metal/Core ML stack. Other non-Debian Linux distros are not officially supported. See [Why does NOMAD require a Debian-based OS?](#why-does-nomad-require-a-debian-based-os) for details. ## Why does NOMAD require a Debian-based OS? -Project N.O.M.A.D. is currently designed to run on Debian-based Linux distributions (with Ubuntu being the recommended distro) because our installation scripts and Docker configurations are optimized for this environment. While it's technically possible to run the Docker containers on other operating systems that support Docker, we have not tested or optimized the installation process for non-Debian-based systems, so we cannot guarantee a smooth experience on those platforms at this time. +Project N.O.M.A.D. was originally designed to run on Debian-based Linux distributions (with Ubuntu being the recommended distro) because the installation scripts and Docker configurations were optimized for that environment. The macOS Apple Silicon installer is now a separate supported path for users who want a self-contained MacBook setup with native MLX/Core ML model execution. Support for other operating systems will come in the future, but because our development resources are limited as a free and open-source project, we needed to prioritize our efforts and focus on a narrower set of supported platforms for the initial release. We chose Debian-based Linux as our starting point because it's widely used, easy to spin up, and provides a stable environment for running Docker containers. For Windows users, the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) provides a community-supported path. Community members have also published guides for other platforms (e.g. macOS) in our Discord community and [Github Discussions](https://github.com/Crosstalk-Solutions/project-nomad/discussions), so if you're interested in running N.O.M.A.D. on a non-Debian-based system, we recommend checking there for any available resources or guides. However, keep in mind that if you choose to run N.O.M.A.D. on a non-Debian-based system, you may encounter issues that we won't be able to provide support for, and you may need to have a higher level of technical expertise to troubleshoot and resolve any problems that arise. ## Can I run NOMAD on a Raspberry Pi or other ARM-based device? + Project N.O.M.A.D. is currently designed to run on x86-64 architecture, and we have not yet tested or optimized it for ARM-based devices like the Raspberry Pi (and have not published any official images for ARM architecture). Support for ARM-based devices is on our roadmap, but our initial focus was on x86-64 hardware due to its widespread use and compatibility with a wide range of applications. @@ -50,6 +51,7 @@ As of March 2026, Project N.O.M.A.D.'s UI is only available in English, and the ## What technologies is NOMAD built with? Project N.O.M.A.D. is built using a combination of technologies, including: + - **Docker:** for containerization of the Command Center and its dependencies - **Node.js & TypeScript:** for the backend of the Command Center, particularly the [AdonisJS](https://adonisjs.com/) framework - **React:** for the frontend of the Command Center, utilizing [Vite](https://vitejs.dev/) and [Inertia.js](https://inertiajs.com/) under the hood @@ -59,6 +61,7 @@ Project N.O.M.A.D. is built using a combination of technologies, including: NOMAD makes use of the Docker-outside-of-Docker ("DooD") pattern, which allows the Command Center to manage and orchestrate other Docker containers on the host machine without needing to run Docker itself inside a container. This approach provides better performance and compatibility with a wider range of host environments while still allowing for powerful container management capabilities through the Command Center's UI. ## Can I run NOMAD if I have existing Docker containers on my machine? + Yes, you can safely run Project N.O.M.A.D. on a machine that already has existing Docker containers. NOMAD is designed to coexist with other Docker containers and will not interfere with them as long as there are no port conflicts or resource constraints. All of NOMAD's containers are prefixed with `nomad_` in their names, so they can be easily identified and managed separately from any other containers you may have running. Just make sure to review the ports that NOMAD's core services (Command Center, MySQL, Redis) use during installation and adjust them if necessary to avoid conflicts with your existing containers. @@ -68,7 +71,8 @@ All of NOMAD's containers are prefixed with `nomad_` in their names, so they can See [What technologies is NOMAD built with?](#what-technologies-is-nomad-built-with) ## Can I use any AI models? -NOMAD by default uses Ollama inside of a docker container to run LLM Models for the AI Assistant. So if you find a model on HuggingFace for example, you won't be able to use that model in NOMAD. The list of available models in the AI Assistant settings (/settings/models) may not show all of the models you are looking for. If you found a model from https://ollama.com/search that you'd like to try and its not in the settings page, you can use a curl command to download the model. + +NOMAD by default uses Ollama inside of a docker container to run LLM Models for the AI Assistant. On Apple Silicon Macs, NOMAD can also use the native Mac AI worker for MLX-compatible models and Core ML packages. MLX LLMs are usable for chat when they can be loaded by `mlx-lm`; Core ML packages are shown separately because generic Core ML artifacts need a model-specific chat adapter before NOMAD can treat them as chat models. If you found a model from https://ollama.com/search that you'd like to try and its not in the settings page, you can use a curl command to download the model. `curl -X POST -H "Content-Type: application/json" -d '{"model":"MODEL_NAME_HERE"}' http://localhost:8080/api/ollama/models` replacing MODEL_NAME_HERE with the model name from whats in the ollama website. ## Do I have to install the AI features in NOMAD? @@ -76,6 +80,7 @@ NOMAD by default uses Ollama inside of a docker container to run LLM Models for No, the AI features in NOMAD (Ollama, Qdrant, custom RAG pipeline, etc.) are all optional and not required to use the core functionality of NOMAD. ## Is NOMAD actually free? Are there any hidden costs? + Yes, Project N.O.M.A.D. is completely free and open-source software licensed under the Apache License 2.0. There are no hidden costs or fees associated with using NOMAD itself, and we don't have any plans to introduce "premium" features or paid tiers. Aside from the cost of the hardware you choose to run it on, there are no costs associated with using NOMAD. @@ -95,6 +100,7 @@ We also encourage community involvement in troubleshooting and resolving issues, We aim to release updates and new features on a regular basis, but the exact timing can vary based on the complexity of the features being developed, the resources available to our volunteer development team, and the feedback and needs of our community. We typically release smaller "patch" versions more frequently to address bugs and make minor improvements, while larger feature releases may take more time to develop and test before they're ready for release. ## I opened a PR to contribute a new feature or fix a bug. How long does it usually take for PRs to be reviewed and merged? + We appreciate all contributions to the project and strive to review and merge pull requests (PRs) as quickly as possible. The time it takes for a PR to be reviewed and merged can vary based on several factors, including the complexity of the changes, the current workload of our maintainers, and the need for any additional testing or revisions. Because NOMAD is still a young project, some PRs (particularly those for new features) may take longer to review and merge as we prioritize building out the core functionality and ensuring stability before adding new features. However, we do our best to provide timely feedback on all PRs and keep contributors informed about the status of their contributions. diff --git a/README.md b/README.md index a7e1100..5d5f5f0 100644 --- a/README.md +++ b/README.md @@ -2,6 +2,7 @@ # Project N.O.M.A.D. + ### Node for Offline Media, Archives, and Data **Knowledge That Never Goes Offline** @@ -17,11 +18,13 @@ Project N.O.M.A.D. is a self-contained, offline-first knowledge and education server packed with critical tools, knowledge, and AI to keep you informed and empowered—anytime, anywhere. ## Installation & Quickstart + Project N.O.M.A.D. can be installed on any Debian-based operating system (we recommend Ubuntu). Installation is completely terminal-based, and all tools and resources are designed to be accessed through the browser, so there's no need for a desktop environment if you'd rather setup N.O.M.A.D. as a "server" and access it through other clients. -*Note: sudo/root privileges are required to run the install script* +_Note: sudo/root privileges are required to run the install script_ ### Quick Install (Debian-based OS Only) + ```bash sudo apt-get update && \ sudo apt-get install -y curl && \ @@ -32,17 +35,33 @@ sudo bash install_nomad.sh Project N.O.M.A.D. is now installed on your device! Open a browser and navigate to `http://localhost:8080` (or `http://DEVICE_IP:8080`) to start exploring! +### macOS Apple Silicon Install + +N.O.M.A.D. can also run on Apple Silicon Macs with Docker Desktop or Colima for the core services and a native launchd-managed Mac AI worker for MLX/Core ML inference. The macOS path stores data under your home directory and keeps MLX/Core ML execution on the host so it can use Metal/Core ML acceleration. + +```bash +curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/macos/install_macos_nomad.sh \ + -o install_macos_nomad.sh && \ +bash install_macos_nomad.sh +``` + +After installation, open `http://localhost:8080`, go to AI Assistant settings, choose **Native MLX** or **Native Core ML**, and place compatible model folders/packages in `~/.project-nomad/mac-ai/models`. + For a complete step-by-step walkthrough (including Ubuntu installation), see the [Installation Guide](https://www.projectnomad.us/install). For Windows users, see the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) — community-supported path covering native Docker and Docker Desktop install routes. ### Advanced Installation + For more control over the installation process, copy and paste the [Docker Compose template](https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/management_compose.yaml) into a `docker-compose.yml` file and customize it to your liking (be sure to replace any placeholders with your actual values). Then, run `docker compose up -d` to start the Command Center and its dependencies. Note: this method is recommended for advanced users only, as it requires familiarity with Docker and manual configuration before starting. ## How It Works + N.O.M.A.D. is a management UI ("Command Center") and API that orchestrates a collection of containerized tools and resources via [Docker](https://www.docker.com/). It handles installation, configuration, and updates for everything — so you don't have to. **Built-in capabilities include:** + - **AI Chat with Knowledge Base** — local AI chat powered by [Ollama](https://ollama.com/) or you can use OpenAI API compatible software such as LM Studio or llama.cpp, with document upload and semantic search (RAG via [Qdrant](https://qdrant.tech/)) - **Information Library** — offline Wikipedia, medical references, ebooks, and more via [Kiwix](https://kiwix.org/) +- **Local Library** — in-app viewing for your own PDFs and local documents, with optional Knowledge Base indexing - **Education Platform** — Khan Academy courses with progress tracking via [Kolibri](https://learningequality.org/kolibri/) - **Offline Maps** — downloadable regional maps via [ProtoMaps](https://protomaps.com) - **Data Tools** — encryption, encoding, and analysis via [CyberChef](https://gchq.github.io/CyberChef/) @@ -54,25 +73,27 @@ N.O.M.A.D. also includes built-in tools like a Wikipedia content selector, ZIM l ## What's Included -| Capability | Powered By | What You Get | -|-----------|-----------|-------------| -| Information Library | Kiwix | Offline Wikipedia, medical references, survival guides, ebooks | -| AI Assistant | Ollama + Qdrant | Built-in chat with document upload and semantic search | -| Education Platform | Kolibri | Khan Academy courses, progress tracking, multi-user support | -| Offline Maps | ProtoMaps | Downloadable regional maps with search and navigation | -| Data Tools | CyberChef | Encryption, encoding, hashing, and data analysis | -| Notes | FlatNotes | Local note-taking with markdown support | -| System Benchmark | Built-in | Hardware scoring, Builder Tags, and community leaderboard | +| Capability | Powered By | What You Get | +| ------------------- | --------------- | -------------------------------------------------------------- | +| Information Library | Kiwix | Offline Wikipedia, medical references, survival guides, ebooks | +| AI Assistant | Ollama + Qdrant | Built-in chat with document upload and semantic search | +| Education Platform | Kolibri | Khan Academy courses, progress tracking, multi-user support | +| Offline Maps | ProtoMaps | Downloadable regional maps with search and navigation | +| Data Tools | CyberChef | Encryption, encoding, hashing, and data analysis | +| Notes | FlatNotes | Local note-taking with markdown support | +| System Benchmark | Built-in | Hardware scoring, Builder Tags, and community leaderboard | ## Device Requirements + While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project N.O.M.A.D. is quite the opposite. To install and run the available AI tools, we highly encourage the use of a beefy, GPU-backed device to make the most of your install. At it's core, however, N.O.M.A.D. is still very lightweight. For a barebones installation of the management application itself, the following minimal specs are required: -*Note: Project N.O.M.A.D. is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The harware listed below is for example/comparison use only* +_Note: Project N.O.M.A.D. is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The harware listed below is for example/comparison use only_ #### Minimum Specs + - Processor: 2 GHz dual-core processor or better - RAM: 4GB system memory - Storage: At least 5 GB free disk space @@ -82,6 +103,7 @@ At it's core, however, N.O.M.A.D. is still very lightweight. For a barebones ins To run LLM's and other included AI tools: #### Optimal Specs + - Processor: AMD Ryzen 7 or Intel Core i7 or better - RAM: 32 GB system memory - Graphics: NVIDIA RTX 3060 or AMD equivalent or better (more VRAM = run larger models) @@ -94,20 +116,24 @@ To run LLM's and other included AI tools: Again, Project N.O.M.A.D. itself is quite lightweight - it's the tools and resources you choose to install with N.O.M.A.D. that will determine the specs required for your unique deployment #### Running AI models on a different host + By default, N.O.M.A.D.'s installer will attempt to setup Ollama on the host when the AI Assistant is installed. However, if you would like to run the AI model on a different host, you can go to the settings of of the AI assistant and input a URL for either an ollama or OpenAI-compatible API server (such as LM Studio). Note that if you use Ollama on a different host, you must start the server with this option `OLLAMA_HOST=0.0.0.0`. Ollama is the preferred way to use the AI assistant as it has features such as model download that OpenAI API does not support. So when using LM Studio for example, you will have to use LM Studio to download models. You are responsible for the setup of Ollama/OpenAI server on the other host. ## Frequently Asked Questions (FAQ) + For answers to common questions about Project N.O.M.A.D., please see our [FAQ](FAQ.md) page. ## About Internet Usage & Privacy + Project N.O.M.A.D. is designed for offline usage. An internet connection is only required during the initial installation (to download dependencies) and if you (the user) decide to download additional tools and resources at a later time. Otherwise, N.O.M.A.D. does not require an internet connection and has ZERO built-in telemetry. To test internet connectivity, N.O.M.A.D. attempts to make a request to Cloudflare's utility endpoint, `https://1.1.1.1/cdn-cgi/trace` and checks for a successful response. ## About Security + By design, Project N.O.M.A.D. is intended to be open and available without hurdles - it includes no authentication. If you decide to connect your device to a local network after install (e.g. for allowing other devices to access it's resources), you can block/open ports to control which services are exposed. **Will authentication be added in the future?** Maybe. It's not currently a priority, but if there's enough demand for it, we may consider building in an optional authentication layer in a future release to support uses cases where multiple users need access to the same instance but with different permission levels (e.g. family use with parental controls, classroom use with teacher/admin accounts, etc.). We have a suggestion for this on our public roadmap, so if this is something you'd like to see, please upvote it here: https://roadmap.projectnomad.us/posts/1/user-authentication-please-build-in-user-auth-with-admin-user-roles @@ -115,6 +141,7 @@ By design, Project N.O.M.A.D. is intended to be open and available without hurdl For now, we recommend using network-level controls to manage access if you're planning to expose your N.O.M.A.D. instance to other devices on a local network. N.O.M.A.D. is not designed to be exposed directly to the internet, and we strongly advise against doing so unless you really know what you're doing, have taken appropriate security measures, and understand the risks involved. ## Contributing + Contributions are welcome and appreciated! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to contribute to the project. ## Community & Resources @@ -131,28 +158,35 @@ Contributions are welcome and appreciated! Please see [CONTRIBUTING.md](CONTRIBU Project N.O.M.A.D. is licensed under the [Apache License 2.0](LICENSE). ## Helper Scripts + Once installed, Project N.O.M.A.D. has a few helper scripts should you ever need to troubleshoot issues or perform maintenance that can't be done through the Command Center. All of these scripts are found in Project N.O.M.A.D.'s install directory, `/opt/project-nomad` ### ###### Start Script - Starts all installed project containers + ```bash sudo bash /opt/project-nomad/start_nomad.sh ``` + ### ###### Stop Script - Stops all installed project containers + ```bash sudo bash /opt/project-nomad/stop_nomad.sh ``` + ### -###### Update Script - Attempts to pull the latest images for the Command Center and its dependencies (i.e. mysql) and recreate the containers. Note: this *only* updates the Command Center containers. It does not update the installable application containers - that should be done through the Command Center UI +###### Update Script - Attempts to pull the latest images for the Command Center and its dependencies (i.e. mysql) and recreate the containers. Note: this _only_ updates the Command Center containers. It does not update the installable application containers - that should be done through the Command Center UI + ```bash sudo bash /opt/project-nomad/update_nomad.sh ``` ###### Uninstall Script - Need to start fresh? Use the uninstall script to make your life easy. Note: this cannot be undone! + ```bash curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/uninstall_nomad.sh -o uninstall_nomad.sh && sudo bash uninstall_nomad.sh ``` diff --git a/admin/app/controllers/local_library_controller.ts b/admin/app/controllers/local_library_controller.ts new file mode 100644 index 0000000..aa26dd9 --- /dev/null +++ b/admin/app/controllers/local_library_controller.ts @@ -0,0 +1,150 @@ +import { EmbedFileJob } from '#jobs/embed_file_job' +import { LocalLibraryService } from '#services/local_library_service' +import { + localLibraryFilenameValidator, + localLibraryRenameValidator, +} from '#validators/local_library' +import { inject } from '@adonisjs/core' +import type { HttpContext } from '@adonisjs/core/http' + +@inject() +export default class LocalLibraryController { + constructor(private localLibraryService: LocalLibraryService) {} + + async page({ inertia }: HttpContext) { + return inertia.render('local-library') + } + + async list({ response }: HttpContext) { + return response.status(200).json({ files: await this.localLibraryService.list() }) + } + + async upload({ request, response }: HttpContext) { + const uploadedFile = request.file('file') + if (!uploadedFile) return response.status(400).json({ message: 'No file uploaded' }) + + try { + await this.localLibraryService.ensureStorage() + const name = this.localLibraryService.sanitizeUploadName(uploadedFile.clientName) + await uploadedFile.move(this.localLibraryService.getStoragePath(), { name, overwrite: true }) + return response + .status(201) + .json({ message: 'File uploaded', files: await this.localLibraryService.list() }) + } catch (error: any) { + if (error.message === 'unsupported_file_type') { + return response.status(400).json({ message: 'Unsupported library file type' }) + } + if (error.message === 'invalid_filename') { + return response.status(400).json({ message: 'Invalid filename' }) + } + throw error + } + } + + async view({ request, response }: HttpContext) { + const payload = await request.validateUsing(localLibraryFilenameValidator) + try { + const file = await this.localLibraryService.stream(payload.params.filename) + response.header('Content-Type', file.contentType) + response.header('Content-Length', String(file.size)) + response.header( + 'Content-Disposition', + `inline; filename="${encodeURIComponent(file.filename)}"` + ) + return response.stream(file.stream) + } catch (error: any) { + if (['not_found', 'invalid_filename'].includes(error.message)) { + return response.status(404).json({ message: 'File not found' }) + } + throw error + } + } + + async download({ request, response }: HttpContext) { + const payload = await request.validateUsing(localLibraryFilenameValidator) + try { + const file = await this.localLibraryService.stream(payload.params.filename) + response.header('Content-Type', file.contentType) + response.header('Content-Length', String(file.size)) + response.header( + 'Content-Disposition', + `attachment; filename="${encodeURIComponent(file.filename)}"` + ) + return response.stream(file.stream) + } catch (error: any) { + if (['not_found', 'invalid_filename'].includes(error.message)) { + return response.status(404).json({ message: 'File not found' }) + } + throw error + } + } + + async preview({ request, response }: HttpContext) { + const payload = await request.validateUsing(localLibraryFilenameValidator) + try { + return response + .status(200) + .json(await this.localLibraryService.preview(payload.params.filename)) + } catch (error: any) { + if (error.message === 'preview_unsupported') { + return response.status(415).json({ message: 'Preview is not supported for this file type' }) + } + if (['not_found', 'invalid_filename'].includes(error.message)) { + return response.status(404).json({ message: 'File not found' }) + } + throw error + } + } + + async rename({ request, response }: HttpContext) { + const payload = await request.validateUsing(localLibraryRenameValidator) + try { + return response.status(200).json({ + file: await this.localLibraryService.rename(payload.params.filename, payload.name), + }) + } catch (error: any) { + if (['not_found', 'invalid_filename'].includes(error.message)) { + return response.status(404).json({ message: 'File not found' }) + } + if (error.message === 'unsupported_file_type') { + return response.status(400).json({ message: 'Unsupported library file type' }) + } + throw error + } + } + + async delete({ request, response }: HttpContext) { + const payload = await request.validateUsing(localLibraryFilenameValidator) + try { + await this.localLibraryService.remove(payload.params.filename) + return response.status(200).json({ message: 'File deleted' }) + } catch (error: any) { + if (['not_found', 'invalid_filename'].includes(error.message) || error.code === 'ENOENT') { + return response.status(404).json({ message: 'File not found' }) + } + throw error + } + } + + async index({ request, response }: HttpContext) { + const payload = await request.validateUsing(localLibraryFilenameValidator) + try { + const file = await this.localLibraryService.stream(payload.params.filename) + const result = await EmbedFileJob.dispatch({ + filePath: file.fullPath, + fileName: file.filename, + fileSize: file.size, + }) + return response.status(202).json({ + message: result.message, + jobId: result.jobId, + alreadyProcessing: !result.created, + }) + } catch (error: any) { + if (['not_found', 'invalid_filename'].includes(error.message)) { + return response.status(404).json({ message: 'File not found' }) + } + throw error + } + } +} diff --git a/admin/app/controllers/mac_ai_controller.ts b/admin/app/controllers/mac_ai_controller.ts new file mode 100644 index 0000000..51e7a46 --- /dev/null +++ b/admin/app/controllers/mac_ai_controller.ts @@ -0,0 +1,62 @@ +import { MacAiService } from '#services/mac_ai_service' +import { RagService } from '#services/rag_service' +import { assertNotCloudMetadataUrl } from '#validators/common' +import { inject } from '@adonisjs/core' +import type { HttpContext } from '@adonisjs/core/http' +import type { MacAiProvider } from '../../types/mac_ai.js' +import Service from '#models/service' +import KVStore from '#models/kv_store' +import { SERVICE_NAMES } from '../../constants/service_names.js' +import logger from '@adonisjs/core/services/logger' + +@inject() +export default class MacAiController { + constructor( + private macAiService: MacAiService, + private ragService: RagService + ) {} + + async status({ response }: HttpContext) { + return response.status(200).json(await this.macAiService.status()) + } + + async configure({ request, response }: HttpContext) { + const provider = request.input('provider', 'ollama') as MacAiProvider + if (!['ollama', 'remote', 'native_mlx', 'native_coreml'].includes(provider)) { + return response.status(400).json({ success: false, message: 'Unsupported AI provider.' }) + } + + const workerUrl = request.input('workerUrl', undefined) as string | null | undefined + if (workerUrl && workerUrl.trim()) { + try { + assertNotCloudMetadataUrl(workerUrl) + } catch (error) { + return response.status(400).json({ + success: false, + message: error instanceof Error ? error.message : 'Invalid native worker URL.', + }) + } + } + + const result = await this.macAiService.configure({ + provider, + workerUrl, + modelRoot: request.input('modelRoot', undefined), + }) + + if (provider === 'native_mlx' || provider === 'native_coreml' || provider === 'remote') { + const aiService = await Service.query().where('service_name', SERVICE_NAMES.OLLAMA).first() + if (aiService) { + aiService.installed = true + aiService.installation_status = 'idle' + await aiService.save() + } + await KVStore.setValue('chat.suggestionsEnabled', false) + this.ragService.discoverNomadDocs().catch((error) => { + logger.error('[MacAiController] Failed to discover Nomad docs:', error) + }) + } + + return response.status(200).json(result) + } +} diff --git a/admin/app/controllers/ollama_controller.ts b/admin/app/controllers/ollama_controller.ts index 3bf9e73..9067a16 100644 --- a/admin/app/controllers/ollama_controller.ts +++ b/admin/app/controllers/ollama_controller.ts @@ -21,7 +21,7 @@ export default class OllamaController { private dockerService: DockerService, private ollamaService: OllamaService, private ragService: RagService - ) { } + ) {} async availableModels({ request }: HttpContext) { const reqData = await request.validateUsing(getAvailableModelsSchema) @@ -83,7 +83,9 @@ export default class OllamaController { 0.3 // Minimum similarity score of 0.3 ) - logger.debug(`[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"`) + logger.debug( + `[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"` + ) // If relevant context is found, inject as a system message with adaptive limits if (relevantDocs.length > 0) { @@ -107,7 +109,10 @@ export default class OllamaController { ) const contextText = trimmedDocs - .map((doc, idx) => `[Context ${idx + 1}] (Relevance: ${(doc.score * 100).toFixed(1)}%)\n${doc.text}`) + .map( + (doc, idx) => + `[Context ${idx + 1}] (Relevance: ${(doc.score * 100).toFixed(1)}%)\n${doc.text}` + ) .join('\n\n') const systemMessage = { @@ -132,13 +137,19 @@ export default class OllamaController { if (estimatedSystemTokens > 3000) { const needed = estimatedSystemTokens + 2048 // leave room for conversation + response numCtx = [8192, 16384, 32768, 65536].find((n) => n >= needed) ?? 65536 - logger.debug(`[OllamaController] Large system prompt (~${estimatedSystemTokens} tokens), requesting num_ctx: ${numCtx}`) + logger.debug( + `[OllamaController] Large system prompt (~${estimatedSystemTokens} tokens), requesting num_ctx: ${numCtx}` + ) } // Check if the model supports "thinking" capability for enhanced response generation // If gpt-oss model, it requires a text param for "think" https://docs.ollama.com/api/chat const thinkingCapability = await this.ollamaService.checkModelHasThinking(reqData.model) - const think: boolean | 'medium' = thinkingCapability ? (reqData.model.startsWith('gpt-oss') ? 'medium' : true) : false + const think: boolean | 'medium' = thinkingCapability + ? reqData.model.startsWith('gpt-oss') + ? 'medium' + : true + : false // Separate sessionId from the Ollama request payload — Ollama rejects unknown fields const { sessionId, ...ollamaRequest } = reqData @@ -154,7 +165,9 @@ export default class OllamaController { } if (reqData.stream) { - logger.debug(`[OllamaController] Initiating streaming response for model: "${reqData.model}" with think: ${think}`) + logger.debug( + `[OllamaController] Initiating streaming response for model: "${reqData.model}" with think: ${think}` + ) // Headers already flushed above const stream = await this.ollamaService.chatStream({ ...ollamaRequest, think, numCtx }) let fullContent = '' @@ -171,9 +184,13 @@ export default class OllamaController { await this.chatService.addMessage(sessionId, 'assistant', fullContent) const messageCount = await this.chatService.getMessageCount(sessionId) if (messageCount <= 2 && userContent) { - this.chatService.generateTitle(sessionId, userContent, fullContent, reqData.model).catch((err) => { - logger.error(`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`) - }) + this.chatService + .generateTitle(sessionId, userContent, fullContent, reqData.model) + .catch((err) => { + logger.error( + `[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}` + ) + }) } } return @@ -186,9 +203,13 @@ export default class OllamaController { await this.chatService.addMessage(sessionId, 'assistant', result.message.content) const messageCount = await this.chatService.getMessageCount(sessionId) if (messageCount <= 2 && userContent) { - this.chatService.generateTitle(sessionId, userContent, result.message.content, reqData.model).catch((err) => { - logger.error(`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`) - }) + this.chatService + .generateTitle(sessionId, userContent, result.message.content, reqData.model) + .catch((err) => { + logger.error( + `[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}` + ) + }) } } @@ -223,7 +244,9 @@ export default class OllamaController { const ollamaService = await Service.query().where('service_name', SERVICE_NAMES.OLLAMA).first() if (!ollamaService) { - return response.status(404).send({ success: false, message: 'Ollama service record not found.' }) + return response + .status(404) + .send({ success: false, message: 'Ollama service record not found.' }) } // Clear path: null or empty URL removes remote config. If a local nomad_ollama container @@ -231,6 +254,7 @@ export default class OllamaController { // the service marked installed. Otherwise fall back to uninstalled. if (!remoteUrl || remoteUrl.trim() === '') { await KVStore.clearValue('ai.remoteOllamaUrl') + await KVStore.setValue('ai.provider', 'ollama') const hasLocalContainer = await this._startLocalOllamaContainerIfExists() ollamaService.installed = hasLocalContainer ollamaService.installation_status = 'idle' @@ -272,6 +296,7 @@ export default class OllamaController { // Save remote URL and mark service as installed await KVStore.setValue('ai.remoteOllamaUrl', remoteUrl.trim()) + await KVStore.setValue('ai.provider', 'remote') ollamaService.installed = true ollamaService.installation_status = 'idle' await ollamaService.save() @@ -300,9 +325,7 @@ export default class OllamaController { private async _stopLocalOllamaContainer(): Promise { try { const containers = await this.dockerService.docker.listContainers({ all: true }) - const ollamaContainer = containers.find((c) => - c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`) - ) + const ollamaContainer = containers.find((c) => c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`)) if (!ollamaContainer || ollamaContainer.State !== 'running') { return } @@ -320,9 +343,7 @@ export default class OllamaController { private async _startLocalOllamaContainerIfExists(): Promise { try { const containers = await this.dockerService.docker.listContainers({ all: true }) - const ollamaContainer = containers.find((c) => - c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`) - ) + const ollamaContainer = containers.find((c) => c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`)) if (!ollamaContainer) { return false } @@ -359,7 +380,7 @@ export default class OllamaController { } } - async installedModels({ }: HttpContext) { + async installedModels({}: HttpContext) { return await this.ollamaService.getModels() } @@ -386,7 +407,7 @@ export default class OllamaController { messages: Message[], model: string ): Promise { - const lastUserMessage = [...messages].reverse().find(msg => msg.role === 'user') + const lastUserMessage = [...messages].reverse().find((msg) => msg.role === 'user') try { // Skip the entire RAG pipeline if there are no documents to search @@ -405,18 +426,19 @@ export default class OllamaController { // entities and topics from earlier turns ("the bars" → "Hershey's bars // chocolate poisoning dog"); without it, embeddings match nothing and // the assistant loses the thread. - const userMessages = recentMessages.filter(msg => msg.role === 'user') + const userMessages = recentMessages.filter((msg) => msg.role === 'user') if (userMessages.length < 2) { return lastUserMessage?.content || null } const conversationContext = recentMessages - .map(msg => { + .map((msg) => { const role = msg.role === 'user' ? 'User' : 'Assistant' // Truncate assistant messages to first 200 chars to keep context manageable - const content = msg.role === 'assistant' - ? msg.content.slice(0, 200) + (msg.content.length > 200 ? '...' : '') - : msg.content + const content = + msg.role === 'assistant' + ? msg.content.slice(0, 200) + (msg.content.length > 200 ? '...' : '') + : msg.content return `${role}: "${content}"` }) .join('\n') diff --git a/admin/app/controllers/settings_controller.ts b/admin/app/controllers/settings_controller.ts index 24cb4ce..6915660 100644 --- a/admin/app/controllers/settings_controller.ts +++ b/admin/app/controllers/settings_controller.ts @@ -64,6 +64,9 @@ export default class SettingsController { const chatSuggestionsEnabled = await KVStore.getValue('chat.suggestionsEnabled') const aiAssistantCustomName = await KVStore.getValue('ai.assistantCustomName') const remoteOllamaUrl = await KVStore.getValue('ai.remoteOllamaUrl') + const aiProvider = await KVStore.getValue('ai.provider') + const macNativeWorkerUrl = await KVStore.getValue('ai.macNativeWorkerUrl') + const macNativeModelRoot = await KVStore.getValue('ai.macNativeModelRoot') const ollamaFlashAttention = await KVStore.getValue('ai.ollamaFlashAttention') return inertia.render('settings/models', { models: { @@ -73,6 +76,9 @@ export default class SettingsController { chatSuggestionsEnabled: chatSuggestionsEnabled ?? false, aiAssistantCustomName: aiAssistantCustomName ?? '', remoteOllamaUrl: remoteOllamaUrl ?? '', + aiProvider: aiProvider ?? (remoteOllamaUrl ? 'remote' : 'ollama'), + macNativeWorkerUrl: macNativeWorkerUrl ?? '', + macNativeModelRoot: macNativeModelRoot ?? '', ollamaFlashAttention: ollamaFlashAttention ?? true, }, }, @@ -111,9 +117,9 @@ export default class SettingsController { } async getSetting({ request, response }: HttpContext) { - const { key } = await getSettingSchema.validate({ key: request.qs().key }); - const value = await KVStore.getValue(key); - return response.status(200).send({ key, value }); + const { key } = await getSettingSchema.validate({ key: request.qs().key }) + const value = await KVStore.getValue(key) + return response.status(200).send({ key, value }) } async updateSetting({ request, response }: HttpContext) { diff --git a/admin/app/services/local_library_service.ts b/admin/app/services/local_library_service.ts new file mode 100644 index 0000000..cc9f78d --- /dev/null +++ b/admin/app/services/local_library_service.ts @@ -0,0 +1,219 @@ +import { ensureDirectoryExists, getFileStatsIfExists, sanitizeFilename } from '../utils/fs.js' +import { createReadStream } from 'node:fs' +import { readFile, readdir, rename, unlink } from 'node:fs/promises' +import app from '@adonisjs/core/services/app' +import JSZip from 'jszip' +import * as cheerio from 'cheerio' +import { extname, resolve } from 'node:path' +import type { + LocalLibraryFile, + LocalLibraryFileType, + LocalLibraryPreviewResponse, +} from '../../types/local_library.js' + +const SUPPORTED_LIBRARY_EXTENSIONS = new Set([ + '.pdf', + '.epub', + '.mobi', + '.azw', + '.azw3', + '.txt', + '.md', + '.rtf', + '.docx', +]) + +const MIME_TYPES: Record = { + '.pdf': 'application/pdf', + '.epub': 'application/epub+zip', + '.mobi': 'application/x-mobipocket-ebook', + '.azw': 'application/vnd.amazon.ebook', + '.azw3': 'application/vnd.amazon.ebook', + '.txt': 'text/plain; charset=utf-8', + '.md': 'text/markdown; charset=utf-8', + '.rtf': 'application/rtf', + '.docx': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document', +} + +export class LocalLibraryService { + public static STORAGE_PATH = 'storage/local_library' + + getStoragePath(): string { + return app.makePath(LocalLibraryService.STORAGE_PATH) + } + + async ensureStorage(): Promise { + await ensureDirectoryExists(this.getStoragePath()) + } + + sanitizeUploadName(name: string): string { + const cleaned = sanitizeFilename(name).replace(/^_+/, '') + if (!cleaned || cleaned === '.' || cleaned === '..') { + throw new Error('invalid_filename') + } + if (!this.isSupportedName(cleaned)) { + throw new Error('unsupported_file_type') + } + return cleaned + } + + isSupportedName(name: string): boolean { + return SUPPORTED_LIBRARY_EXTENSIONS.has(extname(name).toLowerCase()) + } + + resolveLibraryPath(filename: string): string { + const decoded = decodeURIComponent(filename) + if (decoded.includes('/') || decoded.includes('\\') || decoded === '.' || decoded === '..') { + throw new Error('invalid_filename') + } + + const root = resolve(this.getStoragePath()) + const fullPath = resolve(root, decoded) + if (fullPath !== root && fullPath.startsWith(`${root}/`)) { + return fullPath + } + throw new Error('invalid_filename') + } + + detectType(name: string): LocalLibraryFileType { + const ext = extname(name).toLowerCase() + if (ext === '.pdf') return 'pdf' + if (ext === '.epub') return 'epub' + if (['.mobi', '.azw', '.azw3'].includes(ext)) return 'mobi' + if (['.txt', '.md', '.rtf', '.docx'].includes(ext)) return 'text' + return 'unknown' + } + + contentType(name: string): string { + return MIME_TYPES[extname(name).toLowerCase()] ?? 'application/octet-stream' + } + + async list(): Promise { + await this.ensureStorage() + const names = await readdir(this.getStoragePath()) + const files: LocalLibraryFile[] = [] + + for (const name of names) { + if (!this.isSupportedName(name)) continue + const fullPath = this.resolveLibraryPath(name) + const stats = await getFileStatsIfExists(fullPath) + if (!stats) continue + const type = this.detectType(name) + files.push({ + name, + displayName: name, + type, + size: stats.size, + modifiedTime: stats.modifiedTime.toISOString(), + viewUrl: + type === 'pdf' ? `/api/local-library/files/${encodeURIComponent(name)}/view` : null, + downloadUrl: `/api/local-library/files/${encodeURIComponent(name)}/download`, + canPreview: type === 'pdf' || type === 'epub' || type === 'text', + canIndex: type !== 'unknown' && type !== 'mobi', + indexedSource: fullPath, + }) + } + + return files.sort((a, b) => a.displayName.localeCompare(b.displayName)) + } + + async stream(filename: string) { + const fullPath = this.resolveLibraryPath(filename) + const stats = await getFileStatsIfExists(fullPath) + if (!stats) throw new Error('not_found') + return { + stream: createReadStream(fullPath), + fullPath, + filename: decodeURIComponent(filename), + size: stats.size, + contentType: this.contentType(filename), + } + } + + async remove(filename: string): Promise { + const fullPath = this.resolveLibraryPath(filename) + await unlink(fullPath) + } + + async rename(filename: string, requestedName: string): Promise { + const fullPath = this.resolveLibraryPath(filename) + const newName = this.sanitizeUploadName(requestedName) + const newPath = this.resolveLibraryPath(newName) + await rename(fullPath, newPath) + const files = await this.list() + const [file] = files.filter((item) => item.name === newName) + return file + } + + async preview(filename: string): Promise { + const fullPath = this.resolveLibraryPath(filename) + const type = this.detectType(filename) + if (type === 'mobi' || type === 'unknown') { + throw new Error('preview_unsupported') + } + + if (type === 'epub') { + return { + name: decodeURIComponent(filename), + type, + title: decodeURIComponent(filename), + text: await this.extractEpubText(await readFile(fullPath)), + } + } + + if (type === 'text') { + const text = await readFile(fullPath, 'utf-8') + return { name: decodeURIComponent(filename), type, title: decodeURIComponent(filename), text } + } + + return { + name: decodeURIComponent(filename), + type, + title: decodeURIComponent(filename), + text: '', + } + } + + private async extractEpubText(fileBuffer: Buffer): Promise { + const zip = await JSZip.loadAsync(fileBuffer) + const containerXml = await zip.file('META-INF/container.xml')?.async('text') + if (!containerXml) throw new Error('invalid_epub') + + const $container = cheerio.load(containerXml, { xml: true }) + const opfPath = $container('rootfile').attr('full-path') + if (!opfPath) throw new Error('invalid_epub') + + const opfDir = opfPath.includes('/') ? opfPath.substring(0, opfPath.lastIndexOf('/') + 1) : '' + const opfContent = await zip.file(opfPath)?.async('text') + if (!opfContent) throw new Error('invalid_epub') + + const $opf = cheerio.load(opfContent, { xml: true }) + const manifestItems = new Map() + $opf('manifest item').each((_, el) => { + const id = $opf(el).attr('id') + const href = $opf(el).attr('href') + const mediaType = $opf(el).attr('media-type') || '' + if (id && href && (mediaType.includes('html') || mediaType.includes('xml'))) { + manifestItems.set(id, href) + } + }) + + const contentFiles: string[] = [] + $opf('spine itemref').each((_, el) => { + const idref = $opf(el).attr('idref') + if (idref && manifestItems.has(idref)) contentFiles.push(manifestItems.get(idref)!) + }) + if (contentFiles.length === 0) contentFiles.push(...manifestItems.values()) + + const textParts: string[] = [] + for (const href of contentFiles) { + const content = await zip.file(opfDir + href)?.async('text') + if (!content) continue + const $ = cheerio.load(content) + $('script, style').remove() + const text = $('body').text().replace(/\s+/g, ' ').trim() + if (text) textParts.push(text) + } + return textParts.join('\n\n') + } +} diff --git a/admin/app/services/mac_ai_service.ts b/admin/app/services/mac_ai_service.ts new file mode 100644 index 0000000..e0e55a1 --- /dev/null +++ b/admin/app/services/mac_ai_service.ts @@ -0,0 +1,87 @@ +import KVStore from '#models/kv_store' +import type { MacAiProvider, MacNativeStatus } from '../../types/mac_ai.js' + +export const DEFAULT_MAC_NATIVE_WORKER_URL = 'http://host.docker.internal:8765' + +export class MacAiService { + async getProvider(): Promise { + const value = await KVStore.getValue('ai.provider') + if (value === 'remote' || value === 'native_mlx' || value === 'native_coreml') return value + return 'ollama' + } + + async getWorkerUrl(): Promise { + const value = await KVStore.getValue('ai.macNativeWorkerUrl') + return typeof value === 'string' && value.trim() + ? value.trim().replace(/\/$/, '') + : DEFAULT_MAC_NATIVE_WORKER_URL + } + + async configure(input: { + provider: MacAiProvider + workerUrl?: string | null + modelRoot?: string | null + }): Promise<{ success: true; message: string }> { + await KVStore.setValue('ai.provider', input.provider) + if (input.workerUrl !== undefined) { + if (input.workerUrl && input.workerUrl.trim()) { + await KVStore.setValue('ai.macNativeWorkerUrl', input.workerUrl.trim().replace(/\/$/, '')) + } else { + await KVStore.clearValue('ai.macNativeWorkerUrl') + } + } + if (input.modelRoot !== undefined) { + if (input.modelRoot && input.modelRoot.trim()) { + await KVStore.setValue('ai.macNativeModelRoot', input.modelRoot.trim()) + } else { + await KVStore.clearValue('ai.macNativeModelRoot') + } + } + return { success: true, message: 'AI provider settings saved.' } + } + + async status(): Promise { + const provider = await this.getProvider() + const url = await this.getWorkerUrl() + const nativeProvider = provider === 'native_mlx' || provider === 'native_coreml' + if (!nativeProvider) { + return { configured: false, connected: false, active: false, provider, url, models: [] } + } + + try { + const [healthResponse, modelResponse] = await Promise.all([ + fetch(`${url}/health`, { signal: AbortSignal.timeout(2500) }), + fetch(`${url}/v1/models`, { signal: AbortSignal.timeout(2500) }), + ]) + const modelsPayload: any = modelResponse.ok ? await modelResponse.json() : { data: [] } + const models = Array.isArray(modelsPayload?.data) + ? modelsPayload.data.map((model: any) => ({ + id: String(model.id), + name: String(model.name || model.id), + backend: model.backend === 'coreml' ? 'coreml' : 'mlx', + path: typeof model.path === 'string' ? model.path : undefined, + usableForChat: Boolean(model.usable_for_chat ?? model.usableForChat), + notes: typeof model.notes === 'string' ? model.notes : undefined, + })) + : [] + return { + configured: true, + connected: healthResponse.ok, + active: healthResponse.ok, + provider, + url, + models, + } + } catch (error) { + return { + configured: true, + connected: false, + active: false, + provider, + url, + models: [], + message: error instanceof Error ? error.message : 'Native Mac AI worker is unreachable.', + } + } + } +} diff --git a/admin/app/services/ollama_service.ts b/admin/app/services/ollama_service.ts index 526407c..6e54693 100644 --- a/admin/app/services/ollama_service.ts +++ b/admin/app/services/ollama_service.ts @@ -1,6 +1,9 @@ import { inject } from '@adonisjs/core' import OpenAI from 'openai' -import type { ChatCompletionChunk, ChatCompletionMessageParam } from 'openai/resources/chat/completions.js' +import type { + ChatCompletionChunk, + ChatCompletionMessageParam, +} from 'openai/resources/chat/completions.js' import type { Stream } from 'openai/streaming.js' import { NomadOllamaModel } from '../../types/ollama.js' import { EMBEDDING_MODEL_NAME, FALLBACK_RECOMMENDED_OLLAMA_MODELS } from '../../constants/ollama.js' @@ -16,6 +19,7 @@ import { BROADCAST_CHANNELS } from '../../constants/broadcast.js' import env from '#start/env' import { NOMAD_API_DEFAULT_BASE_URL } from '../../constants/misc.js' import KVStore from '#models/kv_store' +import { DEFAULT_MAC_NATIVE_WORKER_URL } from './mac_ai_service.js' const NOMAD_MODELS_API_PATH = '/api/v1/ollama/models' const MODELS_CACHE_FILE = path.join(process.cwd(), 'storage', 'ollama-models-cache.json') @@ -53,17 +57,27 @@ export class OllamaService { private baseUrl: string | null = null private initPromise: Promise | null = null private isOllamaNative: boolean | null = null - private activeDownloads: Map> = new Map() + private activeDownloads: Map< + string, + Promise<{ success: boolean; message: string; retryable?: boolean }> + > = new Map() constructor() {} private async _initialize() { if (!this.initPromise) { this.initPromise = (async () => { - // Check KVStore for a custom base URL (remote Ollama, LM Studio, llama.cpp, etc.) + // Check KVStore for a custom provider before falling back to local Ollama. + const provider = (await KVStore.getValue('ai.provider')) as string | null const customUrl = (await KVStore.getValue('ai.remoteOllamaUrl')) as string | null - if (customUrl && customUrl.trim()) { + const nativeUrl = (await KVStore.getValue('ai.macNativeWorkerUrl')) as string | null + + if (provider === 'native_mlx' || provider === 'native_coreml') { + this.baseUrl = (nativeUrl?.trim() || DEFAULT_MAC_NATIVE_WORKER_URL).replace(/\/$/, '') + this.isOllamaNative = false + } else if ((provider === 'remote' || !provider) && customUrl && customUrl.trim()) { this.baseUrl = customUrl.trim().replace(/\/$/, '') + this.isOllamaNative = false } else { // Fall back to the local Ollama container managed by Docker const dockerService = new (await import('./docker_service.js')).DockerService() @@ -111,7 +125,9 @@ export class OllamaService { // Deduplicate concurrent downloads of the same model const existing = this.activeDownloads.get(model) if (existing) { - logger.info(`[OllamaService] Download already in progress for "${model}", waiting on existing download.`) + logger.info( + `[OllamaService] Download already in progress for "${model}", waiting on existing download.` + ) return existing } @@ -222,9 +238,8 @@ export class OllamaService { aggTotal += total } - const percent = aggTotal > 0 - ? parseFloat(((aggCompleted / aggTotal) * 100).toFixed(2)) - : 0 + const percent = + aggTotal > 0 ? parseFloat(((aggCompleted / aggTotal) * 100).toFixed(2)) : 0 // Throttle broadcasts. Always call the progressCallback though — the worker // uses it to update job state in Redis, which should reflect the latest view. @@ -274,9 +289,7 @@ export class OllamaService { } const errorMessage = error instanceof Error ? error.message : String(error) - logger.error( - `[OllamaService] Failed to download model "${model}": ${errorMessage}` - ) + logger.error(`[OllamaService] Failed to download model "${model}": ${errorMessage}`) // Check for version mismatch (Ollama 412 response) const isVersionMismatch = errorMessage.includes('newer version of Ollama') @@ -364,7 +377,9 @@ export class OllamaService { params.num_ctx = chatRequest.numCtx } - const stream = (await this.openai.chat.completions.create(params)) as unknown as Stream + const stream = (await this.openai.chat.completions.create( + params + )) as unknown as Stream // Returns how many trailing chars of `text` could be the start of `tag` function partialTagSuffix(tag: string, text: string): number { @@ -424,7 +439,9 @@ export class OllamaService { content: parsedContent, thinking: nativeThinking + parsedThinking, }, - done: chunk.choices[0]?.finish_reason !== null && chunk.choices[0]?.finish_reason !== undefined, + done: + chunk.choices[0]?.finish_reason !== null && + chunk.choices[0]?.finish_reason !== undefined, } } } @@ -442,7 +459,10 @@ export class OllamaService { { model: modelName }, { timeout: 5000 } ) - return Array.isArray(response.data?.capabilities) && response.data.capabilities.includes('thinking') + return ( + Array.isArray(response.data?.capabilities) && + response.data.capabilities.includes('thinking') + ) } catch { // Non-Ollama backends don't expose /api/show — assume no thinking support return false @@ -465,7 +485,10 @@ export class OllamaService { logger.error( `[OllamaService] Failed to delete model "${modelName}": ${error instanceof Error ? error.message : error}` ) - return { success: false, message: 'Failed to delete model. This may not be an Ollama backend.' } + return { + success: false, + message: 'Failed to delete model. This may not be an Ollama backend.', + } } } @@ -581,9 +604,7 @@ export class OllamaService { const models: Array<{ name?: string; size_vram?: number }> = response.data?.models ?? [] // Match any loaded model whose name signals it's an embedding model. // nomic-embed-text, mxbai-embed-large, snowflake-arctic-embed, etc. all follow this convention. - return models.some( - (m) => m.name?.toLowerCase().includes('embed') && (m.size_vram ?? 0) > 0 - ) + return models.some((m) => m.name?.toLowerCase().includes('embed') && (m.size_vram ?? 0) > 0) } catch (err: any) { // /api/ps unreachable (Ollama down, non-native backend, etc.) — fail closed: assume CPU, // which means we'll pace. Better to over-pace than risk box-killing CPU saturation. diff --git a/admin/app/services/rag_service.ts b/admin/app/services/rag_service.ts index dcbbd25..76db28a 100644 --- a/admin/app/services/rag_service.ts +++ b/admin/app/services/rag_service.ts @@ -4,7 +4,14 @@ import { inject } from '@adonisjs/core' import logger from '@adonisjs/core/services/logger' import { TokenChunker } from '@chonkiejs/core' import sharp from 'sharp' -import { deleteFileIfExists, determineFileType, getFile, getFileStatsIfExists, listDirectoryContentsRecursive, ZIM_STORAGE_PATH } from '../utils/fs.js' +import { + deleteFileIfExists, + determineFileType, + getFile, + getFileStatsIfExists, + listDirectoryContentsRecursive, + ZIM_STORAGE_PATH, +} from '../utils/fs.js' import { PDFParse } from 'pdf-parse' import { createWorker } from 'tesseract.js' import { fromBuffer } from 'pdf2pic' @@ -25,7 +32,13 @@ import type { KbIngestStateValue } from '../../types/kb_ingest_state.js' import { ZIMExtractionService } from './zim_extraction_service.js' import { ZIM_BATCH_SIZE } from '../../constants/zim_extraction.js' import { EMBEDDING_MODEL_NAME } from '../../constants/ollama.js' -import { ProcessAndEmbedFileResponse, ProcessZIMFileResponse, RAGResult, RerankedRAGResult } from '../../types/rag.js' +import { + ProcessAndEmbedFileResponse, + ProcessZIMFileResponse, + RAGResult, + RerankedRAGResult, +} from '../../types/rag.js' +import { LocalLibraryService } from './local_library_service.js' export type EmbedSingleFileFailureCode = | 'not_found' @@ -51,8 +64,8 @@ export class RagService { public static TARGET_TOKENS_PER_CHUNK = 1500 // Target 1500 tokens per chunk for embedding public static PREFIX_TOKEN_BUDGET = 10 // Reserve ~10 tokens for prefixes public static CHAR_TO_TOKEN_RATIO = 2 // Conservative chars-per-token estimate; technical docs - // (numbers, symbols, abbreviations) tokenize denser - // than plain prose (~3), so 2 avoids context overflows + // (numbers, symbols, abbreviations) tokenize denser + // than plain prose (~3), so 2 avoids context overflows // Nomic Embed Text v1.5 uses task-specific prefixes for optimal performance public static SEARCH_DOCUMENT_PREFIX = 'search_document: ' public static SEARCH_QUERY_PREFIX = 'search_query: ' @@ -61,14 +74,16 @@ export class RagService { constructor( private dockerService: DockerService, private ollamaService: OllamaService - ) { } + ) {} private async _initializeQdrantClient() { if (!this.qdrantInitPromise) { this.qdrantInitPromise = (async () => { const qdrantUrl = await this.dockerService.getServiceURL(SERVICE_NAMES.QDRANT) if (!qdrantUrl) { - throw new Error('Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.') + throw new Error( + 'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.' + ) } this.qdrant = new QdrantClient({ url: qdrantUrl }) })().catch((err) => { @@ -90,7 +105,8 @@ export class RagService { this.qdrantInitPromise = null return { online: false, - message: 'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.', + message: + 'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.', } } } @@ -142,15 +158,17 @@ export class RagService { * - Control characters (except newlines, tabs, and carriage returns) */ private sanitizeText(text: string): string { - return text - // Null bytes - .replace(/\x00/g, '') - // Problematic control characters (keep \n, \r, \t) - .replace(/[\x01-\x08\x0B-\x0C\x0E-\x1F\x7F]/g, '') - // Invalid Unicode surrogates - .replace(/[\uD800-\uDFFF]/g, '') - // Trim extra whitespace - .trim() + return ( + text + // Null bytes + .replace(/\x00/g, '') + // Problematic control characters (keep \n, \r, \t) + .replace(/[\x01-\x08\x0B-\x0C\x0E-\x1F\x7F]/g, '') + // Invalid Unicode surrogates + .replace(/[\uD800-\uDFFF]/g, '') + // Trim extra whitespace + .trim() + ) } /** @@ -204,33 +222,33 @@ export class RagService { * TODO: We could probably move this to a separate QueryPreprocessor class if it grows more complex, but for now it's manageable here. */ private static QUERY_EXPANSION_DICTIONARY: Record = { - 'bob': 'bug out bag', - 'bov': 'bug out vehicle', - 'bol': 'bug out location', - 'edc': 'every day carry', - 'mre': 'meal ready to eat', - 'shtf': 'shit hits the fan', - 'teotwawki': 'the end of the world as we know it', - 'opsec': 'operational security', - 'ifak': 'individual first aid kit', - 'ghb': 'get home bag', - 'ghi': 'get home in', - 'wrol': 'without rule of law', - 'emp': 'electromagnetic pulse', - 'ham': 'ham amateur radio', - 'nbr': 'nuclear biological radiological', - 'cbrn': 'chemical biological radiological nuclear', - 'sar': 'search and rescue', - 'comms': 'communications radio', - 'fifo': 'first in first out', - 'mylar': 'mylar bag food storage', - 'paracord': 'paracord 550 cord', - 'ferro': 'ferro rod fire starter', - 'bivvy': 'bivvy bivy emergency shelter', - 'bdu': 'battle dress uniform', - 'gmrs': 'general mobile radio service', - 'frs': 'family radio service', - 'nbc': 'nuclear biological chemical', + bob: 'bug out bag', + bov: 'bug out vehicle', + bol: 'bug out location', + edc: 'every day carry', + mre: 'meal ready to eat', + shtf: 'shit hits the fan', + teotwawki: 'the end of the world as we know it', + opsec: 'operational security', + ifak: 'individual first aid kit', + ghb: 'get home bag', + ghi: 'get home in', + wrol: 'without rule of law', + emp: 'electromagnetic pulse', + ham: 'ham amateur radio', + nbr: 'nuclear biological radiological', + cbrn: 'chemical biological radiological nuclear', + sar: 'search and rescue', + comms: 'communications radio', + fifo: 'first in first out', + mylar: 'mylar bag food storage', + paracord: 'paracord 550 cord', + ferro: 'ferro rod fire starter', + bivvy: 'bivvy bivy emergency shelter', + bdu: 'battle dress uniform', + gmrs: 'general mobile radio service', + frs: 'family radio service', + nbc: 'nuclear biological chemical', } private preprocessQuery(query: string): string { @@ -311,7 +329,9 @@ export class RagService { // TokenChunker uses character-based tokenization (1 char = 1 token) // We need to convert our embedding model's token counts to character counts // since nomic-embed-text tokenizer uses ~3 chars per token - const targetCharsPerChunk = Math.floor(RagService.TARGET_TOKENS_PER_CHUNK * RagService.CHAR_TO_TOKEN_RATIO) + const targetCharsPerChunk = Math.floor( + RagService.TARGET_TOKENS_PER_CHUNK * RagService.CHAR_TO_TOKEN_RATIO + ) const overlapChars = Math.floor(150 * RagService.CHAR_TO_TOKEN_RATIO) const chunker = await TokenChunker.create({ @@ -359,9 +379,14 @@ export class RagService { const batchStart = batchIdx * batchSize const batch = prefixedChunks.slice(batchStart, batchStart + batchSize) - logger.debug(`[RAG] Embedding batch ${batchIdx + 1}/${totalBatches} (${batch.length} chunks)`) + logger.debug( + `[RAG] Embedding batch ${batchIdx + 1}/${totalBatches} (${batch.length} chunks)` + ) - const response = await this.ollamaService.embed(this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME, batch) + const response = await this.ollamaService.embed( + this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME, + batch + ) embeddings.push(...response.embeddings) @@ -392,13 +417,14 @@ export class RagService { logger.debug(`[RAG] Extracted keywords for chunk ${index}: [${allKeywords.join(', ')}]`) if (structuralKeywords.length > 0) { - logger.debug(`[RAG] - Structural: [${structuralKeywords.join(', ')}], Content: [${contentKeywords.join(', ')}]`) + logger.debug( + `[RAG] - Structural: [${structuralKeywords.join(', ')}], Content: [${contentKeywords.join(', ')}]` + ) } // Sanitize source metadata as well - const sanitizedSource = typeof metadata.source === 'string' - ? this.sanitizeText(metadata.source) - : 'unknown' + const sanitizedSource = + typeof metadata.source === 'string' ? this.sanitizeText(metadata.source) : 'unknown' return { id: randomUUID(), // qdrant requires either uuid or unsigned int @@ -411,7 +437,7 @@ export class RagService { keywords: allKeywords.join(' '), // store as space-separated string for text search char_count: sanitizedText.length, created_at: timestamp, - source: sanitizedSource + source: sanitizedSource, }, } }) @@ -661,9 +687,7 @@ export class RagService { }) // If no spine found, fall back to all manifest items - const contentFiles = spineOrder.length > 0 - ? spineOrder - : Array.from(manifestItems.values()) + const contentFiles = spineOrder.length > 0 ? spineOrder : Array.from(manifestItems.values()) // Extract text from each content file in order const textParts: string[] = [] @@ -682,7 +706,9 @@ export class RagService { } const fullText = textParts.join('\n\n') - logger.debug(`[RAG] EPUB extracted ${textParts.length} chapters, ${fullText.length} characters total`) + logger.debug( + `[RAG] EPUB extracted ${textParts.length} chapters, ${fullText.length} characters total` + ) return fullText } @@ -693,12 +719,19 @@ export class RagService { onProgress?: (percent: number) => Promise ): Promise<{ success: boolean; message: string; chunks?: number }> { if (!extractedText || extractedText.trim().length === 0) { - return { success: false, message: 'Process completed succesfully, but no text was found to embed.' } + return { + success: false, + message: 'Process completed succesfully, but no text was found to embed.', + } } - const embedResult = await this.embedAndStoreText(extractedText, { - source: filepath - }, onProgress) + const embedResult = await this.embedAndStoreText( + extractedText, + { + source: filepath, + }, + onProgress + ) if (!embedResult) { return { success: false, message: 'Failed to embed and store the extracted text.' } @@ -719,7 +752,7 @@ export class RagService { /** * Main pipeline to process and embed an uploaded file into the RAG knowledge base. * This includes text extraction, chunking, embedding, and storing in Qdrant. - * + * * Orchestrates file type detection and delegates to specialized processors. * For ZIM files, supports batch processing via batchOffset parameter. */ @@ -733,7 +766,7 @@ export class RagService { const fileType = determineFileType(filepath) logger.debug(`[RAG] Processing file: ${filepath} (detected type: ${fileType})`) - if (fileType === 'unknown') { + if (fileType === 'unknown' || fileType === 'mobi') { return { success: false, message: 'Unsupported file type.' } } @@ -771,12 +804,15 @@ export class RagService { // Extraction done — scale remaining embedding progress from 15% to 100% if (onProgress) await onProgress(15) - const scaledProgress = onProgress - ? (p: number) => onProgress(15 + p * 0.85) - : undefined + const scaledProgress = onProgress ? (p: number) => onProgress(15 + p * 0.85) : undefined // Embed extracted text and cleanup - return await this.embedTextAndCleanup(extractedText, filepath, deleteAfterEmbedding, scaledProgress) + return await this.embedTextAndCleanup( + extractedText, + filepath, + deleteAfterEmbedding, + scaledProgress + ) } catch (error) { logger.error('[RAG] Error processing and embedding file:', error) return { success: false, message: 'Error processing and embedding file.' } @@ -822,9 +858,7 @@ export class RagService { allModels.find((model) => model.name.toLowerCase().includes('nomic-embed-text')) if (!embeddingModel) { - logger.warn( - `[RAG] ${EMBEDDING_MODEL_NAME} not found. Cannot perform similarity search.` - ) + logger.warn(`[RAG] ${EMBEDDING_MODEL_NAME} not found. Cannot perform similarity search.`) this.embeddingModelVerified = false return [] } @@ -855,7 +889,10 @@ export class RagService { return [] } - const response = await this.ollamaService.embed(this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME, [prefixedQuery]) + const response = await this.ollamaService.embed( + this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME, + [prefixedQuery] + ) // Perform semantic search with a higher limit to enable reranking const searchLimit = limit * 3 // Get more results for reranking @@ -1022,9 +1059,7 @@ export class RagService { * Uses greedy selection: for each result, apply 0.85^n penalty where n is the * number of results already selected from the same source. */ - private applySourceDiversity( - results: Array - ) { + private applySourceDiversity(results: Array) { const sourceCounts = new Map() const DIVERSITY_PENALTY = 0.85 @@ -1054,7 +1089,10 @@ export class RagService { */ public async hasDocuments(): Promise { try { - await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION) + await this._ensureCollection( + RagService.CONTENT_COLLECTION_NAME, + RagService.EMBEDDING_DIMENSION + ) const collectionInfo = await this.qdrant!.getCollection(RagService.CONTENT_COLLECTION_NAME) return (collectionInfo.points_count ?? 0) > 0 } catch { @@ -1103,7 +1141,11 @@ export class RagService { // the vector store?". Both are needed to render the KB UI honestly. const stateByPath = new Map() try { - const stateRows = await KbIngestState.query().select('file_path', 'state', 'chunks_embedded') + const stateRows = await KbIngestState.query().select( + 'file_path', + 'state', + 'chunks_embedded' + ) for (const row of stateRows) { sources.add(row.file_path) stateByPath.set(row.file_path, { @@ -1231,9 +1273,7 @@ export class RagService { const chunksInQdrant = chunksBySource.get(source) ?? 0 const fileName = source.split(/[/\\]/).pop() ?? source const expectedChunks = - fileSizeBytes > 0 - ? await KbRatioRegistry.estimateChunks(fileName, fileSizeBytes) - : null + fileSizeBytes > 0 ? await KbRatioRegistry.estimateChunks(fileName, fileSizeBytes) : null const warnings = decideWarnings({ fileSizeBytes, chunksInQdrant, expectedChunks }) if (warnings.length > 0) out[source] = warnings @@ -1267,17 +1307,19 @@ export class RagService { logger.info(`[RAG] Deleted all points for source: ${source}`) /** Delete the physical file only if it lives inside the uploads directory. - * resolve() normalises path traversal sequences (e.g. "/../..") before the - * check to prevent path traversal vulns - * The trailing sep is to ensure a prefix like "kb_uploads_{something_incorrect}" can't slip through. - */ + * resolve() normalises path traversal sequences (e.g. "/../..") before the + * check to prevent path traversal vulns + * The trailing sep is to ensure a prefix like "kb_uploads_{something_incorrect}" can't slip through. + */ const uploadsAbsPath = join(process.cwd(), RagService.UPLOADS_STORAGE_PATH) const resolvedSource = resolve(source) if (resolvedSource.startsWith(uploadsAbsPath + sep)) { await deleteFileIfExists(resolvedSource) logger.info(`[RAG] Deleted uploaded file from disk: ${resolvedSource}`) } else { - logger.warn(`[RAG] File was removed from knowledge base but doesn't live in Nomad's uploads directory, so it can't be safely removed. Skipping deletion of physical file...`) + logger.warn( + `[RAG] File was removed from knowledge base but doesn't live in Nomad's uploads directory, so it can't be safely removed. Skipping deletion of physical file...` + ) } // Drop the ingest state row last so the file disappears entirely. Without @@ -1300,7 +1342,10 @@ export class RagService { const alreadyEmbeddedRaw = await KVStore.getValue('rag.docsEmbedded') if (alreadyEmbeddedRaw && !force) { logger.info('[RAG] Nomad docs have already been discovered and queued. Skipping.') - return { success: true, message: 'Nomad docs have already been discovered and queued. Skipping.' } + return { + success: true, + message: 'Nomad docs have already been discovered and queued. Skipping.', + } } const filesToEmbed: Array<{ path: string; source: string }> = [] @@ -1332,17 +1377,17 @@ export class RagService { }) logger.info(`[RAG] Successfully dispatched job for ${fileInfo.source}`) } catch (fileError) { - logger.error( - `[RAG] Error dispatching job for file ${fileInfo.source}:`, - fileError - ) + logger.error(`[RAG] Error dispatching job for file ${fileInfo.source}:`, fileError) } } // Update KV store to mark docs as discovered so we don't redo this unnecessarily await KVStore.setValue('rag.docsEmbedded', true) - return { success: true, message: `Nomad docs discovery completed. Dispatched ${filesToEmbed.length} embedding jobs.` } + return { + success: true, + message: `Nomad docs discovery completed. Dispatched ${filesToEmbed.length} embedding jobs.`, + } } catch (error) { logger.error('Error discovering Nomad docs:', error) return { success: false, message: 'Error discovering Nomad docs.' } @@ -1350,18 +1395,20 @@ export class RagService { } /** - * Walk kb_uploads and zim storage directories, returning the full path of + * Walk kb_uploads, local library, and zim storage directories, returning the full path of * every embeddable file. Non-embeddable types (e.g. kiwix-library.xml) are * filtered out so they aren't dispatched only to fail with "Unsupported file * type" and retry on every sync. */ private async _discoverKbFiles(): Promise { const KB_UPLOADS_PATH = join(process.cwd(), RagService.UPLOADS_STORAGE_PATH) + const LOCAL_LIBRARY_PATH = join(process.cwd(), LocalLibraryService.STORAGE_PATH) const ZIM_PATH = join(process.cwd(), ZIM_STORAGE_PATH) const filesInStorage: string[] = [] for (const [label, dirPath] of [ [RagService.UPLOADS_STORAGE_PATH, KB_UPLOADS_PATH] as const, + [LocalLibraryService.STORAGE_PATH, LOCAL_LIBRARY_PATH] as const, [ZIM_STORAGE_PATH, ZIM_PATH] as const, ]) { try { @@ -1461,7 +1508,8 @@ export class RagService { return { success: false, code: 'inflight', - message: 'A job for this file is already in progress. Wait for it to finish before re-queuing.', + message: + 'A job for this file is already in progress. Wait for it to finish before re-queuing.', } } @@ -1498,10 +1546,7 @@ export class RagService { * by reembedAll() where the file must remain so it can be re-ingested. */ private async _deletePointsBySource(source: string): Promise { - await this._ensureCollection( - RagService.CONTENT_COLLECTION_NAME, - RagService.EMBEDDING_DIMENSION - ) + await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION) await this.qdrant!.delete(RagService.CONTENT_COLLECTION_NAME, { filter: { must: [{ key: 'source', match: { value: source } }] }, }) @@ -1518,7 +1563,10 @@ export class RagService { const { QueueService } = await import('#services/queue_service') const queue = QueueService.getInstance().getQueue(EmbedFileJob.queue) const counts = await queue.getJobCounts('waiting', 'active', 'delayed', 'paused') - return (counts.waiting || 0) + (counts.active || 0) + (counts.delayed || 0) + (counts.paused || 0) > 0 + return ( + (counts.waiting || 0) + (counts.active || 0) + (counts.delayed || 0) + (counts.paused || 0) > + 0 + ) } /** @@ -1684,7 +1732,8 @@ export class RagService { if (await this._hasInflightEmbedJobs()) { return { success: false, - message: 'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a bulk re-embed.', + message: + 'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a bulk re-embed.', } } @@ -1717,7 +1766,10 @@ export class RagService { try { await this._deletePointsBySource(filePath) } catch (err) { - logger.error(`[RAG] Failed to delete prior points for ${filePath}; skipping dispatch:`, err) + logger.error( + `[RAG] Failed to delete prior points for ${filePath}; skipping dispatch:`, + err + ) failedPaths.push(filePath) continue } @@ -1732,7 +1784,10 @@ export class RagService { } catch (fileError) { // Old points already deleted but the new job never made it onto the // queue. Logged + surfaced so an operator can rerun a sync. - logger.error(`[RAG] Re-embed dispatch failed for ${filePath} after delete; file is now unindexed until next sync:`, fileError) + logger.error( + `[RAG] Re-embed dispatch failed for ${filePath} after delete; file is now unindexed until next sync:`, + fileError + ) failedPaths.push(filePath) } } @@ -1781,7 +1836,8 @@ export class RagService { if (await this._hasInflightEmbedJobs()) { return { success: false, - message: 'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a reset.', + message: + 'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a reset.', } } diff --git a/admin/app/utils/fs.ts b/admin/app/utils/fs.ts index d6cb5e0..ca79ba4 100644 --- a/admin/app/utils/fs.ts +++ b/admin/app/utils/fs.ts @@ -174,7 +174,9 @@ export function matchesDevice(fsPath: string, deviceName: string): boolean { return false } -export function determineFileType(filename: string): 'image' | 'pdf' | 'text' | 'epub' | 'zim' | 'unknown' { +export function determineFileType( + filename: string +): 'image' | 'pdf' | 'text' | 'epub' | 'mobi' | 'zim' | 'unknown' { const ext = path.extname(filename).toLowerCase() if (['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp'].includes(ext)) { return 'image' @@ -184,6 +186,8 @@ export function determineFileType(filename: string): 'image' | 'pdf' | 'text' | return 'text' } else if (ext === '.epub') { return 'epub' + } else if (['.mobi', '.azw', '.azw3'].includes(ext)) { + return 'mobi' } else if (ext === '.zim') { return 'zim' } else { @@ -198,4 +202,4 @@ export function determineFileType(filename: string): 'image' | 'pdf' | 'text' | */ export function sanitizeFilename(filename: string): string { return filename.replace(/[^a-zA-Z0-9._-]/g, '_') -} \ No newline at end of file +} diff --git a/admin/app/validators/local_library.ts b/admin/app/validators/local_library.ts new file mode 100644 index 0000000..99a87d1 --- /dev/null +++ b/admin/app/validators/local_library.ts @@ -0,0 +1,18 @@ +import vine from '@vinejs/vine' + +export const localLibraryFilenameValidator = vine.compile( + vine.object({ + params: vine.object({ + filename: vine.string().minLength(1).maxLength(255), + }), + }) +) + +export const localLibraryRenameValidator = vine.compile( + vine.object({ + params: vine.object({ + filename: vine.string().minLength(1).maxLength(255), + }), + name: vine.string().trim().minLength(1).maxLength(255), + }) +) diff --git a/admin/constants/kv_store.ts b/admin/constants/kv_store.ts index 6caeb72..b71acf3 100644 --- a/admin/constants/kv_store.ts +++ b/admin/constants/kv_store.ts @@ -1,3 +1,16 @@ -import { KVStoreKey } from "../types/kv_store.js"; +import { KVStoreKey } from '../types/kv_store.js' -export const SETTINGS_KEYS: KVStoreKey[] = ['chat.suggestionsEnabled', 'chat.lastModel', 'ui.hasVisitedEasySetup', 'ui.theme', 'system.earlyAccess', 'ai.assistantCustomName', 'ai.remoteOllamaUrl', 'ai.ollamaFlashAttention', 'rag.defaultIngestPolicy']; \ No newline at end of file +export const SETTINGS_KEYS: KVStoreKey[] = [ + 'chat.suggestionsEnabled', + 'chat.lastModel', + 'ui.hasVisitedEasySetup', + 'ui.theme', + 'system.earlyAccess', + 'ai.assistantCustomName', + 'ai.remoteOllamaUrl', + 'ai.provider', + 'ai.macNativeWorkerUrl', + 'ai.macNativeModelRoot', + 'ai.ollamaFlashAttention', + 'rag.defaultIngestPolicy', +] diff --git a/admin/inertia/lib/api.ts b/admin/inertia/lib/api.ts index b6372e9..0ba0851 100644 --- a/admin/inertia/lib/api.ts +++ b/admin/inertia/lib/api.ts @@ -2,15 +2,40 @@ import axios, { AxiosError, AxiosInstance } from 'axios' import { ListRemoteZimFilesResponse, ListZimFilesResponse } from '../../types/zim' import { ServiceSlim } from '../../types/services' import { FileEntry } from '../../types/files' -import { CheckLatestVersionResult, SystemInformationResponse, SystemUpdateStatus } from '../../types/system' +import { + CheckLatestVersionResult, + SystemInformationResponse, + SystemUpdateStatus, +} from '../../types/system' import { DownloadJobWithProgress, WikipediaState } from '../../types/downloads' import type { Country, CountryCode, CountryGroup, MapExtractPreflight } from '../../types/maps' import { EmbedJobWithProgress, FileWarningsResult, StoredFileInfo } from '../../types/rag' -import type { CategoryWithStatus, CollectionWithStatus, ContentUpdateCheckResult, ResourceUpdateInfo } from '../../types/collections' +import type { + CategoryWithStatus, + CollectionWithStatus, + ContentUpdateCheckResult, + ResourceUpdateInfo, +} from '../../types/collections' import { catchInternal } from './util' -import { NomadChatResponse, NomadInstalledModel, NomadOllamaModel, OllamaChatRequest } from '../../types/ollama' +import { + NomadChatResponse, + NomadInstalledModel, + NomadOllamaModel, + OllamaChatRequest, +} from '../../types/ollama' import BenchmarkResult from '#models/benchmark_result' -import { BenchmarkType, RunBenchmarkResponse, SubmitBenchmarkResponse, UpdateBuilderTagResponse } from '../../types/benchmark' +import { + BenchmarkType, + RunBenchmarkResponse, + SubmitBenchmarkResponse, + UpdateBuilderTagResponse, +} from '../../types/benchmark' +import type { + LocalLibraryFile, + LocalLibraryListResponse, + LocalLibraryPreviewResponse, +} from '../../types/local_library' +import type { MacAiProvider, MacNativeStatus } from '../../types/mac_ai' class API { private client: AxiosInstance @@ -58,7 +83,9 @@ class API { })() } - async configureRemoteOllama(remoteUrl: string | null): Promise<{ success: boolean; message: string }> { + async configureRemoteOllama( + remoteUrl: string | null + ): Promise<{ success: boolean; message: string }> { return catchInternal(async () => { const response = await this.client.post<{ success: boolean; message: string }>( '/ollama/configure-remote', @@ -68,6 +95,27 @@ class API { })() } + async getMacAiStatus(): Promise { + return catchInternal(async () => { + const response = await this.client.get('/mac-ai/status') + return response.data + })() + } + + async configureMacAi(input: { + provider: MacAiProvider + workerUrl?: string | null + modelRoot?: string | null + }): Promise<{ success: boolean; message: string } | undefined> { + return catchInternal(async () => { + const response = await this.client.post<{ success: boolean; message: string }>( + '/mac-ai/configure', + input + ) + return response.data + })() + } + async deleteModel(model: string): Promise<{ success: boolean; message: string }> { return catchInternal(async () => { const response = await this.client.delete('/ollama/models', { data: { model } }) @@ -100,14 +148,20 @@ class API { })() } - async downloadCategoryTier(categorySlug: string, tierSlug: string): Promise<{ + async downloadCategoryTier( + categorySlug: string, + tierSlug: string + ): Promise<{ message: string categorySlug: string tierSlug: string resources: string[] | null }> { return catchInternal(async () => { - const response = await this.client.post('/zim/download-category-tier', { categorySlug, tierSlug }) + const response = await this.client.post('/zim/download-category-tier', { + categorySlug, + tierSlug, + }) return response.data })() } @@ -188,11 +242,14 @@ class API { })() } - async refreshManifests(): Promise<{ success: boolean; changed: Record } | undefined> { + async refreshManifests(): Promise< + { success: boolean; changed: Record } | undefined + > { return catchInternal(async () => { - const response = await this.client.post<{ success: boolean; changed: Record }>( - '/manifests/refresh' - ) + const response = await this.client.post<{ + success: boolean + changed: Record + }>('/manifests/refresh') return response.data })() } @@ -243,10 +300,9 @@ class API { async getChatSuggestions(signal?: AbortSignal) { return catchInternal(async () => { - const response = await this.client.get<{ suggestions: string[] }>( - '/chat/suggestions', - { signal } - ) + const response = await this.client.get<{ suggestions: string[] }>('/chat/suggestions', { + signal, + }) return response.data.suggestions })() } @@ -290,7 +346,12 @@ class API { })() } - async getAvailableModels(params: { query?: string; recommendedOnly?: boolean; limit?: number; force?: boolean }) { + async getAvailableModels(params: { + query?: string + recommendedOnly?: boolean + limit?: number + force?: boolean + }) { return catchInternal(async () => { const response = await this.client.get<{ models: NomadOllamaModel[] @@ -344,15 +405,13 @@ class API { let data: any try { data = JSON.parse(line.slice(6)) - } catch { continue /* skip malformed chunks */ } + } catch { + continue /* skip malformed chunks */ + } if (data.error) throw new Error('The model encountered an error. Please try again.') - onChunk( - data.message?.content ?? '', - data.message?.thinking ?? '', - data.done ?? false - ) + onChunk(data.message?.content ?? '', data.message?.thinking ?? '', data.done ?? false) } } } finally { @@ -362,14 +421,18 @@ class API { async getBenchmarkResults() { return catchInternal(async () => { - const response = await this.client.get<{ results: BenchmarkResult[], total: number }>('/benchmark/results') + const response = await this.client.get<{ results: BenchmarkResult[]; total: number }>( + '/benchmark/results' + ) return response.data })() } async getLatestBenchmarkResult() { return catchInternal(async () => { - const response = await this.client.get<{ result: BenchmarkResult | null }>('/benchmark/results/latest') + const response = await this.client.get<{ result: BenchmarkResult | null }>( + '/benchmark/results/latest' + ) return response.data })() } @@ -472,9 +535,15 @@ class API { })() } - async cleanupFailedEmbedJobs(): Promise<{ message: string; cleaned: number; filesDeleted: number } | undefined> { + async cleanupFailedEmbedJobs(): Promise< + { message: string; cleaned: number; filesDeleted: number } | undefined + > { return catchInternal(async () => { - const response = await this.client.delete<{ message: string; cleaned: number; filesDeleted: number }>('/rag/failed-jobs') + const response = await this.client.delete<{ + message: string + cleaned: number + filesDeleted: number + }>('/rag/failed-jobs') return response.data })() } @@ -495,7 +564,10 @@ class API { async embedSingleRAGFile(source: string, force: boolean = false) { return catchInternal(async () => { - const response = await this.client.post<{ message: string }>('/rag/files/embed', { source, force }) + const response = await this.client.post<{ message: string }>('/rag/files/embed', { + source, + force, + }) return response.data })() } @@ -509,7 +581,9 @@ class API { async deleteRAGFile(source: string) { return catchInternal(async () => { - const response = await this.client.delete<{ message: string }>('/rag/files', { data: { source } }) + const response = await this.client.delete<{ message: string }>('/rag/files', { + data: { source }, + }) return response.data })() } @@ -632,9 +706,7 @@ class API { async listCuratedMapCollections() { return catchInternal(async () => { - const response = await this.client.get( - '/maps/curated-collections' - ) + const response = await this.client.get('/maps/curated-collections') return response.data })() } @@ -663,26 +735,49 @@ class API { async listMapMarkers() { return catchInternal(async () => { const response = await this.client.get< - Array<{ id: number; name: string; longitude: number; latitude: number; color: string; notes: string | null; created_at: string }> + Array<{ + id: number + name: string + longitude: number + latitude: number + color: string + notes: string | null + created_at: string + }> >('/maps/markers') return response.data })() } - async createMapMarker(data: { name: string; longitude: number; latitude: number; color?: string }) { + async createMapMarker(data: { + name: string + longitude: number + latitude: number + color?: string + }) { return catchInternal(async () => { - const response = await this.client.post< - { id: number; name: string; longitude: number; latitude: number; color: string; notes: string | null; created_at: string } - >('/maps/markers', data) + const response = await this.client.post<{ + id: number + name: string + longitude: number + latitude: number + color: string + notes: string | null + created_at: string + }>('/maps/markers', data) return response.data })() } async updateMapMarker(id: number, data: { name?: string; color?: string }) { return catchInternal(async () => { - const response = await this.client.patch< - { id: number; name: string; longitude: number; latitude: number; color: string } - >(`/maps/markers/${id}`, data) + const response = await this.client.patch<{ + id: number + name: string + longitude: number + latitude: number + color: string + }>(`/maps/markers/${id}`, data) return response.data })() } @@ -715,9 +810,10 @@ class API { async listCustomLibraries() { return catchInternal(async () => { - const response = await this.client.get<{ id: number; name: string; base_url: string; is_default: boolean }[]>( - '/zim/custom-libraries' - ) + const response = + await this.client.get< + { id: number; name: string; base_url: string; is_default: boolean }[] + >('/zim/custom-libraries') return response.data })() } @@ -776,7 +872,9 @@ class API { })() } - async cancelDownloadJob(jobId: string): Promise<{ success: boolean; message: string } | undefined> { + async cancelDownloadJob( + jobId: string + ): Promise<{ success: boolean; message: string } | undefined> { return catchInternal(async () => { const response = await this.client.post<{ success: boolean; message: string }>( `/downloads/jobs/${jobId}/cancel` @@ -789,7 +887,7 @@ class API { return catchInternal(async () => { const response = await this.client.post( `/benchmark/run${sync ? '?sync=true' : ''}`, - { benchmark_type: type }, + { benchmark_type: type } ) return response.data })() @@ -806,17 +904,24 @@ class API { async submitBenchmark(benchmark_id: string, anonymous: boolean) { try { - const response = await this.client.post('/benchmark/submit', { benchmark_id, anonymous }) + const response = await this.client.post('/benchmark/submit', { + benchmark_id, + anonymous, + }) return response.data } catch (error: any) { // For 409 Conflict errors, throw a specific error that the UI can handle if (error.response?.status === 409) { - const err = new Error(error.response?.data?.error || 'This benchmark has already been submitted to the repository') - ; (err as any).status = 409 + const err = new Error( + error.response?.data?.error || + 'This benchmark has already been submitted to the repository' + ) + ;(err as any).status = 409 throw err } // For other errors, extract the message and throw - const errorMessage = error.response?.data?.error || error.message || 'Failed to submit benchmark' + const errorMessage = + error.response?.data?.error || error.message || 'Failed to submit benchmark' throw new Error(errorMessage) } } @@ -913,10 +1018,10 @@ class API { async updateBuilderTag(benchmark_id: string, builder_tag: string) { return catchInternal(async () => { - const response = await this.client.post( - '/benchmark/builder-tag', - { benchmark_id, builder_tag } - ) + const response = await this.client.post('/benchmark/builder-tag', { + benchmark_id, + builder_tag, + }) return response.data })() } @@ -938,12 +1043,75 @@ class API { })() } + async listLocalLibraryFiles(): Promise { + return catchInternal(async () => { + const response = await this.client.get('/local-library/files') + return response.data + })() + } + + async uploadLocalLibraryFile(file: File): Promise { + return catchInternal(async () => { + const formData = new FormData() + formData.append('file', file) + const response = await this.client.post( + '/local-library/files', + formData, + { headers: { 'Content-Type': 'multipart/form-data' } } + ) + return response.data + })() + } + + async previewLocalLibraryFile( + filename: string + ): Promise { + return catchInternal(async () => { + const response = await this.client.get( + `/local-library/files/${encodeURIComponent(filename)}/preview` + ) + return response.data + })() + } + + async renameLocalLibraryFile( + filename: string, + name: string + ): Promise<{ file: LocalLibraryFile } | undefined> { + return catchInternal(async () => { + const response = await this.client.patch<{ file: LocalLibraryFile }>( + `/local-library/files/${encodeURIComponent(filename)}`, + { name } + ) + return response.data + })() + } + + async deleteLocalLibraryFile(filename: string): Promise<{ message: string } | undefined> { + return catchInternal(async () => { + const response = await this.client.delete<{ message: string }>( + `/local-library/files/${encodeURIComponent(filename)}` + ) + return response.data + })() + } + + async indexLocalLibraryFile( + filename: string + ): Promise<{ message: string; jobId?: string } | undefined> { + return catchInternal(async () => { + const response = await this.client.post<{ message: string; jobId?: string }>( + `/local-library/files/${encodeURIComponent(filename)}/index` + ) + return response.data + })() + } + async getSetting(key: string) { return catchInternal(async () => { - const response = await this.client.get<{ key: string; value: any }>( - '/system/settings', - { params: { key } } - ) + const response = await this.client.get<{ key: string; value: any }>('/system/settings', { + params: { key }, + }) return response.data })() } diff --git a/admin/inertia/pages/home.tsx b/admin/inertia/pages/home.tsx index 1feebb2..70fd444 100644 --- a/admin/inertia/pages/home.tsx +++ b/admin/inertia/pages/home.tsx @@ -1,6 +1,7 @@ import { IconBolt, IconHelp, + IconLibrary, IconMapRoute, IconPlus, IconSettings, @@ -28,6 +29,17 @@ const MAPS_ITEM = { poweredBy: null, } +const LOCAL_LIBRARY_ITEM = { + label: 'Local Library', + to: '/local-library', + target: '', + description: 'View PDFs and personal offline documents', + icon: , + installed: true, + displayOrder: 5, + poweredBy: null, +} + // System items shown after all apps const SYSTEM_ITEMS = [ { @@ -90,14 +102,16 @@ export default function Home(props: { } }) { const items: DashboardItem[] = [] - const updateInfo = useUpdateAvailable(); + const updateInfo = useUpdateAvailable() const { aiAssistantName } = usePage<{ aiAssistantName: string }>().props // Check if user has visited Easy Setup const { data: easySetupVisited } = useSystemSetting({ - key: 'ui.hasVisitedEasySetup' + key: 'ui.hasVisitedEasySetup', }) - const shouldHighlightEasySetup = easySetupVisited?.value ? String(easySetupVisited.value) !== 'true' : false + const shouldHighlightEasySetup = easySetupVisited?.value + ? String(easySetupVisited.value) !== 'true' + : false // Add installed services (non-dependency services only) props.system.services @@ -105,7 +119,10 @@ export default function Home(props: { .forEach((service) => { items.push({ // Inject custom AI Assistant name if this is the chat service - label: service.service_name === SERVICE_NAMES.OLLAMA && aiAssistantName ? aiAssistantName : (service.friendly_name || service.service_name), + label: + service.service_name === SERVICE_NAMES.OLLAMA && aiAssistantName + ? aiAssistantName + : service.friendly_name || service.service_name, to: service.ui_location ? getServiceLink(service.ui_location) : '#', target: '_blank', description: @@ -124,6 +141,7 @@ export default function Home(props: { // Add Maps as a Core Capability items.push(MAPS_ITEM) + items.push(LOCAL_LIBRARY_ITEM) // Add system items items.push(...SYSTEM_ITEMS) @@ -134,24 +152,22 @@ export default function Home(props: { return ( - { - updateInfo?.updateAvailable && ( -
- router.visit('/settings/update'), - }} - /> -
- ) - } + {updateInfo?.updateAvailable && ( +
+ router.visit('/settings/update'), + }} + /> +
+ )}
{items.map((item) => { const isEasySetup = item.label === 'Easy Setup' diff --git a/admin/inertia/pages/local-library.tsx b/admin/inertia/pages/local-library.tsx new file mode 100644 index 0000000..2b57d1f --- /dev/null +++ b/admin/inertia/pages/local-library.tsx @@ -0,0 +1,252 @@ +import { Head } from '@inertiajs/react' +import { useEffect, useMemo, useState } from 'react' +import { + IconBook, + IconDownload, + IconFileText, + IconFileTypePdf, + IconRefresh, + IconSearch, + IconUpload, +} from '@tabler/icons-react' +import AppLayout from '~/layouts/AppLayout' +import StyledButton from '~/components/StyledButton' +import Input from '~/components/inputs/Input' +import api from '~/lib/api' +import { formatBytes } from '~/lib/util' +import type { LocalLibraryFile, LocalLibraryPreviewResponse } from '../../types/local_library' +import { useNotifications } from '~/context/NotificationContext' + +function fileIcon(type: LocalLibraryFile['type']) { + if (type === 'pdf') return + if (type === 'epub') return + return +} + +export default function LocalLibraryPage() { + const { addNotification } = useNotifications() + const [files, setFiles] = useState([]) + const [loading, setLoading] = useState(true) + const [uploading, setUploading] = useState(false) + const [query, setQuery] = useState('') + const [selected, setSelected] = useState(null) + const [preview, setPreview] = useState(null) + const [previewLoading, setPreviewLoading] = useState(false) + + async function refresh() { + setLoading(true) + const result = await api.listLocalLibraryFiles() + setFiles(result?.files ?? []) + setLoading(false) + } + + useEffect(() => { + refresh() + }, []) + + const filteredFiles = useMemo(() => { + const normalized = query.trim().toLowerCase() + if (!normalized) return files + return files.filter((file) => file.displayName.toLowerCase().includes(normalized)) + }, [files, query]) + + async function selectFile(file: LocalLibraryFile) { + setSelected(file) + setPreview(null) + if (file.type === 'epub' || file.type === 'text') { + setPreviewLoading(true) + const result = await api.previewLocalLibraryFile(file.name) + setPreview(result ?? null) + setPreviewLoading(false) + } + } + + async function uploadFile(file: File | undefined) { + if (!file) return + setUploading(true) + const result = await api.uploadLocalLibraryFile(file) + if (result?.files) { + setFiles(result.files) + addNotification({ message: 'File added to Local Library.', type: 'success' }) + } + setUploading(false) + } + + async function deleteFile(file: LocalLibraryFile) { + await api.deleteLocalLibraryFile(file.name) + if (selected?.name === file.name) { + setSelected(null) + setPreview(null) + } + await refresh() + } + + async function indexFile(file: LocalLibraryFile) { + const result = await api.indexLocalLibraryFile(file.name) + addNotification({ + message: result?.message ?? 'Indexing queued for this file.', + type: 'success', + }) + } + + return ( + + +
+ + +
+ {!selected ? ( +
+
+ +

Choose a file

+

+ PDFs open here without leaving Command Center. +

+
+
+ ) : ( + <> +
+
+

+ {selected.displayName} +

+

+ {selected.type.toUpperCase()} • {formatBytes(selected.size)} +

+
+
+ {selected.canIndex && ( + indexFile(selected)} + > + Index for AI + + )} + + + + Download + + + deleteFile(selected)} + > + Delete + +
+
+ +
+ {selected.type === 'pdf' && selected.viewUrl ? ( +