feat: add local library and native mac ai support
This commit is contained in:
Reza Sayar
parent
6fb070832d
commit
0befd149ec
14
FAQ.md
14
FAQ.md
|
|
@ -16,23 +16,24 @@ Yes, you can customize the storage location for NOMAD's content by modifying the
|
|||
|
||||
Short answer: yes, but we can't do it for you (and we recommend a local drive for best performance).
|
||||
|
||||
Long answer: Custom storage paths, mount points, and external drives (like iSCSI or SMB/NFS volumes) **are possible**, but this will be up to your individual configuration on the host before NOMAD starts, and then passed in via the compose.yml as this is a *host-level concern*, not a NOMAD-level concern (see above for details). NOMAD itself can't configure this for you, nor could we support all possible configurations in the install script.
|
||||
Long answer: Custom storage paths, mount points, and external drives (like iSCSI or SMB/NFS volumes) **are possible**, but this will be up to your individual configuration on the host before NOMAD starts, and then passed in via the compose.yml as this is a _host-level concern_, not a NOMAD-level concern (see above for details). NOMAD itself can't configure this for you, nor could we support all possible configurations in the install script.
|
||||
|
||||
## Can I run NOMAD on MAC, WSL2, or a non-Debian-based Distro?
|
||||
|
||||
**WSL2 on Windows** is community-supported via the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) — covers two install paths (native Docker and Docker Desktop) with all known gotchas documented and empirical performance numbers comparing WSL2 to bare-metal.
|
||||
|
||||
**macOS and other non-Debian Linux distros** aren't officially supported. See [Why does NOMAD require a Debian-based OS?](#why-does-nomad-require-a-debian-based-os) for details.
|
||||
**macOS Apple Silicon** is supported through the macOS installer path. It uses Docker Desktop or Colima for NOMAD's existing services and a native launchd-managed `nomad-mac-ai` worker for MLX/Core ML inference, because those runtimes need the macOS host Metal/Core ML stack. Other non-Debian Linux distros are not officially supported. See [Why does NOMAD require a Debian-based OS?](#why-does-nomad-require-a-debian-based-os) for details.
|
||||
|
||||
## Why does NOMAD require a Debian-based OS?
|
||||
|
||||
Project N.O.M.A.D. is currently designed to run on Debian-based Linux distributions (with Ubuntu being the recommended distro) because our installation scripts and Docker configurations are optimized for this environment. While it's technically possible to run the Docker containers on other operating systems that support Docker, we have not tested or optimized the installation process for non-Debian-based systems, so we cannot guarantee a smooth experience on those platforms at this time.
|
||||
Project N.O.M.A.D. was originally designed to run on Debian-based Linux distributions (with Ubuntu being the recommended distro) because the installation scripts and Docker configurations were optimized for that environment. The macOS Apple Silicon installer is now a separate supported path for users who want a self-contained MacBook setup with native MLX/Core ML model execution.
|
||||
|
||||
Support for other operating systems will come in the future, but because our development resources are limited as a free and open-source project, we needed to prioritize our efforts and focus on a narrower set of supported platforms for the initial release. We chose Debian-based Linux as our starting point because it's widely used, easy to spin up, and provides a stable environment for running Docker containers.
|
||||
|
||||
For Windows users, the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) provides a community-supported path. Community members have also published guides for other platforms (e.g. macOS) in our Discord community and [Github Discussions](https://github.com/Crosstalk-Solutions/project-nomad/discussions), so if you're interested in running N.O.M.A.D. on a non-Debian-based system, we recommend checking there for any available resources or guides. However, keep in mind that if you choose to run N.O.M.A.D. on a non-Debian-based system, you may encounter issues that we won't be able to provide support for, and you may need to have a higher level of technical expertise to troubleshoot and resolve any problems that arise.
|
||||
|
||||
## Can I run NOMAD on a Raspberry Pi or other ARM-based device?
|
||||
|
||||
Project N.O.M.A.D. is currently designed to run on x86-64 architecture, and we have not yet tested or optimized it for ARM-based devices like the Raspberry Pi (and have not published any official images for ARM architecture).
|
||||
|
||||
Support for ARM-based devices is on our roadmap, but our initial focus was on x86-64 hardware due to its widespread use and compatibility with a wide range of applications.
|
||||
|
|
@ -50,6 +51,7 @@ As of March 2026, Project N.O.M.A.D.'s UI is only available in English, and the
|
|||
## What technologies is NOMAD built with?
|
||||
|
||||
Project N.O.M.A.D. is built using a combination of technologies, including:
|
||||
|
||||
- **Docker:** for containerization of the Command Center and its dependencies
|
||||
- **Node.js & TypeScript:** for the backend of the Command Center, particularly the [AdonisJS](https://adonisjs.com/) framework
|
||||
- **React:** for the frontend of the Command Center, utilizing [Vite](https://vitejs.dev/) and [Inertia.js](https://inertiajs.com/) under the hood
|
||||
|
|
@ -59,6 +61,7 @@ Project N.O.M.A.D. is built using a combination of technologies, including:
|
|||
NOMAD makes use of the Docker-outside-of-Docker ("DooD") pattern, which allows the Command Center to manage and orchestrate other Docker containers on the host machine without needing to run Docker itself inside a container. This approach provides better performance and compatibility with a wider range of host environments while still allowing for powerful container management capabilities through the Command Center's UI.
|
||||
|
||||
## Can I run NOMAD if I have existing Docker containers on my machine?
|
||||
|
||||
Yes, you can safely run Project N.O.M.A.D. on a machine that already has existing Docker containers. NOMAD is designed to coexist with other Docker containers and will not interfere with them as long as there are no port conflicts or resource constraints.
|
||||
|
||||
All of NOMAD's containers are prefixed with `nomad_` in their names, so they can be easily identified and managed separately from any other containers you may have running. Just make sure to review the ports that NOMAD's core services (Command Center, MySQL, Redis) use during installation and adjust them if necessary to avoid conflicts with your existing containers.
|
||||
|
|
@ -68,7 +71,8 @@ All of NOMAD's containers are prefixed with `nomad_` in their names, so they can
|
|||
See [What technologies is NOMAD built with?](#what-technologies-is-nomad-built-with)
|
||||
|
||||
## Can I use any AI models?
|
||||
NOMAD by default uses Ollama inside of a docker container to run LLM Models for the AI Assistant. So if you find a model on HuggingFace for example, you won't be able to use that model in NOMAD. The list of available models in the AI Assistant settings (/settings/models) may not show all of the models you are looking for. If you found a model from https://ollama.com/search that you'd like to try and its not in the settings page, you can use a curl command to download the model.
|
||||
|
||||
NOMAD by default uses Ollama inside of a docker container to run LLM Models for the AI Assistant. On Apple Silicon Macs, NOMAD can also use the native Mac AI worker for MLX-compatible models and Core ML packages. MLX LLMs are usable for chat when they can be loaded by `mlx-lm`; Core ML packages are shown separately because generic Core ML artifacts need a model-specific chat adapter before NOMAD can treat them as chat models. If you found a model from https://ollama.com/search that you'd like to try and its not in the settings page, you can use a curl command to download the model.
|
||||
`curl -X POST -H "Content-Type: application/json" -d '{"model":"MODEL_NAME_HERE"}' http://localhost:8080/api/ollama/models` replacing MODEL_NAME_HERE with the model name from whats in the ollama website.
|
||||
|
||||
## Do I have to install the AI features in NOMAD?
|
||||
|
|
@ -76,6 +80,7 @@ NOMAD by default uses Ollama inside of a docker container to run LLM Models for
|
|||
No, the AI features in NOMAD (Ollama, Qdrant, custom RAG pipeline, etc.) are all optional and not required to use the core functionality of NOMAD.
|
||||
|
||||
## Is NOMAD actually free? Are there any hidden costs?
|
||||
|
||||
Yes, Project N.O.M.A.D. is completely free and open-source software licensed under the Apache License 2.0. There are no hidden costs or fees associated with using NOMAD itself, and we don't have any plans to introduce "premium" features or paid tiers.
|
||||
|
||||
Aside from the cost of the hardware you choose to run it on, there are no costs associated with using NOMAD.
|
||||
|
|
@ -95,6 +100,7 @@ We also encourage community involvement in troubleshooting and resolving issues,
|
|||
We aim to release updates and new features on a regular basis, but the exact timing can vary based on the complexity of the features being developed, the resources available to our volunteer development team, and the feedback and needs of our community. We typically release smaller "patch" versions more frequently to address bugs and make minor improvements, while larger feature releases may take more time to develop and test before they're ready for release.
|
||||
|
||||
## I opened a PR to contribute a new feature or fix a bug. How long does it usually take for PRs to be reviewed and merged?
|
||||
|
||||
We appreciate all contributions to the project and strive to review and merge pull requests (PRs) as quickly as possible. The time it takes for a PR to be reviewed and merged can vary based on several factors, including the complexity of the changes, the current workload of our maintainers, and the need for any additional testing or revisions.
|
||||
|
||||
Because NOMAD is still a young project, some PRs (particularly those for new features) may take longer to review and merge as we prioritize building out the core functionality and ensuring stability before adding new features. However, we do our best to provide timely feedback on all PRs and keep contributors informed about the status of their contributions.
|
||||
|
|
|
|||
58
README.md
58
README.md
|
|
@ -2,6 +2,7 @@
|
|||
<img src="admin/public/project_nomad_logo.webp" width="200" height="200"/>
|
||||
|
||||
# Project N.O.M.A.D.
|
||||
|
||||
### Node for Offline Media, Archives, and Data
|
||||
|
||||
**Knowledge That Never Goes Offline**
|
||||
|
|
@ -17,11 +18,13 @@
|
|||
Project N.O.M.A.D. is a self-contained, offline-first knowledge and education server packed with critical tools, knowledge, and AI to keep you informed and empowered—anytime, anywhere.
|
||||
|
||||
## Installation & Quickstart
|
||||
|
||||
Project N.O.M.A.D. can be installed on any Debian-based operating system (we recommend Ubuntu). Installation is completely terminal-based, and all tools and resources are designed to be accessed through the browser, so there's no need for a desktop environment if you'd rather setup N.O.M.A.D. as a "server" and access it through other clients.
|
||||
|
||||
*Note: sudo/root privileges are required to run the install script*
|
||||
_Note: sudo/root privileges are required to run the install script_
|
||||
|
||||
### Quick Install (Debian-based OS Only)
|
||||
|
||||
```bash
|
||||
sudo apt-get update && \
|
||||
sudo apt-get install -y curl && \
|
||||
|
|
@ -32,17 +35,33 @@ sudo bash install_nomad.sh
|
|||
|
||||
Project N.O.M.A.D. is now installed on your device! Open a browser and navigate to `http://localhost:8080` (or `http://DEVICE_IP:8080`) to start exploring!
|
||||
|
||||
### macOS Apple Silicon Install
|
||||
|
||||
N.O.M.A.D. can also run on Apple Silicon Macs with Docker Desktop or Colima for the core services and a native launchd-managed Mac AI worker for MLX/Core ML inference. The macOS path stores data under your home directory and keeps MLX/Core ML execution on the host so it can use Metal/Core ML acceleration.
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/macos/install_macos_nomad.sh \
|
||||
-o install_macos_nomad.sh && \
|
||||
bash install_macos_nomad.sh
|
||||
```
|
||||
|
||||
After installation, open `http://localhost:8080`, go to AI Assistant settings, choose **Native MLX** or **Native Core ML**, and place compatible model folders/packages in `~/.project-nomad/mac-ai/models`.
|
||||
|
||||
For a complete step-by-step walkthrough (including Ubuntu installation), see the [Installation Guide](https://www.projectnomad.us/install). For Windows users, see the [WSL2 install guide](https://www.projectnomad.us/install/wsl2) — community-supported path covering native Docker and Docker Desktop install routes.
|
||||
|
||||
### Advanced Installation
|
||||
|
||||
For more control over the installation process, copy and paste the [Docker Compose template](https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/management_compose.yaml) into a `docker-compose.yml` file and customize it to your liking (be sure to replace any placeholders with your actual values). Then, run `docker compose up -d` to start the Command Center and its dependencies. Note: this method is recommended for advanced users only, as it requires familiarity with Docker and manual configuration before starting.
|
||||
|
||||
## How It Works
|
||||
|
||||
N.O.M.A.D. is a management UI ("Command Center") and API that orchestrates a collection of containerized tools and resources via [Docker](https://www.docker.com/). It handles installation, configuration, and updates for everything — so you don't have to.
|
||||
|
||||
**Built-in capabilities include:**
|
||||
|
||||
- **AI Chat with Knowledge Base** — local AI chat powered by [Ollama](https://ollama.com/) or you can use OpenAI API compatible software such as LM Studio or llama.cpp, with document upload and semantic search (RAG via [Qdrant](https://qdrant.tech/))
|
||||
- **Information Library** — offline Wikipedia, medical references, ebooks, and more via [Kiwix](https://kiwix.org/)
|
||||
- **Local Library** — in-app viewing for your own PDFs and local documents, with optional Knowledge Base indexing
|
||||
- **Education Platform** — Khan Academy courses with progress tracking via [Kolibri](https://learningequality.org/kolibri/)
|
||||
- **Offline Maps** — downloadable regional maps via [ProtoMaps](https://protomaps.com)
|
||||
- **Data Tools** — encryption, encoding, and analysis via [CyberChef](https://gchq.github.io/CyberChef/)
|
||||
|
|
@ -54,25 +73,27 @@ N.O.M.A.D. also includes built-in tools like a Wikipedia content selector, ZIM l
|
|||
|
||||
## What's Included
|
||||
|
||||
| Capability | Powered By | What You Get |
|
||||
|-----------|-----------|-------------|
|
||||
| Information Library | Kiwix | Offline Wikipedia, medical references, survival guides, ebooks |
|
||||
| AI Assistant | Ollama + Qdrant | Built-in chat with document upload and semantic search |
|
||||
| Education Platform | Kolibri | Khan Academy courses, progress tracking, multi-user support |
|
||||
| Offline Maps | ProtoMaps | Downloadable regional maps with search and navigation |
|
||||
| Data Tools | CyberChef | Encryption, encoding, hashing, and data analysis |
|
||||
| Notes | FlatNotes | Local note-taking with markdown support |
|
||||
| System Benchmark | Built-in | Hardware scoring, Builder Tags, and community leaderboard |
|
||||
| Capability | Powered By | What You Get |
|
||||
| ------------------- | --------------- | -------------------------------------------------------------- |
|
||||
| Information Library | Kiwix | Offline Wikipedia, medical references, survival guides, ebooks |
|
||||
| AI Assistant | Ollama + Qdrant | Built-in chat with document upload and semantic search |
|
||||
| Education Platform | Kolibri | Khan Academy courses, progress tracking, multi-user support |
|
||||
| Offline Maps | ProtoMaps | Downloadable regional maps with search and navigation |
|
||||
| Data Tools | CyberChef | Encryption, encoding, hashing, and data analysis |
|
||||
| Notes | FlatNotes | Local note-taking with markdown support |
|
||||
| System Benchmark | Built-in | Hardware scoring, Builder Tags, and community leaderboard |
|
||||
|
||||
## Device Requirements
|
||||
|
||||
While many similar offline survival computers are designed to be run on bare-minimum, lightweight hardware, Project N.O.M.A.D. is quite the opposite. To install and run the
|
||||
available AI tools, we highly encourage the use of a beefy, GPU-backed device to make the most of your install.
|
||||
|
||||
At it's core, however, N.O.M.A.D. is still very lightweight. For a barebones installation of the management application itself, the following minimal specs are required:
|
||||
|
||||
*Note: Project N.O.M.A.D. is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The harware listed below is for example/comparison use only*
|
||||
_Note: Project N.O.M.A.D. is not sponsored by any hardware manufacturer and is designed to be as hardware-agnostic as possible. The harware listed below is for example/comparison use only_
|
||||
|
||||
#### Minimum Specs
|
||||
|
||||
- Processor: 2 GHz dual-core processor or better
|
||||
- RAM: 4GB system memory
|
||||
- Storage: At least 5 GB free disk space
|
||||
|
|
@ -82,6 +103,7 @@ At it's core, however, N.O.M.A.D. is still very lightweight. For a barebones ins
|
|||
To run LLM's and other included AI tools:
|
||||
|
||||
#### Optimal Specs
|
||||
|
||||
- Processor: AMD Ryzen 7 or Intel Core i7 or better
|
||||
- RAM: 32 GB system memory
|
||||
- Graphics: NVIDIA RTX 3060 or AMD equivalent or better (more VRAM = run larger models)
|
||||
|
|
@ -94,20 +116,24 @@ To run LLM's and other included AI tools:
|
|||
Again, Project N.O.M.A.D. itself is quite lightweight - it's the tools and resources you choose to install with N.O.M.A.D. that will determine the specs required for your unique deployment
|
||||
|
||||
#### Running AI models on a different host
|
||||
|
||||
By default, N.O.M.A.D.'s installer will attempt to setup Ollama on the host when the AI Assistant is installed. However, if you would like to run the AI model on a different host, you can go to the settings of of the AI assistant and input a URL for either an ollama or OpenAI-compatible API server (such as LM Studio).
|
||||
Note that if you use Ollama on a different host, you must start the server with this option `OLLAMA_HOST=0.0.0.0`.
|
||||
Ollama is the preferred way to use the AI assistant as it has features such as model download that OpenAI API does not support. So when using LM Studio for example, you will have to use LM Studio to download models.
|
||||
You are responsible for the setup of Ollama/OpenAI server on the other host.
|
||||
|
||||
## Frequently Asked Questions (FAQ)
|
||||
|
||||
For answers to common questions about Project N.O.M.A.D., please see our [FAQ](FAQ.md) page.
|
||||
|
||||
## About Internet Usage & Privacy
|
||||
|
||||
Project N.O.M.A.D. is designed for offline usage. An internet connection is only required during the initial installation (to download dependencies) and if you (the user) decide to download additional tools and resources at a later time. Otherwise, N.O.M.A.D. does not require an internet connection and has ZERO built-in telemetry.
|
||||
|
||||
To test internet connectivity, N.O.M.A.D. attempts to make a request to Cloudflare's utility endpoint, `https://1.1.1.1/cdn-cgi/trace` and checks for a successful response.
|
||||
|
||||
## About Security
|
||||
|
||||
By design, Project N.O.M.A.D. is intended to be open and available without hurdles - it includes no authentication. If you decide to connect your device to a local network after install (e.g. for allowing other devices to access it's resources), you can block/open ports to control which services are exposed.
|
||||
|
||||
**Will authentication be added in the future?** Maybe. It's not currently a priority, but if there's enough demand for it, we may consider building in an optional authentication layer in a future release to support uses cases where multiple users need access to the same instance but with different permission levels (e.g. family use with parental controls, classroom use with teacher/admin accounts, etc.). We have a suggestion for this on our public roadmap, so if this is something you'd like to see, please upvote it here: https://roadmap.projectnomad.us/posts/1/user-authentication-please-build-in-user-auth-with-admin-user-roles
|
||||
|
|
@ -115,6 +141,7 @@ By design, Project N.O.M.A.D. is intended to be open and available without hurdl
|
|||
For now, we recommend using network-level controls to manage access if you're planning to expose your N.O.M.A.D. instance to other devices on a local network. N.O.M.A.D. is not designed to be exposed directly to the internet, and we strongly advise against doing so unless you really know what you're doing, have taken appropriate security measures, and understand the risks involved.
|
||||
|
||||
## Contributing
|
||||
|
||||
Contributions are welcome and appreciated! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines on how to contribute to the project.
|
||||
|
||||
## Community & Resources
|
||||
|
|
@ -131,28 +158,35 @@ Contributions are welcome and appreciated! Please see [CONTRIBUTING.md](CONTRIBU
|
|||
Project N.O.M.A.D. is licensed under the [Apache License 2.0](LICENSE).
|
||||
|
||||
## Helper Scripts
|
||||
|
||||
Once installed, Project N.O.M.A.D. has a few helper scripts should you ever need to troubleshoot issues or perform maintenance that can't be done through the Command Center. All of these scripts are found in Project N.O.M.A.D.'s install directory, `/opt/project-nomad`
|
||||
|
||||
###
|
||||
|
||||
###### Start Script - Starts all installed project containers
|
||||
|
||||
```bash
|
||||
sudo bash /opt/project-nomad/start_nomad.sh
|
||||
```
|
||||
|
||||
###
|
||||
|
||||
###### Stop Script - Stops all installed project containers
|
||||
|
||||
```bash
|
||||
sudo bash /opt/project-nomad/stop_nomad.sh
|
||||
```
|
||||
|
||||
###
|
||||
|
||||
###### Update Script - Attempts to pull the latest images for the Command Center and its dependencies (i.e. mysql) and recreate the containers. Note: this *only* updates the Command Center containers. It does not update the installable application containers - that should be done through the Command Center UI
|
||||
###### Update Script - Attempts to pull the latest images for the Command Center and its dependencies (i.e. mysql) and recreate the containers. Note: this _only_ updates the Command Center containers. It does not update the installable application containers - that should be done through the Command Center UI
|
||||
|
||||
```bash
|
||||
sudo bash /opt/project-nomad/update_nomad.sh
|
||||
```
|
||||
|
||||
###### Uninstall Script - Need to start fresh? Use the uninstall script to make your life easy. Note: this cannot be undone!
|
||||
|
||||
```bash
|
||||
curl -fsSL https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/uninstall_nomad.sh -o uninstall_nomad.sh && sudo bash uninstall_nomad.sh
|
||||
```
|
||||
|
|
|
|||
|
|
@ -0,0 +1,150 @@
|
|||
import { EmbedFileJob } from '#jobs/embed_file_job'
|
||||
import { LocalLibraryService } from '#services/local_library_service'
|
||||
import {
|
||||
localLibraryFilenameValidator,
|
||||
localLibraryRenameValidator,
|
||||
} from '#validators/local_library'
|
||||
import { inject } from '@adonisjs/core'
|
||||
import type { HttpContext } from '@adonisjs/core/http'
|
||||
|
||||
@inject()
|
||||
export default class LocalLibraryController {
|
||||
constructor(private localLibraryService: LocalLibraryService) {}
|
||||
|
||||
async page({ inertia }: HttpContext) {
|
||||
return inertia.render('local-library')
|
||||
}
|
||||
|
||||
async list({ response }: HttpContext) {
|
||||
return response.status(200).json({ files: await this.localLibraryService.list() })
|
||||
}
|
||||
|
||||
async upload({ request, response }: HttpContext) {
|
||||
const uploadedFile = request.file('file')
|
||||
if (!uploadedFile) return response.status(400).json({ message: 'No file uploaded' })
|
||||
|
||||
try {
|
||||
await this.localLibraryService.ensureStorage()
|
||||
const name = this.localLibraryService.sanitizeUploadName(uploadedFile.clientName)
|
||||
await uploadedFile.move(this.localLibraryService.getStoragePath(), { name, overwrite: true })
|
||||
return response
|
||||
.status(201)
|
||||
.json({ message: 'File uploaded', files: await this.localLibraryService.list() })
|
||||
} catch (error: any) {
|
||||
if (error.message === 'unsupported_file_type') {
|
||||
return response.status(400).json({ message: 'Unsupported library file type' })
|
||||
}
|
||||
if (error.message === 'invalid_filename') {
|
||||
return response.status(400).json({ message: 'Invalid filename' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async view({ request, response }: HttpContext) {
|
||||
const payload = await request.validateUsing(localLibraryFilenameValidator)
|
||||
try {
|
||||
const file = await this.localLibraryService.stream(payload.params.filename)
|
||||
response.header('Content-Type', file.contentType)
|
||||
response.header('Content-Length', String(file.size))
|
||||
response.header(
|
||||
'Content-Disposition',
|
||||
`inline; filename="${encodeURIComponent(file.filename)}"`
|
||||
)
|
||||
return response.stream(file.stream)
|
||||
} catch (error: any) {
|
||||
if (['not_found', 'invalid_filename'].includes(error.message)) {
|
||||
return response.status(404).json({ message: 'File not found' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async download({ request, response }: HttpContext) {
|
||||
const payload = await request.validateUsing(localLibraryFilenameValidator)
|
||||
try {
|
||||
const file = await this.localLibraryService.stream(payload.params.filename)
|
||||
response.header('Content-Type', file.contentType)
|
||||
response.header('Content-Length', String(file.size))
|
||||
response.header(
|
||||
'Content-Disposition',
|
||||
`attachment; filename="${encodeURIComponent(file.filename)}"`
|
||||
)
|
||||
return response.stream(file.stream)
|
||||
} catch (error: any) {
|
||||
if (['not_found', 'invalid_filename'].includes(error.message)) {
|
||||
return response.status(404).json({ message: 'File not found' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async preview({ request, response }: HttpContext) {
|
||||
const payload = await request.validateUsing(localLibraryFilenameValidator)
|
||||
try {
|
||||
return response
|
||||
.status(200)
|
||||
.json(await this.localLibraryService.preview(payload.params.filename))
|
||||
} catch (error: any) {
|
||||
if (error.message === 'preview_unsupported') {
|
||||
return response.status(415).json({ message: 'Preview is not supported for this file type' })
|
||||
}
|
||||
if (['not_found', 'invalid_filename'].includes(error.message)) {
|
||||
return response.status(404).json({ message: 'File not found' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async rename({ request, response }: HttpContext) {
|
||||
const payload = await request.validateUsing(localLibraryRenameValidator)
|
||||
try {
|
||||
return response.status(200).json({
|
||||
file: await this.localLibraryService.rename(payload.params.filename, payload.name),
|
||||
})
|
||||
} catch (error: any) {
|
||||
if (['not_found', 'invalid_filename'].includes(error.message)) {
|
||||
return response.status(404).json({ message: 'File not found' })
|
||||
}
|
||||
if (error.message === 'unsupported_file_type') {
|
||||
return response.status(400).json({ message: 'Unsupported library file type' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async delete({ request, response }: HttpContext) {
|
||||
const payload = await request.validateUsing(localLibraryFilenameValidator)
|
||||
try {
|
||||
await this.localLibraryService.remove(payload.params.filename)
|
||||
return response.status(200).json({ message: 'File deleted' })
|
||||
} catch (error: any) {
|
||||
if (['not_found', 'invalid_filename'].includes(error.message) || error.code === 'ENOENT') {
|
||||
return response.status(404).json({ message: 'File not found' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
|
||||
async index({ request, response }: HttpContext) {
|
||||
const payload = await request.validateUsing(localLibraryFilenameValidator)
|
||||
try {
|
||||
const file = await this.localLibraryService.stream(payload.params.filename)
|
||||
const result = await EmbedFileJob.dispatch({
|
||||
filePath: file.fullPath,
|
||||
fileName: file.filename,
|
||||
fileSize: file.size,
|
||||
})
|
||||
return response.status(202).json({
|
||||
message: result.message,
|
||||
jobId: result.jobId,
|
||||
alreadyProcessing: !result.created,
|
||||
})
|
||||
} catch (error: any) {
|
||||
if (['not_found', 'invalid_filename'].includes(error.message)) {
|
||||
return response.status(404).json({ message: 'File not found' })
|
||||
}
|
||||
throw error
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,62 @@
|
|||
import { MacAiService } from '#services/mac_ai_service'
|
||||
import { RagService } from '#services/rag_service'
|
||||
import { assertNotCloudMetadataUrl } from '#validators/common'
|
||||
import { inject } from '@adonisjs/core'
|
||||
import type { HttpContext } from '@adonisjs/core/http'
|
||||
import type { MacAiProvider } from '../../types/mac_ai.js'
|
||||
import Service from '#models/service'
|
||||
import KVStore from '#models/kv_store'
|
||||
import { SERVICE_NAMES } from '../../constants/service_names.js'
|
||||
import logger from '@adonisjs/core/services/logger'
|
||||
|
||||
@inject()
|
||||
export default class MacAiController {
|
||||
constructor(
|
||||
private macAiService: MacAiService,
|
||||
private ragService: RagService
|
||||
) {}
|
||||
|
||||
async status({ response }: HttpContext) {
|
||||
return response.status(200).json(await this.macAiService.status())
|
||||
}
|
||||
|
||||
async configure({ request, response }: HttpContext) {
|
||||
const provider = request.input('provider', 'ollama') as MacAiProvider
|
||||
if (!['ollama', 'remote', 'native_mlx', 'native_coreml'].includes(provider)) {
|
||||
return response.status(400).json({ success: false, message: 'Unsupported AI provider.' })
|
||||
}
|
||||
|
||||
const workerUrl = request.input('workerUrl', undefined) as string | null | undefined
|
||||
if (workerUrl && workerUrl.trim()) {
|
||||
try {
|
||||
assertNotCloudMetadataUrl(workerUrl)
|
||||
} catch (error) {
|
||||
return response.status(400).json({
|
||||
success: false,
|
||||
message: error instanceof Error ? error.message : 'Invalid native worker URL.',
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
const result = await this.macAiService.configure({
|
||||
provider,
|
||||
workerUrl,
|
||||
modelRoot: request.input('modelRoot', undefined),
|
||||
})
|
||||
|
||||
if (provider === 'native_mlx' || provider === 'native_coreml' || provider === 'remote') {
|
||||
const aiService = await Service.query().where('service_name', SERVICE_NAMES.OLLAMA).first()
|
||||
if (aiService) {
|
||||
aiService.installed = true
|
||||
aiService.installation_status = 'idle'
|
||||
await aiService.save()
|
||||
}
|
||||
await KVStore.setValue('chat.suggestionsEnabled', false)
|
||||
this.ragService.discoverNomadDocs().catch((error) => {
|
||||
logger.error('[MacAiController] Failed to discover Nomad docs:', error)
|
||||
})
|
||||
}
|
||||
|
||||
return response.status(200).json(result)
|
||||
}
|
||||
}
|
||||
|
|
@ -21,7 +21,7 @@ export default class OllamaController {
|
|||
private dockerService: DockerService,
|
||||
private ollamaService: OllamaService,
|
||||
private ragService: RagService
|
||||
) { }
|
||||
) {}
|
||||
|
||||
async availableModels({ request }: HttpContext) {
|
||||
const reqData = await request.validateUsing(getAvailableModelsSchema)
|
||||
|
|
@ -83,7 +83,9 @@ export default class OllamaController {
|
|||
0.3 // Minimum similarity score of 0.3
|
||||
)
|
||||
|
||||
logger.debug(`[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"`)
|
||||
logger.debug(
|
||||
`[RAG] Retrieved ${relevantDocs.length} relevant documents for query: "${rewrittenQuery}"`
|
||||
)
|
||||
|
||||
// If relevant context is found, inject as a system message with adaptive limits
|
||||
if (relevantDocs.length > 0) {
|
||||
|
|
@ -107,7 +109,10 @@ export default class OllamaController {
|
|||
)
|
||||
|
||||
const contextText = trimmedDocs
|
||||
.map((doc, idx) => `[Context ${idx + 1}] (Relevance: ${(doc.score * 100).toFixed(1)}%)\n${doc.text}`)
|
||||
.map(
|
||||
(doc, idx) =>
|
||||
`[Context ${idx + 1}] (Relevance: ${(doc.score * 100).toFixed(1)}%)\n${doc.text}`
|
||||
)
|
||||
.join('\n\n')
|
||||
|
||||
const systemMessage = {
|
||||
|
|
@ -132,13 +137,19 @@ export default class OllamaController {
|
|||
if (estimatedSystemTokens > 3000) {
|
||||
const needed = estimatedSystemTokens + 2048 // leave room for conversation + response
|
||||
numCtx = [8192, 16384, 32768, 65536].find((n) => n >= needed) ?? 65536
|
||||
logger.debug(`[OllamaController] Large system prompt (~${estimatedSystemTokens} tokens), requesting num_ctx: ${numCtx}`)
|
||||
logger.debug(
|
||||
`[OllamaController] Large system prompt (~${estimatedSystemTokens} tokens), requesting num_ctx: ${numCtx}`
|
||||
)
|
||||
}
|
||||
|
||||
// Check if the model supports "thinking" capability for enhanced response generation
|
||||
// If gpt-oss model, it requires a text param for "think" https://docs.ollama.com/api/chat
|
||||
const thinkingCapability = await this.ollamaService.checkModelHasThinking(reqData.model)
|
||||
const think: boolean | 'medium' = thinkingCapability ? (reqData.model.startsWith('gpt-oss') ? 'medium' : true) : false
|
||||
const think: boolean | 'medium' = thinkingCapability
|
||||
? reqData.model.startsWith('gpt-oss')
|
||||
? 'medium'
|
||||
: true
|
||||
: false
|
||||
|
||||
// Separate sessionId from the Ollama request payload — Ollama rejects unknown fields
|
||||
const { sessionId, ...ollamaRequest } = reqData
|
||||
|
|
@ -154,7 +165,9 @@ export default class OllamaController {
|
|||
}
|
||||
|
||||
if (reqData.stream) {
|
||||
logger.debug(`[OllamaController] Initiating streaming response for model: "${reqData.model}" with think: ${think}`)
|
||||
logger.debug(
|
||||
`[OllamaController] Initiating streaming response for model: "${reqData.model}" with think: ${think}`
|
||||
)
|
||||
// Headers already flushed above
|
||||
const stream = await this.ollamaService.chatStream({ ...ollamaRequest, think, numCtx })
|
||||
let fullContent = ''
|
||||
|
|
@ -171,9 +184,13 @@ export default class OllamaController {
|
|||
await this.chatService.addMessage(sessionId, 'assistant', fullContent)
|
||||
const messageCount = await this.chatService.getMessageCount(sessionId)
|
||||
if (messageCount <= 2 && userContent) {
|
||||
this.chatService.generateTitle(sessionId, userContent, fullContent, reqData.model).catch((err) => {
|
||||
logger.error(`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`)
|
||||
})
|
||||
this.chatService
|
||||
.generateTitle(sessionId, userContent, fullContent, reqData.model)
|
||||
.catch((err) => {
|
||||
logger.error(
|
||||
`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
return
|
||||
|
|
@ -186,9 +203,13 @@ export default class OllamaController {
|
|||
await this.chatService.addMessage(sessionId, 'assistant', result.message.content)
|
||||
const messageCount = await this.chatService.getMessageCount(sessionId)
|
||||
if (messageCount <= 2 && userContent) {
|
||||
this.chatService.generateTitle(sessionId, userContent, result.message.content, reqData.model).catch((err) => {
|
||||
logger.error(`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`)
|
||||
})
|
||||
this.chatService
|
||||
.generateTitle(sessionId, userContent, result.message.content, reqData.model)
|
||||
.catch((err) => {
|
||||
logger.error(
|
||||
`[OllamaController] Title generation failed: ${err instanceof Error ? err.message : err}`
|
||||
)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -223,7 +244,9 @@ export default class OllamaController {
|
|||
|
||||
const ollamaService = await Service.query().where('service_name', SERVICE_NAMES.OLLAMA).first()
|
||||
if (!ollamaService) {
|
||||
return response.status(404).send({ success: false, message: 'Ollama service record not found.' })
|
||||
return response
|
||||
.status(404)
|
||||
.send({ success: false, message: 'Ollama service record not found.' })
|
||||
}
|
||||
|
||||
// Clear path: null or empty URL removes remote config. If a local nomad_ollama container
|
||||
|
|
@ -231,6 +254,7 @@ export default class OllamaController {
|
|||
// the service marked installed. Otherwise fall back to uninstalled.
|
||||
if (!remoteUrl || remoteUrl.trim() === '') {
|
||||
await KVStore.clearValue('ai.remoteOllamaUrl')
|
||||
await KVStore.setValue('ai.provider', 'ollama')
|
||||
const hasLocalContainer = await this._startLocalOllamaContainerIfExists()
|
||||
ollamaService.installed = hasLocalContainer
|
||||
ollamaService.installation_status = 'idle'
|
||||
|
|
@ -272,6 +296,7 @@ export default class OllamaController {
|
|||
|
||||
// Save remote URL and mark service as installed
|
||||
await KVStore.setValue('ai.remoteOllamaUrl', remoteUrl.trim())
|
||||
await KVStore.setValue('ai.provider', 'remote')
|
||||
ollamaService.installed = true
|
||||
ollamaService.installation_status = 'idle'
|
||||
await ollamaService.save()
|
||||
|
|
@ -300,9 +325,7 @@ export default class OllamaController {
|
|||
private async _stopLocalOllamaContainer(): Promise<void> {
|
||||
try {
|
||||
const containers = await this.dockerService.docker.listContainers({ all: true })
|
||||
const ollamaContainer = containers.find((c) =>
|
||||
c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`)
|
||||
)
|
||||
const ollamaContainer = containers.find((c) => c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`))
|
||||
if (!ollamaContainer || ollamaContainer.State !== 'running') {
|
||||
return
|
||||
}
|
||||
|
|
@ -320,9 +343,7 @@ export default class OllamaController {
|
|||
private async _startLocalOllamaContainerIfExists(): Promise<boolean> {
|
||||
try {
|
||||
const containers = await this.dockerService.docker.listContainers({ all: true })
|
||||
const ollamaContainer = containers.find((c) =>
|
||||
c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`)
|
||||
)
|
||||
const ollamaContainer = containers.find((c) => c.Names.includes(`/${SERVICE_NAMES.OLLAMA}`))
|
||||
if (!ollamaContainer) {
|
||||
return false
|
||||
}
|
||||
|
|
@ -359,7 +380,7 @@ export default class OllamaController {
|
|||
}
|
||||
}
|
||||
|
||||
async installedModels({ }: HttpContext) {
|
||||
async installedModels({}: HttpContext) {
|
||||
return await this.ollamaService.getModels()
|
||||
}
|
||||
|
||||
|
|
@ -386,7 +407,7 @@ export default class OllamaController {
|
|||
messages: Message[],
|
||||
model: string
|
||||
): Promise<string | null> {
|
||||
const lastUserMessage = [...messages].reverse().find(msg => msg.role === 'user')
|
||||
const lastUserMessage = [...messages].reverse().find((msg) => msg.role === 'user')
|
||||
|
||||
try {
|
||||
// Skip the entire RAG pipeline if there are no documents to search
|
||||
|
|
@ -405,18 +426,19 @@ export default class OllamaController {
|
|||
// entities and topics from earlier turns ("the bars" → "Hershey's bars
|
||||
// chocolate poisoning dog"); without it, embeddings match nothing and
|
||||
// the assistant loses the thread.
|
||||
const userMessages = recentMessages.filter(msg => msg.role === 'user')
|
||||
const userMessages = recentMessages.filter((msg) => msg.role === 'user')
|
||||
if (userMessages.length < 2) {
|
||||
return lastUserMessage?.content || null
|
||||
}
|
||||
|
||||
const conversationContext = recentMessages
|
||||
.map(msg => {
|
||||
.map((msg) => {
|
||||
const role = msg.role === 'user' ? 'User' : 'Assistant'
|
||||
// Truncate assistant messages to first 200 chars to keep context manageable
|
||||
const content = msg.role === 'assistant'
|
||||
? msg.content.slice(0, 200) + (msg.content.length > 200 ? '...' : '')
|
||||
: msg.content
|
||||
const content =
|
||||
msg.role === 'assistant'
|
||||
? msg.content.slice(0, 200) + (msg.content.length > 200 ? '...' : '')
|
||||
: msg.content
|
||||
return `${role}: "${content}"`
|
||||
})
|
||||
.join('\n')
|
||||
|
|
|
|||
|
|
@ -64,6 +64,9 @@ export default class SettingsController {
|
|||
const chatSuggestionsEnabled = await KVStore.getValue('chat.suggestionsEnabled')
|
||||
const aiAssistantCustomName = await KVStore.getValue('ai.assistantCustomName')
|
||||
const remoteOllamaUrl = await KVStore.getValue('ai.remoteOllamaUrl')
|
||||
const aiProvider = await KVStore.getValue('ai.provider')
|
||||
const macNativeWorkerUrl = await KVStore.getValue('ai.macNativeWorkerUrl')
|
||||
const macNativeModelRoot = await KVStore.getValue('ai.macNativeModelRoot')
|
||||
const ollamaFlashAttention = await KVStore.getValue('ai.ollamaFlashAttention')
|
||||
return inertia.render('settings/models', {
|
||||
models: {
|
||||
|
|
@ -73,6 +76,9 @@ export default class SettingsController {
|
|||
chatSuggestionsEnabled: chatSuggestionsEnabled ?? false,
|
||||
aiAssistantCustomName: aiAssistantCustomName ?? '',
|
||||
remoteOllamaUrl: remoteOllamaUrl ?? '',
|
||||
aiProvider: aiProvider ?? (remoteOllamaUrl ? 'remote' : 'ollama'),
|
||||
macNativeWorkerUrl: macNativeWorkerUrl ?? '',
|
||||
macNativeModelRoot: macNativeModelRoot ?? '',
|
||||
ollamaFlashAttention: ollamaFlashAttention ?? true,
|
||||
},
|
||||
},
|
||||
|
|
@ -111,9 +117,9 @@ export default class SettingsController {
|
|||
}
|
||||
|
||||
async getSetting({ request, response }: HttpContext) {
|
||||
const { key } = await getSettingSchema.validate({ key: request.qs().key });
|
||||
const value = await KVStore.getValue(key);
|
||||
return response.status(200).send({ key, value });
|
||||
const { key } = await getSettingSchema.validate({ key: request.qs().key })
|
||||
const value = await KVStore.getValue(key)
|
||||
return response.status(200).send({ key, value })
|
||||
}
|
||||
|
||||
async updateSetting({ request, response }: HttpContext) {
|
||||
|
|
|
|||
|
|
@ -0,0 +1,219 @@
|
|||
import { ensureDirectoryExists, getFileStatsIfExists, sanitizeFilename } from '../utils/fs.js'
|
||||
import { createReadStream } from 'node:fs'
|
||||
import { readFile, readdir, rename, unlink } from 'node:fs/promises'
|
||||
import app from '@adonisjs/core/services/app'
|
||||
import JSZip from 'jszip'
|
||||
import * as cheerio from 'cheerio'
|
||||
import { extname, resolve } from 'node:path'
|
||||
import type {
|
||||
LocalLibraryFile,
|
||||
LocalLibraryFileType,
|
||||
LocalLibraryPreviewResponse,
|
||||
} from '../../types/local_library.js'
|
||||
|
||||
const SUPPORTED_LIBRARY_EXTENSIONS = new Set([
|
||||
'.pdf',
|
||||
'.epub',
|
||||
'.mobi',
|
||||
'.azw',
|
||||
'.azw3',
|
||||
'.txt',
|
||||
'.md',
|
||||
'.rtf',
|
||||
'.docx',
|
||||
])
|
||||
|
||||
const MIME_TYPES: Record<string, string> = {
|
||||
'.pdf': 'application/pdf',
|
||||
'.epub': 'application/epub+zip',
|
||||
'.mobi': 'application/x-mobipocket-ebook',
|
||||
'.azw': 'application/vnd.amazon.ebook',
|
||||
'.azw3': 'application/vnd.amazon.ebook',
|
||||
'.txt': 'text/plain; charset=utf-8',
|
||||
'.md': 'text/markdown; charset=utf-8',
|
||||
'.rtf': 'application/rtf',
|
||||
'.docx': 'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
|
||||
}
|
||||
|
||||
export class LocalLibraryService {
|
||||
public static STORAGE_PATH = 'storage/local_library'
|
||||
|
||||
getStoragePath(): string {
|
||||
return app.makePath(LocalLibraryService.STORAGE_PATH)
|
||||
}
|
||||
|
||||
async ensureStorage(): Promise<void> {
|
||||
await ensureDirectoryExists(this.getStoragePath())
|
||||
}
|
||||
|
||||
sanitizeUploadName(name: string): string {
|
||||
const cleaned = sanitizeFilename(name).replace(/^_+/, '')
|
||||
if (!cleaned || cleaned === '.' || cleaned === '..') {
|
||||
throw new Error('invalid_filename')
|
||||
}
|
||||
if (!this.isSupportedName(cleaned)) {
|
||||
throw new Error('unsupported_file_type')
|
||||
}
|
||||
return cleaned
|
||||
}
|
||||
|
||||
isSupportedName(name: string): boolean {
|
||||
return SUPPORTED_LIBRARY_EXTENSIONS.has(extname(name).toLowerCase())
|
||||
}
|
||||
|
||||
resolveLibraryPath(filename: string): string {
|
||||
const decoded = decodeURIComponent(filename)
|
||||
if (decoded.includes('/') || decoded.includes('\\') || decoded === '.' || decoded === '..') {
|
||||
throw new Error('invalid_filename')
|
||||
}
|
||||
|
||||
const root = resolve(this.getStoragePath())
|
||||
const fullPath = resolve(root, decoded)
|
||||
if (fullPath !== root && fullPath.startsWith(`${root}/`)) {
|
||||
return fullPath
|
||||
}
|
||||
throw new Error('invalid_filename')
|
||||
}
|
||||
|
||||
detectType(name: string): LocalLibraryFileType {
|
||||
const ext = extname(name).toLowerCase()
|
||||
if (ext === '.pdf') return 'pdf'
|
||||
if (ext === '.epub') return 'epub'
|
||||
if (['.mobi', '.azw', '.azw3'].includes(ext)) return 'mobi'
|
||||
if (['.txt', '.md', '.rtf', '.docx'].includes(ext)) return 'text'
|
||||
return 'unknown'
|
||||
}
|
||||
|
||||
contentType(name: string): string {
|
||||
return MIME_TYPES[extname(name).toLowerCase()] ?? 'application/octet-stream'
|
||||
}
|
||||
|
||||
async list(): Promise<LocalLibraryFile[]> {
|
||||
await this.ensureStorage()
|
||||
const names = await readdir(this.getStoragePath())
|
||||
const files: LocalLibraryFile[] = []
|
||||
|
||||
for (const name of names) {
|
||||
if (!this.isSupportedName(name)) continue
|
||||
const fullPath = this.resolveLibraryPath(name)
|
||||
const stats = await getFileStatsIfExists(fullPath)
|
||||
if (!stats) continue
|
||||
const type = this.detectType(name)
|
||||
files.push({
|
||||
name,
|
||||
displayName: name,
|
||||
type,
|
||||
size: stats.size,
|
||||
modifiedTime: stats.modifiedTime.toISOString(),
|
||||
viewUrl:
|
||||
type === 'pdf' ? `/api/local-library/files/${encodeURIComponent(name)}/view` : null,
|
||||
downloadUrl: `/api/local-library/files/${encodeURIComponent(name)}/download`,
|
||||
canPreview: type === 'pdf' || type === 'epub' || type === 'text',
|
||||
canIndex: type !== 'unknown' && type !== 'mobi',
|
||||
indexedSource: fullPath,
|
||||
})
|
||||
}
|
||||
|
||||
return files.sort((a, b) => a.displayName.localeCompare(b.displayName))
|
||||
}
|
||||
|
||||
async stream(filename: string) {
|
||||
const fullPath = this.resolveLibraryPath(filename)
|
||||
const stats = await getFileStatsIfExists(fullPath)
|
||||
if (!stats) throw new Error('not_found')
|
||||
return {
|
||||
stream: createReadStream(fullPath),
|
||||
fullPath,
|
||||
filename: decodeURIComponent(filename),
|
||||
size: stats.size,
|
||||
contentType: this.contentType(filename),
|
||||
}
|
||||
}
|
||||
|
||||
async remove(filename: string): Promise<void> {
|
||||
const fullPath = this.resolveLibraryPath(filename)
|
||||
await unlink(fullPath)
|
||||
}
|
||||
|
||||
async rename(filename: string, requestedName: string): Promise<LocalLibraryFile> {
|
||||
const fullPath = this.resolveLibraryPath(filename)
|
||||
const newName = this.sanitizeUploadName(requestedName)
|
||||
const newPath = this.resolveLibraryPath(newName)
|
||||
await rename(fullPath, newPath)
|
||||
const files = await this.list()
|
||||
const [file] = files.filter((item) => item.name === newName)
|
||||
return file
|
||||
}
|
||||
|
||||
async preview(filename: string): Promise<LocalLibraryPreviewResponse> {
|
||||
const fullPath = this.resolveLibraryPath(filename)
|
||||
const type = this.detectType(filename)
|
||||
if (type === 'mobi' || type === 'unknown') {
|
||||
throw new Error('preview_unsupported')
|
||||
}
|
||||
|
||||
if (type === 'epub') {
|
||||
return {
|
||||
name: decodeURIComponent(filename),
|
||||
type,
|
||||
title: decodeURIComponent(filename),
|
||||
text: await this.extractEpubText(await readFile(fullPath)),
|
||||
}
|
||||
}
|
||||
|
||||
if (type === 'text') {
|
||||
const text = await readFile(fullPath, 'utf-8')
|
||||
return { name: decodeURIComponent(filename), type, title: decodeURIComponent(filename), text }
|
||||
}
|
||||
|
||||
return {
|
||||
name: decodeURIComponent(filename),
|
||||
type,
|
||||
title: decodeURIComponent(filename),
|
||||
text: '',
|
||||
}
|
||||
}
|
||||
|
||||
private async extractEpubText(fileBuffer: Buffer): Promise<string> {
|
||||
const zip = await JSZip.loadAsync(fileBuffer)
|
||||
const containerXml = await zip.file('META-INF/container.xml')?.async('text')
|
||||
if (!containerXml) throw new Error('invalid_epub')
|
||||
|
||||
const $container = cheerio.load(containerXml, { xml: true })
|
||||
const opfPath = $container('rootfile').attr('full-path')
|
||||
if (!opfPath) throw new Error('invalid_epub')
|
||||
|
||||
const opfDir = opfPath.includes('/') ? opfPath.substring(0, opfPath.lastIndexOf('/') + 1) : ''
|
||||
const opfContent = await zip.file(opfPath)?.async('text')
|
||||
if (!opfContent) throw new Error('invalid_epub')
|
||||
|
||||
const $opf = cheerio.load(opfContent, { xml: true })
|
||||
const manifestItems = new Map<string, string>()
|
||||
$opf('manifest item').each((_, el) => {
|
||||
const id = $opf(el).attr('id')
|
||||
const href = $opf(el).attr('href')
|
||||
const mediaType = $opf(el).attr('media-type') || ''
|
||||
if (id && href && (mediaType.includes('html') || mediaType.includes('xml'))) {
|
||||
manifestItems.set(id, href)
|
||||
}
|
||||
})
|
||||
|
||||
const contentFiles: string[] = []
|
||||
$opf('spine itemref').each((_, el) => {
|
||||
const idref = $opf(el).attr('idref')
|
||||
if (idref && manifestItems.has(idref)) contentFiles.push(manifestItems.get(idref)!)
|
||||
})
|
||||
if (contentFiles.length === 0) contentFiles.push(...manifestItems.values())
|
||||
|
||||
const textParts: string[] = []
|
||||
for (const href of contentFiles) {
|
||||
const content = await zip.file(opfDir + href)?.async('text')
|
||||
if (!content) continue
|
||||
const $ = cheerio.load(content)
|
||||
$('script, style').remove()
|
||||
const text = $('body').text().replace(/\s+/g, ' ').trim()
|
||||
if (text) textParts.push(text)
|
||||
}
|
||||
return textParts.join('\n\n')
|
||||
}
|
||||
}
|
||||
|
|
@ -0,0 +1,87 @@
|
|||
import KVStore from '#models/kv_store'
|
||||
import type { MacAiProvider, MacNativeStatus } from '../../types/mac_ai.js'
|
||||
|
||||
export const DEFAULT_MAC_NATIVE_WORKER_URL = 'http://host.docker.internal:8765'
|
||||
|
||||
export class MacAiService {
|
||||
async getProvider(): Promise<MacAiProvider> {
|
||||
const value = await KVStore.getValue('ai.provider')
|
||||
if (value === 'remote' || value === 'native_mlx' || value === 'native_coreml') return value
|
||||
return 'ollama'
|
||||
}
|
||||
|
||||
async getWorkerUrl(): Promise<string> {
|
||||
const value = await KVStore.getValue('ai.macNativeWorkerUrl')
|
||||
return typeof value === 'string' && value.trim()
|
||||
? value.trim().replace(/\/$/, '')
|
||||
: DEFAULT_MAC_NATIVE_WORKER_URL
|
||||
}
|
||||
|
||||
async configure(input: {
|
||||
provider: MacAiProvider
|
||||
workerUrl?: string | null
|
||||
modelRoot?: string | null
|
||||
}): Promise<{ success: true; message: string }> {
|
||||
await KVStore.setValue('ai.provider', input.provider)
|
||||
if (input.workerUrl !== undefined) {
|
||||
if (input.workerUrl && input.workerUrl.trim()) {
|
||||
await KVStore.setValue('ai.macNativeWorkerUrl', input.workerUrl.trim().replace(/\/$/, ''))
|
||||
} else {
|
||||
await KVStore.clearValue('ai.macNativeWorkerUrl')
|
||||
}
|
||||
}
|
||||
if (input.modelRoot !== undefined) {
|
||||
if (input.modelRoot && input.modelRoot.trim()) {
|
||||
await KVStore.setValue('ai.macNativeModelRoot', input.modelRoot.trim())
|
||||
} else {
|
||||
await KVStore.clearValue('ai.macNativeModelRoot')
|
||||
}
|
||||
}
|
||||
return { success: true, message: 'AI provider settings saved.' }
|
||||
}
|
||||
|
||||
async status(): Promise<MacNativeStatus> {
|
||||
const provider = await this.getProvider()
|
||||
const url = await this.getWorkerUrl()
|
||||
const nativeProvider = provider === 'native_mlx' || provider === 'native_coreml'
|
||||
if (!nativeProvider) {
|
||||
return { configured: false, connected: false, active: false, provider, url, models: [] }
|
||||
}
|
||||
|
||||
try {
|
||||
const [healthResponse, modelResponse] = await Promise.all([
|
||||
fetch(`${url}/health`, { signal: AbortSignal.timeout(2500) }),
|
||||
fetch(`${url}/v1/models`, { signal: AbortSignal.timeout(2500) }),
|
||||
])
|
||||
const modelsPayload: any = modelResponse.ok ? await modelResponse.json() : { data: [] }
|
||||
const models = Array.isArray(modelsPayload?.data)
|
||||
? modelsPayload.data.map((model: any) => ({
|
||||
id: String(model.id),
|
||||
name: String(model.name || model.id),
|
||||
backend: model.backend === 'coreml' ? 'coreml' : 'mlx',
|
||||
path: typeof model.path === 'string' ? model.path : undefined,
|
||||
usableForChat: Boolean(model.usable_for_chat ?? model.usableForChat),
|
||||
notes: typeof model.notes === 'string' ? model.notes : undefined,
|
||||
}))
|
||||
: []
|
||||
return {
|
||||
configured: true,
|
||||
connected: healthResponse.ok,
|
||||
active: healthResponse.ok,
|
||||
provider,
|
||||
url,
|
||||
models,
|
||||
}
|
||||
} catch (error) {
|
||||
return {
|
||||
configured: true,
|
||||
connected: false,
|
||||
active: false,
|
||||
provider,
|
||||
url,
|
||||
models: [],
|
||||
message: error instanceof Error ? error.message : 'Native Mac AI worker is unreachable.',
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -1,6 +1,9 @@
|
|||
import { inject } from '@adonisjs/core'
|
||||
import OpenAI from 'openai'
|
||||
import type { ChatCompletionChunk, ChatCompletionMessageParam } from 'openai/resources/chat/completions.js'
|
||||
import type {
|
||||
ChatCompletionChunk,
|
||||
ChatCompletionMessageParam,
|
||||
} from 'openai/resources/chat/completions.js'
|
||||
import type { Stream } from 'openai/streaming.js'
|
||||
import { NomadOllamaModel } from '../../types/ollama.js'
|
||||
import { EMBEDDING_MODEL_NAME, FALLBACK_RECOMMENDED_OLLAMA_MODELS } from '../../constants/ollama.js'
|
||||
|
|
@ -16,6 +19,7 @@ import { BROADCAST_CHANNELS } from '../../constants/broadcast.js'
|
|||
import env from '#start/env'
|
||||
import { NOMAD_API_DEFAULT_BASE_URL } from '../../constants/misc.js'
|
||||
import KVStore from '#models/kv_store'
|
||||
import { DEFAULT_MAC_NATIVE_WORKER_URL } from './mac_ai_service.js'
|
||||
|
||||
const NOMAD_MODELS_API_PATH = '/api/v1/ollama/models'
|
||||
const MODELS_CACHE_FILE = path.join(process.cwd(), 'storage', 'ollama-models-cache.json')
|
||||
|
|
@ -53,17 +57,27 @@ export class OllamaService {
|
|||
private baseUrl: string | null = null
|
||||
private initPromise: Promise<void> | null = null
|
||||
private isOllamaNative: boolean | null = null
|
||||
private activeDownloads: Map<string, Promise<{ success: boolean; message: string; retryable?: boolean }>> = new Map()
|
||||
private activeDownloads: Map<
|
||||
string,
|
||||
Promise<{ success: boolean; message: string; retryable?: boolean }>
|
||||
> = new Map()
|
||||
|
||||
constructor() {}
|
||||
|
||||
private async _initialize() {
|
||||
if (!this.initPromise) {
|
||||
this.initPromise = (async () => {
|
||||
// Check KVStore for a custom base URL (remote Ollama, LM Studio, llama.cpp, etc.)
|
||||
// Check KVStore for a custom provider before falling back to local Ollama.
|
||||
const provider = (await KVStore.getValue('ai.provider')) as string | null
|
||||
const customUrl = (await KVStore.getValue('ai.remoteOllamaUrl')) as string | null
|
||||
if (customUrl && customUrl.trim()) {
|
||||
const nativeUrl = (await KVStore.getValue('ai.macNativeWorkerUrl')) as string | null
|
||||
|
||||
if (provider === 'native_mlx' || provider === 'native_coreml') {
|
||||
this.baseUrl = (nativeUrl?.trim() || DEFAULT_MAC_NATIVE_WORKER_URL).replace(/\/$/, '')
|
||||
this.isOllamaNative = false
|
||||
} else if ((provider === 'remote' || !provider) && customUrl && customUrl.trim()) {
|
||||
this.baseUrl = customUrl.trim().replace(/\/$/, '')
|
||||
this.isOllamaNative = false
|
||||
} else {
|
||||
// Fall back to the local Ollama container managed by Docker
|
||||
const dockerService = new (await import('./docker_service.js')).DockerService()
|
||||
|
|
@ -111,7 +125,9 @@ export class OllamaService {
|
|||
// Deduplicate concurrent downloads of the same model
|
||||
const existing = this.activeDownloads.get(model)
|
||||
if (existing) {
|
||||
logger.info(`[OllamaService] Download already in progress for "${model}", waiting on existing download.`)
|
||||
logger.info(
|
||||
`[OllamaService] Download already in progress for "${model}", waiting on existing download.`
|
||||
)
|
||||
return existing
|
||||
}
|
||||
|
||||
|
|
@ -222,9 +238,8 @@ export class OllamaService {
|
|||
aggTotal += total
|
||||
}
|
||||
|
||||
const percent = aggTotal > 0
|
||||
? parseFloat(((aggCompleted / aggTotal) * 100).toFixed(2))
|
||||
: 0
|
||||
const percent =
|
||||
aggTotal > 0 ? parseFloat(((aggCompleted / aggTotal) * 100).toFixed(2)) : 0
|
||||
|
||||
// Throttle broadcasts. Always call the progressCallback though — the worker
|
||||
// uses it to update job state in Redis, which should reflect the latest view.
|
||||
|
|
@ -274,9 +289,7 @@ export class OllamaService {
|
|||
}
|
||||
|
||||
const errorMessage = error instanceof Error ? error.message : String(error)
|
||||
logger.error(
|
||||
`[OllamaService] Failed to download model "${model}": ${errorMessage}`
|
||||
)
|
||||
logger.error(`[OllamaService] Failed to download model "${model}": ${errorMessage}`)
|
||||
|
||||
// Check for version mismatch (Ollama 412 response)
|
||||
const isVersionMismatch = errorMessage.includes('newer version of Ollama')
|
||||
|
|
@ -364,7 +377,9 @@ export class OllamaService {
|
|||
params.num_ctx = chatRequest.numCtx
|
||||
}
|
||||
|
||||
const stream = (await this.openai.chat.completions.create(params)) as unknown as Stream<ChatCompletionChunk>
|
||||
const stream = (await this.openai.chat.completions.create(
|
||||
params
|
||||
)) as unknown as Stream<ChatCompletionChunk>
|
||||
|
||||
// Returns how many trailing chars of `text` could be the start of `tag`
|
||||
function partialTagSuffix(tag: string, text: string): number {
|
||||
|
|
@ -424,7 +439,9 @@ export class OllamaService {
|
|||
content: parsedContent,
|
||||
thinking: nativeThinking + parsedThinking,
|
||||
},
|
||||
done: chunk.choices[0]?.finish_reason !== null && chunk.choices[0]?.finish_reason !== undefined,
|
||||
done:
|
||||
chunk.choices[0]?.finish_reason !== null &&
|
||||
chunk.choices[0]?.finish_reason !== undefined,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -442,7 +459,10 @@ export class OllamaService {
|
|||
{ model: modelName },
|
||||
{ timeout: 5000 }
|
||||
)
|
||||
return Array.isArray(response.data?.capabilities) && response.data.capabilities.includes('thinking')
|
||||
return (
|
||||
Array.isArray(response.data?.capabilities) &&
|
||||
response.data.capabilities.includes('thinking')
|
||||
)
|
||||
} catch {
|
||||
// Non-Ollama backends don't expose /api/show — assume no thinking support
|
||||
return false
|
||||
|
|
@ -465,7 +485,10 @@ export class OllamaService {
|
|||
logger.error(
|
||||
`[OllamaService] Failed to delete model "${modelName}": ${error instanceof Error ? error.message : error}`
|
||||
)
|
||||
return { success: false, message: 'Failed to delete model. This may not be an Ollama backend.' }
|
||||
return {
|
||||
success: false,
|
||||
message: 'Failed to delete model. This may not be an Ollama backend.',
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -581,9 +604,7 @@ export class OllamaService {
|
|||
const models: Array<{ name?: string; size_vram?: number }> = response.data?.models ?? []
|
||||
// Match any loaded model whose name signals it's an embedding model.
|
||||
// nomic-embed-text, mxbai-embed-large, snowflake-arctic-embed, etc. all follow this convention.
|
||||
return models.some(
|
||||
(m) => m.name?.toLowerCase().includes('embed') && (m.size_vram ?? 0) > 0
|
||||
)
|
||||
return models.some((m) => m.name?.toLowerCase().includes('embed') && (m.size_vram ?? 0) > 0)
|
||||
} catch (err: any) {
|
||||
// /api/ps unreachable (Ollama down, non-native backend, etc.) — fail closed: assume CPU,
|
||||
// which means we'll pace. Better to over-pace than risk box-killing CPU saturation.
|
||||
|
|
|
|||
|
|
@ -4,7 +4,14 @@ import { inject } from '@adonisjs/core'
|
|||
import logger from '@adonisjs/core/services/logger'
|
||||
import { TokenChunker } from '@chonkiejs/core'
|
||||
import sharp from 'sharp'
|
||||
import { deleteFileIfExists, determineFileType, getFile, getFileStatsIfExists, listDirectoryContentsRecursive, ZIM_STORAGE_PATH } from '../utils/fs.js'
|
||||
import {
|
||||
deleteFileIfExists,
|
||||
determineFileType,
|
||||
getFile,
|
||||
getFileStatsIfExists,
|
||||
listDirectoryContentsRecursive,
|
||||
ZIM_STORAGE_PATH,
|
||||
} from '../utils/fs.js'
|
||||
import { PDFParse } from 'pdf-parse'
|
||||
import { createWorker } from 'tesseract.js'
|
||||
import { fromBuffer } from 'pdf2pic'
|
||||
|
|
@ -25,7 +32,13 @@ import type { KbIngestStateValue } from '../../types/kb_ingest_state.js'
|
|||
import { ZIMExtractionService } from './zim_extraction_service.js'
|
||||
import { ZIM_BATCH_SIZE } from '../../constants/zim_extraction.js'
|
||||
import { EMBEDDING_MODEL_NAME } from '../../constants/ollama.js'
|
||||
import { ProcessAndEmbedFileResponse, ProcessZIMFileResponse, RAGResult, RerankedRAGResult } from '../../types/rag.js'
|
||||
import {
|
||||
ProcessAndEmbedFileResponse,
|
||||
ProcessZIMFileResponse,
|
||||
RAGResult,
|
||||
RerankedRAGResult,
|
||||
} from '../../types/rag.js'
|
||||
import { LocalLibraryService } from './local_library_service.js'
|
||||
|
||||
export type EmbedSingleFileFailureCode =
|
||||
| 'not_found'
|
||||
|
|
@ -51,8 +64,8 @@ export class RagService {
|
|||
public static TARGET_TOKENS_PER_CHUNK = 1500 // Target 1500 tokens per chunk for embedding
|
||||
public static PREFIX_TOKEN_BUDGET = 10 // Reserve ~10 tokens for prefixes
|
||||
public static CHAR_TO_TOKEN_RATIO = 2 // Conservative chars-per-token estimate; technical docs
|
||||
// (numbers, symbols, abbreviations) tokenize denser
|
||||
// than plain prose (~3), so 2 avoids context overflows
|
||||
// (numbers, symbols, abbreviations) tokenize denser
|
||||
// than plain prose (~3), so 2 avoids context overflows
|
||||
// Nomic Embed Text v1.5 uses task-specific prefixes for optimal performance
|
||||
public static SEARCH_DOCUMENT_PREFIX = 'search_document: '
|
||||
public static SEARCH_QUERY_PREFIX = 'search_query: '
|
||||
|
|
@ -61,14 +74,16 @@ export class RagService {
|
|||
constructor(
|
||||
private dockerService: DockerService,
|
||||
private ollamaService: OllamaService
|
||||
) { }
|
||||
) {}
|
||||
|
||||
private async _initializeQdrantClient() {
|
||||
if (!this.qdrantInitPromise) {
|
||||
this.qdrantInitPromise = (async () => {
|
||||
const qdrantUrl = await this.dockerService.getServiceURL(SERVICE_NAMES.QDRANT)
|
||||
if (!qdrantUrl) {
|
||||
throw new Error('Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.')
|
||||
throw new Error(
|
||||
'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.'
|
||||
)
|
||||
}
|
||||
this.qdrant = new QdrantClient({ url: qdrantUrl })
|
||||
})().catch((err) => {
|
||||
|
|
@ -90,7 +105,8 @@ export class RagService {
|
|||
this.qdrantInitPromise = null
|
||||
return {
|
||||
online: false,
|
||||
message: 'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.',
|
||||
message:
|
||||
'Qdrant vector database is offline. Restart the AI Assistant service in Settings to restore the Knowledge Base.',
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -142,15 +158,17 @@ export class RagService {
|
|||
* - Control characters (except newlines, tabs, and carriage returns)
|
||||
*/
|
||||
private sanitizeText(text: string): string {
|
||||
return text
|
||||
// Null bytes
|
||||
.replace(/\x00/g, '')
|
||||
// Problematic control characters (keep \n, \r, \t)
|
||||
.replace(/[\x01-\x08\x0B-\x0C\x0E-\x1F\x7F]/g, '')
|
||||
// Invalid Unicode surrogates
|
||||
.replace(/[\uD800-\uDFFF]/g, '')
|
||||
// Trim extra whitespace
|
||||
.trim()
|
||||
return (
|
||||
text
|
||||
// Null bytes
|
||||
.replace(/\x00/g, '')
|
||||
// Problematic control characters (keep \n, \r, \t)
|
||||
.replace(/[\x01-\x08\x0B-\x0C\x0E-\x1F\x7F]/g, '')
|
||||
// Invalid Unicode surrogates
|
||||
.replace(/[\uD800-\uDFFF]/g, '')
|
||||
// Trim extra whitespace
|
||||
.trim()
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -204,33 +222,33 @@ export class RagService {
|
|||
* TODO: We could probably move this to a separate QueryPreprocessor class if it grows more complex, but for now it's manageable here.
|
||||
*/
|
||||
private static QUERY_EXPANSION_DICTIONARY: Record<string, string> = {
|
||||
'bob': 'bug out bag',
|
||||
'bov': 'bug out vehicle',
|
||||
'bol': 'bug out location',
|
||||
'edc': 'every day carry',
|
||||
'mre': 'meal ready to eat',
|
||||
'shtf': 'shit hits the fan',
|
||||
'teotwawki': 'the end of the world as we know it',
|
||||
'opsec': 'operational security',
|
||||
'ifak': 'individual first aid kit',
|
||||
'ghb': 'get home bag',
|
||||
'ghi': 'get home in',
|
||||
'wrol': 'without rule of law',
|
||||
'emp': 'electromagnetic pulse',
|
||||
'ham': 'ham amateur radio',
|
||||
'nbr': 'nuclear biological radiological',
|
||||
'cbrn': 'chemical biological radiological nuclear',
|
||||
'sar': 'search and rescue',
|
||||
'comms': 'communications radio',
|
||||
'fifo': 'first in first out',
|
||||
'mylar': 'mylar bag food storage',
|
||||
'paracord': 'paracord 550 cord',
|
||||
'ferro': 'ferro rod fire starter',
|
||||
'bivvy': 'bivvy bivy emergency shelter',
|
||||
'bdu': 'battle dress uniform',
|
||||
'gmrs': 'general mobile radio service',
|
||||
'frs': 'family radio service',
|
||||
'nbc': 'nuclear biological chemical',
|
||||
bob: 'bug out bag',
|
||||
bov: 'bug out vehicle',
|
||||
bol: 'bug out location',
|
||||
edc: 'every day carry',
|
||||
mre: 'meal ready to eat',
|
||||
shtf: 'shit hits the fan',
|
||||
teotwawki: 'the end of the world as we know it',
|
||||
opsec: 'operational security',
|
||||
ifak: 'individual first aid kit',
|
||||
ghb: 'get home bag',
|
||||
ghi: 'get home in',
|
||||
wrol: 'without rule of law',
|
||||
emp: 'electromagnetic pulse',
|
||||
ham: 'ham amateur radio',
|
||||
nbr: 'nuclear biological radiological',
|
||||
cbrn: 'chemical biological radiological nuclear',
|
||||
sar: 'search and rescue',
|
||||
comms: 'communications radio',
|
||||
fifo: 'first in first out',
|
||||
mylar: 'mylar bag food storage',
|
||||
paracord: 'paracord 550 cord',
|
||||
ferro: 'ferro rod fire starter',
|
||||
bivvy: 'bivvy bivy emergency shelter',
|
||||
bdu: 'battle dress uniform',
|
||||
gmrs: 'general mobile radio service',
|
||||
frs: 'family radio service',
|
||||
nbc: 'nuclear biological chemical',
|
||||
}
|
||||
|
||||
private preprocessQuery(query: string): string {
|
||||
|
|
@ -311,7 +329,9 @@ export class RagService {
|
|||
// TokenChunker uses character-based tokenization (1 char = 1 token)
|
||||
// We need to convert our embedding model's token counts to character counts
|
||||
// since nomic-embed-text tokenizer uses ~3 chars per token
|
||||
const targetCharsPerChunk = Math.floor(RagService.TARGET_TOKENS_PER_CHUNK * RagService.CHAR_TO_TOKEN_RATIO)
|
||||
const targetCharsPerChunk = Math.floor(
|
||||
RagService.TARGET_TOKENS_PER_CHUNK * RagService.CHAR_TO_TOKEN_RATIO
|
||||
)
|
||||
const overlapChars = Math.floor(150 * RagService.CHAR_TO_TOKEN_RATIO)
|
||||
|
||||
const chunker = await TokenChunker.create({
|
||||
|
|
@ -359,9 +379,14 @@ export class RagService {
|
|||
const batchStart = batchIdx * batchSize
|
||||
const batch = prefixedChunks.slice(batchStart, batchStart + batchSize)
|
||||
|
||||
logger.debug(`[RAG] Embedding batch ${batchIdx + 1}/${totalBatches} (${batch.length} chunks)`)
|
||||
logger.debug(
|
||||
`[RAG] Embedding batch ${batchIdx + 1}/${totalBatches} (${batch.length} chunks)`
|
||||
)
|
||||
|
||||
const response = await this.ollamaService.embed(this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME, batch)
|
||||
const response = await this.ollamaService.embed(
|
||||
this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME,
|
||||
batch
|
||||
)
|
||||
|
||||
embeddings.push(...response.embeddings)
|
||||
|
||||
|
|
@ -392,13 +417,14 @@ export class RagService {
|
|||
|
||||
logger.debug(`[RAG] Extracted keywords for chunk ${index}: [${allKeywords.join(', ')}]`)
|
||||
if (structuralKeywords.length > 0) {
|
||||
logger.debug(`[RAG] - Structural: [${structuralKeywords.join(', ')}], Content: [${contentKeywords.join(', ')}]`)
|
||||
logger.debug(
|
||||
`[RAG] - Structural: [${structuralKeywords.join(', ')}], Content: [${contentKeywords.join(', ')}]`
|
||||
)
|
||||
}
|
||||
|
||||
// Sanitize source metadata as well
|
||||
const sanitizedSource = typeof metadata.source === 'string'
|
||||
? this.sanitizeText(metadata.source)
|
||||
: 'unknown'
|
||||
const sanitizedSource =
|
||||
typeof metadata.source === 'string' ? this.sanitizeText(metadata.source) : 'unknown'
|
||||
|
||||
return {
|
||||
id: randomUUID(), // qdrant requires either uuid or unsigned int
|
||||
|
|
@ -411,7 +437,7 @@ export class RagService {
|
|||
keywords: allKeywords.join(' '), // store as space-separated string for text search
|
||||
char_count: sanitizedText.length,
|
||||
created_at: timestamp,
|
||||
source: sanitizedSource
|
||||
source: sanitizedSource,
|
||||
},
|
||||
}
|
||||
})
|
||||
|
|
@ -661,9 +687,7 @@ export class RagService {
|
|||
})
|
||||
|
||||
// If no spine found, fall back to all manifest items
|
||||
const contentFiles = spineOrder.length > 0
|
||||
? spineOrder
|
||||
: Array.from(manifestItems.values())
|
||||
const contentFiles = spineOrder.length > 0 ? spineOrder : Array.from(manifestItems.values())
|
||||
|
||||
// Extract text from each content file in order
|
||||
const textParts: string[] = []
|
||||
|
|
@ -682,7 +706,9 @@ export class RagService {
|
|||
}
|
||||
|
||||
const fullText = textParts.join('\n\n')
|
||||
logger.debug(`[RAG] EPUB extracted ${textParts.length} chapters, ${fullText.length} characters total`)
|
||||
logger.debug(
|
||||
`[RAG] EPUB extracted ${textParts.length} chapters, ${fullText.length} characters total`
|
||||
)
|
||||
return fullText
|
||||
}
|
||||
|
||||
|
|
@ -693,12 +719,19 @@ export class RagService {
|
|||
onProgress?: (percent: number) => Promise<void>
|
||||
): Promise<{ success: boolean; message: string; chunks?: number }> {
|
||||
if (!extractedText || extractedText.trim().length === 0) {
|
||||
return { success: false, message: 'Process completed succesfully, but no text was found to embed.' }
|
||||
return {
|
||||
success: false,
|
||||
message: 'Process completed succesfully, but no text was found to embed.',
|
||||
}
|
||||
}
|
||||
|
||||
const embedResult = await this.embedAndStoreText(extractedText, {
|
||||
source: filepath
|
||||
}, onProgress)
|
||||
const embedResult = await this.embedAndStoreText(
|
||||
extractedText,
|
||||
{
|
||||
source: filepath,
|
||||
},
|
||||
onProgress
|
||||
)
|
||||
|
||||
if (!embedResult) {
|
||||
return { success: false, message: 'Failed to embed and store the extracted text.' }
|
||||
|
|
@ -719,7 +752,7 @@ export class RagService {
|
|||
/**
|
||||
* Main pipeline to process and embed an uploaded file into the RAG knowledge base.
|
||||
* This includes text extraction, chunking, embedding, and storing in Qdrant.
|
||||
*
|
||||
*
|
||||
* Orchestrates file type detection and delegates to specialized processors.
|
||||
* For ZIM files, supports batch processing via batchOffset parameter.
|
||||
*/
|
||||
|
|
@ -733,7 +766,7 @@ export class RagService {
|
|||
const fileType = determineFileType(filepath)
|
||||
logger.debug(`[RAG] Processing file: ${filepath} (detected type: ${fileType})`)
|
||||
|
||||
if (fileType === 'unknown') {
|
||||
if (fileType === 'unknown' || fileType === 'mobi') {
|
||||
return { success: false, message: 'Unsupported file type.' }
|
||||
}
|
||||
|
||||
|
|
@ -771,12 +804,15 @@ export class RagService {
|
|||
|
||||
// Extraction done — scale remaining embedding progress from 15% to 100%
|
||||
if (onProgress) await onProgress(15)
|
||||
const scaledProgress = onProgress
|
||||
? (p: number) => onProgress(15 + p * 0.85)
|
||||
: undefined
|
||||
const scaledProgress = onProgress ? (p: number) => onProgress(15 + p * 0.85) : undefined
|
||||
|
||||
// Embed extracted text and cleanup
|
||||
return await this.embedTextAndCleanup(extractedText, filepath, deleteAfterEmbedding, scaledProgress)
|
||||
return await this.embedTextAndCleanup(
|
||||
extractedText,
|
||||
filepath,
|
||||
deleteAfterEmbedding,
|
||||
scaledProgress
|
||||
)
|
||||
} catch (error) {
|
||||
logger.error('[RAG] Error processing and embedding file:', error)
|
||||
return { success: false, message: 'Error processing and embedding file.' }
|
||||
|
|
@ -822,9 +858,7 @@ export class RagService {
|
|||
allModels.find((model) => model.name.toLowerCase().includes('nomic-embed-text'))
|
||||
|
||||
if (!embeddingModel) {
|
||||
logger.warn(
|
||||
`[RAG] ${EMBEDDING_MODEL_NAME} not found. Cannot perform similarity search.`
|
||||
)
|
||||
logger.warn(`[RAG] ${EMBEDDING_MODEL_NAME} not found. Cannot perform similarity search.`)
|
||||
this.embeddingModelVerified = false
|
||||
return []
|
||||
}
|
||||
|
|
@ -855,7 +889,10 @@ export class RagService {
|
|||
return []
|
||||
}
|
||||
|
||||
const response = await this.ollamaService.embed(this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME, [prefixedQuery])
|
||||
const response = await this.ollamaService.embed(
|
||||
this.resolvedEmbeddingModel ?? EMBEDDING_MODEL_NAME,
|
||||
[prefixedQuery]
|
||||
)
|
||||
|
||||
// Perform semantic search with a higher limit to enable reranking
|
||||
const searchLimit = limit * 3 // Get more results for reranking
|
||||
|
|
@ -1022,9 +1059,7 @@ export class RagService {
|
|||
* Uses greedy selection: for each result, apply 0.85^n penalty where n is the
|
||||
* number of results already selected from the same source.
|
||||
*/
|
||||
private applySourceDiversity(
|
||||
results: Array<RerankedRAGResult>
|
||||
) {
|
||||
private applySourceDiversity(results: Array<RerankedRAGResult>) {
|
||||
const sourceCounts = new Map<string, number>()
|
||||
const DIVERSITY_PENALTY = 0.85
|
||||
|
||||
|
|
@ -1054,7 +1089,10 @@ export class RagService {
|
|||
*/
|
||||
public async hasDocuments(): Promise<boolean> {
|
||||
try {
|
||||
await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION)
|
||||
await this._ensureCollection(
|
||||
RagService.CONTENT_COLLECTION_NAME,
|
||||
RagService.EMBEDDING_DIMENSION
|
||||
)
|
||||
const collectionInfo = await this.qdrant!.getCollection(RagService.CONTENT_COLLECTION_NAME)
|
||||
return (collectionInfo.points_count ?? 0) > 0
|
||||
} catch {
|
||||
|
|
@ -1103,7 +1141,11 @@ export class RagService {
|
|||
// the vector store?". Both are needed to render the KB UI honestly.
|
||||
const stateByPath = new Map<string, { state: KbIngestStateValue; chunks_embedded: number }>()
|
||||
try {
|
||||
const stateRows = await KbIngestState.query().select('file_path', 'state', 'chunks_embedded')
|
||||
const stateRows = await KbIngestState.query().select(
|
||||
'file_path',
|
||||
'state',
|
||||
'chunks_embedded'
|
||||
)
|
||||
for (const row of stateRows) {
|
||||
sources.add(row.file_path)
|
||||
stateByPath.set(row.file_path, {
|
||||
|
|
@ -1231,9 +1273,7 @@ export class RagService {
|
|||
const chunksInQdrant = chunksBySource.get(source) ?? 0
|
||||
const fileName = source.split(/[/\\]/).pop() ?? source
|
||||
const expectedChunks =
|
||||
fileSizeBytes > 0
|
||||
? await KbRatioRegistry.estimateChunks(fileName, fileSizeBytes)
|
||||
: null
|
||||
fileSizeBytes > 0 ? await KbRatioRegistry.estimateChunks(fileName, fileSizeBytes) : null
|
||||
|
||||
const warnings = decideWarnings({ fileSizeBytes, chunksInQdrant, expectedChunks })
|
||||
if (warnings.length > 0) out[source] = warnings
|
||||
|
|
@ -1267,17 +1307,19 @@ export class RagService {
|
|||
logger.info(`[RAG] Deleted all points for source: ${source}`)
|
||||
|
||||
/** Delete the physical file only if it lives inside the uploads directory.
|
||||
* resolve() normalises path traversal sequences (e.g. "/../..") before the
|
||||
* check to prevent path traversal vulns
|
||||
* The trailing sep is to ensure a prefix like "kb_uploads_{something_incorrect}" can't slip through.
|
||||
*/
|
||||
* resolve() normalises path traversal sequences (e.g. "/../..") before the
|
||||
* check to prevent path traversal vulns
|
||||
* The trailing sep is to ensure a prefix like "kb_uploads_{something_incorrect}" can't slip through.
|
||||
*/
|
||||
const uploadsAbsPath = join(process.cwd(), RagService.UPLOADS_STORAGE_PATH)
|
||||
const resolvedSource = resolve(source)
|
||||
if (resolvedSource.startsWith(uploadsAbsPath + sep)) {
|
||||
await deleteFileIfExists(resolvedSource)
|
||||
logger.info(`[RAG] Deleted uploaded file from disk: ${resolvedSource}`)
|
||||
} else {
|
||||
logger.warn(`[RAG] File was removed from knowledge base but doesn't live in Nomad's uploads directory, so it can't be safely removed. Skipping deletion of physical file...`)
|
||||
logger.warn(
|
||||
`[RAG] File was removed from knowledge base but doesn't live in Nomad's uploads directory, so it can't be safely removed. Skipping deletion of physical file...`
|
||||
)
|
||||
}
|
||||
|
||||
// Drop the ingest state row last so the file disappears entirely. Without
|
||||
|
|
@ -1300,7 +1342,10 @@ export class RagService {
|
|||
const alreadyEmbeddedRaw = await KVStore.getValue('rag.docsEmbedded')
|
||||
if (alreadyEmbeddedRaw && !force) {
|
||||
logger.info('[RAG] Nomad docs have already been discovered and queued. Skipping.')
|
||||
return { success: true, message: 'Nomad docs have already been discovered and queued. Skipping.' }
|
||||
return {
|
||||
success: true,
|
||||
message: 'Nomad docs have already been discovered and queued. Skipping.',
|
||||
}
|
||||
}
|
||||
|
||||
const filesToEmbed: Array<{ path: string; source: string }> = []
|
||||
|
|
@ -1332,17 +1377,17 @@ export class RagService {
|
|||
})
|
||||
logger.info(`[RAG] Successfully dispatched job for ${fileInfo.source}`)
|
||||
} catch (fileError) {
|
||||
logger.error(
|
||||
`[RAG] Error dispatching job for file ${fileInfo.source}:`,
|
||||
fileError
|
||||
)
|
||||
logger.error(`[RAG] Error dispatching job for file ${fileInfo.source}:`, fileError)
|
||||
}
|
||||
}
|
||||
|
||||
// Update KV store to mark docs as discovered so we don't redo this unnecessarily
|
||||
await KVStore.setValue('rag.docsEmbedded', true)
|
||||
|
||||
return { success: true, message: `Nomad docs discovery completed. Dispatched ${filesToEmbed.length} embedding jobs.` }
|
||||
return {
|
||||
success: true,
|
||||
message: `Nomad docs discovery completed. Dispatched ${filesToEmbed.length} embedding jobs.`,
|
||||
}
|
||||
} catch (error) {
|
||||
logger.error('Error discovering Nomad docs:', error)
|
||||
return { success: false, message: 'Error discovering Nomad docs.' }
|
||||
|
|
@ -1350,18 +1395,20 @@ export class RagService {
|
|||
}
|
||||
|
||||
/**
|
||||
* Walk kb_uploads and zim storage directories, returning the full path of
|
||||
* Walk kb_uploads, local library, and zim storage directories, returning the full path of
|
||||
* every embeddable file. Non-embeddable types (e.g. kiwix-library.xml) are
|
||||
* filtered out so they aren't dispatched only to fail with "Unsupported file
|
||||
* type" and retry on every sync.
|
||||
*/
|
||||
private async _discoverKbFiles(): Promise<string[]> {
|
||||
const KB_UPLOADS_PATH = join(process.cwd(), RagService.UPLOADS_STORAGE_PATH)
|
||||
const LOCAL_LIBRARY_PATH = join(process.cwd(), LocalLibraryService.STORAGE_PATH)
|
||||
const ZIM_PATH = join(process.cwd(), ZIM_STORAGE_PATH)
|
||||
const filesInStorage: string[] = []
|
||||
|
||||
for (const [label, dirPath] of [
|
||||
[RagService.UPLOADS_STORAGE_PATH, KB_UPLOADS_PATH] as const,
|
||||
[LocalLibraryService.STORAGE_PATH, LOCAL_LIBRARY_PATH] as const,
|
||||
[ZIM_STORAGE_PATH, ZIM_PATH] as const,
|
||||
]) {
|
||||
try {
|
||||
|
|
@ -1461,7 +1508,8 @@ export class RagService {
|
|||
return {
|
||||
success: false,
|
||||
code: 'inflight',
|
||||
message: 'A job for this file is already in progress. Wait for it to finish before re-queuing.',
|
||||
message:
|
||||
'A job for this file is already in progress. Wait for it to finish before re-queuing.',
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1498,10 +1546,7 @@ export class RagService {
|
|||
* by reembedAll() where the file must remain so it can be re-ingested.
|
||||
*/
|
||||
private async _deletePointsBySource(source: string): Promise<void> {
|
||||
await this._ensureCollection(
|
||||
RagService.CONTENT_COLLECTION_NAME,
|
||||
RagService.EMBEDDING_DIMENSION
|
||||
)
|
||||
await this._ensureCollection(RagService.CONTENT_COLLECTION_NAME, RagService.EMBEDDING_DIMENSION)
|
||||
await this.qdrant!.delete(RagService.CONTENT_COLLECTION_NAME, {
|
||||
filter: { must: [{ key: 'source', match: { value: source } }] },
|
||||
})
|
||||
|
|
@ -1518,7 +1563,10 @@ export class RagService {
|
|||
const { QueueService } = await import('#services/queue_service')
|
||||
const queue = QueueService.getInstance().getQueue(EmbedFileJob.queue)
|
||||
const counts = await queue.getJobCounts('waiting', 'active', 'delayed', 'paused')
|
||||
return (counts.waiting || 0) + (counts.active || 0) + (counts.delayed || 0) + (counts.paused || 0) > 0
|
||||
return (
|
||||
(counts.waiting || 0) + (counts.active || 0) + (counts.delayed || 0) + (counts.paused || 0) >
|
||||
0
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -1684,7 +1732,8 @@ export class RagService {
|
|||
if (await this._hasInflightEmbedJobs()) {
|
||||
return {
|
||||
success: false,
|
||||
message: 'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a bulk re-embed.',
|
||||
message:
|
||||
'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a bulk re-embed.',
|
||||
}
|
||||
}
|
||||
|
||||
|
|
@ -1717,7 +1766,10 @@ export class RagService {
|
|||
try {
|
||||
await this._deletePointsBySource(filePath)
|
||||
} catch (err) {
|
||||
logger.error(`[RAG] Failed to delete prior points for ${filePath}; skipping dispatch:`, err)
|
||||
logger.error(
|
||||
`[RAG] Failed to delete prior points for ${filePath}; skipping dispatch:`,
|
||||
err
|
||||
)
|
||||
failedPaths.push(filePath)
|
||||
continue
|
||||
}
|
||||
|
|
@ -1732,7 +1784,10 @@ export class RagService {
|
|||
} catch (fileError) {
|
||||
// Old points already deleted but the new job never made it onto the
|
||||
// queue. Logged + surfaced so an operator can rerun a sync.
|
||||
logger.error(`[RAG] Re-embed dispatch failed for ${filePath} after delete; file is now unindexed until next sync:`, fileError)
|
||||
logger.error(
|
||||
`[RAG] Re-embed dispatch failed for ${filePath} after delete; file is now unindexed until next sync:`,
|
||||
fileError
|
||||
)
|
||||
failedPaths.push(filePath)
|
||||
}
|
||||
}
|
||||
|
|
@ -1781,7 +1836,8 @@ export class RagService {
|
|||
if (await this._hasInflightEmbedJobs()) {
|
||||
return {
|
||||
success: false,
|
||||
message: 'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a reset.',
|
||||
message:
|
||||
'Embed jobs are already in progress. Wait for the queue to drain (or clean up failed jobs) before triggering a reset.',
|
||||
}
|
||||
}
|
||||
|
||||
|
|
|
|||
|
|
@ -174,7 +174,9 @@ export function matchesDevice(fsPath: string, deviceName: string): boolean {
|
|||
return false
|
||||
}
|
||||
|
||||
export function determineFileType(filename: string): 'image' | 'pdf' | 'text' | 'epub' | 'zim' | 'unknown' {
|
||||
export function determineFileType(
|
||||
filename: string
|
||||
): 'image' | 'pdf' | 'text' | 'epub' | 'mobi' | 'zim' | 'unknown' {
|
||||
const ext = path.extname(filename).toLowerCase()
|
||||
if (['.jpg', '.jpeg', '.png', '.gif', '.bmp', '.tiff', '.webp'].includes(ext)) {
|
||||
return 'image'
|
||||
|
|
@ -184,6 +186,8 @@ export function determineFileType(filename: string): 'image' | 'pdf' | 'text' |
|
|||
return 'text'
|
||||
} else if (ext === '.epub') {
|
||||
return 'epub'
|
||||
} else if (['.mobi', '.azw', '.azw3'].includes(ext)) {
|
||||
return 'mobi'
|
||||
} else if (ext === '.zim') {
|
||||
return 'zim'
|
||||
} else {
|
||||
|
|
@ -198,4 +202,4 @@ export function determineFileType(filename: string): 'image' | 'pdf' | 'text' |
|
|||
*/
|
||||
export function sanitizeFilename(filename: string): string {
|
||||
return filename.replace(/[^a-zA-Z0-9._-]/g, '_')
|
||||
}
|
||||
}
|
||||
|
|
|
|||
|
|
@ -0,0 +1,18 @@
|
|||
import vine from '@vinejs/vine'
|
||||
|
||||
export const localLibraryFilenameValidator = vine.compile(
|
||||
vine.object({
|
||||
params: vine.object({
|
||||
filename: vine.string().minLength(1).maxLength(255),
|
||||
}),
|
||||
})
|
||||
)
|
||||
|
||||
export const localLibraryRenameValidator = vine.compile(
|
||||
vine.object({
|
||||
params: vine.object({
|
||||
filename: vine.string().minLength(1).maxLength(255),
|
||||
}),
|
||||
name: vine.string().trim().minLength(1).maxLength(255),
|
||||
})
|
||||
)
|
||||
|
|
@ -1,3 +1,16 @@
|
|||
import { KVStoreKey } from "../types/kv_store.js";
|
||||
import { KVStoreKey } from '../types/kv_store.js'
|
||||
|
||||
export const SETTINGS_KEYS: KVStoreKey[] = ['chat.suggestionsEnabled', 'chat.lastModel', 'ui.hasVisitedEasySetup', 'ui.theme', 'system.earlyAccess', 'ai.assistantCustomName', 'ai.remoteOllamaUrl', 'ai.ollamaFlashAttention', 'rag.defaultIngestPolicy'];
|
||||
export const SETTINGS_KEYS: KVStoreKey[] = [
|
||||
'chat.suggestionsEnabled',
|
||||
'chat.lastModel',
|
||||
'ui.hasVisitedEasySetup',
|
||||
'ui.theme',
|
||||
'system.earlyAccess',
|
||||
'ai.assistantCustomName',
|
||||
'ai.remoteOllamaUrl',
|
||||
'ai.provider',
|
||||
'ai.macNativeWorkerUrl',
|
||||
'ai.macNativeModelRoot',
|
||||
'ai.ollamaFlashAttention',
|
||||
'rag.defaultIngestPolicy',
|
||||
]
|
||||
|
|
|
|||
|
|
@ -2,15 +2,40 @@ import axios, { AxiosError, AxiosInstance } from 'axios'
|
|||
import { ListRemoteZimFilesResponse, ListZimFilesResponse } from '../../types/zim'
|
||||
import { ServiceSlim } from '../../types/services'
|
||||
import { FileEntry } from '../../types/files'
|
||||
import { CheckLatestVersionResult, SystemInformationResponse, SystemUpdateStatus } from '../../types/system'
|
||||
import {
|
||||
CheckLatestVersionResult,
|
||||
SystemInformationResponse,
|
||||
SystemUpdateStatus,
|
||||
} from '../../types/system'
|
||||
import { DownloadJobWithProgress, WikipediaState } from '../../types/downloads'
|
||||
import type { Country, CountryCode, CountryGroup, MapExtractPreflight } from '../../types/maps'
|
||||
import { EmbedJobWithProgress, FileWarningsResult, StoredFileInfo } from '../../types/rag'
|
||||
import type { CategoryWithStatus, CollectionWithStatus, ContentUpdateCheckResult, ResourceUpdateInfo } from '../../types/collections'
|
||||
import type {
|
||||
CategoryWithStatus,
|
||||
CollectionWithStatus,
|
||||
ContentUpdateCheckResult,
|
||||
ResourceUpdateInfo,
|
||||
} from '../../types/collections'
|
||||
import { catchInternal } from './util'
|
||||
import { NomadChatResponse, NomadInstalledModel, NomadOllamaModel, OllamaChatRequest } from '../../types/ollama'
|
||||
import {
|
||||
NomadChatResponse,
|
||||
NomadInstalledModel,
|
||||
NomadOllamaModel,
|
||||
OllamaChatRequest,
|
||||
} from '../../types/ollama'
|
||||
import BenchmarkResult from '#models/benchmark_result'
|
||||
import { BenchmarkType, RunBenchmarkResponse, SubmitBenchmarkResponse, UpdateBuilderTagResponse } from '../../types/benchmark'
|
||||
import {
|
||||
BenchmarkType,
|
||||
RunBenchmarkResponse,
|
||||
SubmitBenchmarkResponse,
|
||||
UpdateBuilderTagResponse,
|
||||
} from '../../types/benchmark'
|
||||
import type {
|
||||
LocalLibraryFile,
|
||||
LocalLibraryListResponse,
|
||||
LocalLibraryPreviewResponse,
|
||||
} from '../../types/local_library'
|
||||
import type { MacAiProvider, MacNativeStatus } from '../../types/mac_ai'
|
||||
|
||||
class API {
|
||||
private client: AxiosInstance
|
||||
|
|
@ -58,7 +83,9 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async configureRemoteOllama(remoteUrl: string | null): Promise<{ success: boolean; message: string }> {
|
||||
async configureRemoteOllama(
|
||||
remoteUrl: string | null
|
||||
): Promise<{ success: boolean; message: string }> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<{ success: boolean; message: string }>(
|
||||
'/ollama/configure-remote',
|
||||
|
|
@ -68,6 +95,27 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async getMacAiStatus(): Promise<MacNativeStatus | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<MacNativeStatus>('/mac-ai/status')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async configureMacAi(input: {
|
||||
provider: MacAiProvider
|
||||
workerUrl?: string | null
|
||||
modelRoot?: string | null
|
||||
}): Promise<{ success: boolean; message: string } | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<{ success: boolean; message: string }>(
|
||||
'/mac-ai/configure',
|
||||
input
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async deleteModel(model: string): Promise<{ success: boolean; message: string }> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.delete('/ollama/models', { data: { model } })
|
||||
|
|
@ -100,14 +148,20 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async downloadCategoryTier(categorySlug: string, tierSlug: string): Promise<{
|
||||
async downloadCategoryTier(
|
||||
categorySlug: string,
|
||||
tierSlug: string
|
||||
): Promise<{
|
||||
message: string
|
||||
categorySlug: string
|
||||
tierSlug: string
|
||||
resources: string[] | null
|
||||
}> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post('/zim/download-category-tier', { categorySlug, tierSlug })
|
||||
const response = await this.client.post('/zim/download-category-tier', {
|
||||
categorySlug,
|
||||
tierSlug,
|
||||
})
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -188,11 +242,14 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async refreshManifests(): Promise<{ success: boolean; changed: Record<string, boolean> } | undefined> {
|
||||
async refreshManifests(): Promise<
|
||||
{ success: boolean; changed: Record<string, boolean> } | undefined
|
||||
> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<{ success: boolean; changed: Record<string, boolean> }>(
|
||||
'/manifests/refresh'
|
||||
)
|
||||
const response = await this.client.post<{
|
||||
success: boolean
|
||||
changed: Record<string, boolean>
|
||||
}>('/manifests/refresh')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -243,10 +300,9 @@ class API {
|
|||
|
||||
async getChatSuggestions(signal?: AbortSignal) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<{ suggestions: string[] }>(
|
||||
'/chat/suggestions',
|
||||
{ signal }
|
||||
)
|
||||
const response = await this.client.get<{ suggestions: string[] }>('/chat/suggestions', {
|
||||
signal,
|
||||
})
|
||||
return response.data.suggestions
|
||||
})()
|
||||
}
|
||||
|
|
@ -290,7 +346,12 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async getAvailableModels(params: { query?: string; recommendedOnly?: boolean; limit?: number; force?: boolean }) {
|
||||
async getAvailableModels(params: {
|
||||
query?: string
|
||||
recommendedOnly?: boolean
|
||||
limit?: number
|
||||
force?: boolean
|
||||
}) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<{
|
||||
models: NomadOllamaModel[]
|
||||
|
|
@ -344,15 +405,13 @@ class API {
|
|||
let data: any
|
||||
try {
|
||||
data = JSON.parse(line.slice(6))
|
||||
} catch { continue /* skip malformed chunks */ }
|
||||
} catch {
|
||||
continue /* skip malformed chunks */
|
||||
}
|
||||
|
||||
if (data.error) throw new Error('The model encountered an error. Please try again.')
|
||||
|
||||
onChunk(
|
||||
data.message?.content ?? '',
|
||||
data.message?.thinking ?? '',
|
||||
data.done ?? false
|
||||
)
|
||||
onChunk(data.message?.content ?? '', data.message?.thinking ?? '', data.done ?? false)
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
|
|
@ -362,14 +421,18 @@ class API {
|
|||
|
||||
async getBenchmarkResults() {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<{ results: BenchmarkResult[], total: number }>('/benchmark/results')
|
||||
const response = await this.client.get<{ results: BenchmarkResult[]; total: number }>(
|
||||
'/benchmark/results'
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async getLatestBenchmarkResult() {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<{ result: BenchmarkResult | null }>('/benchmark/results/latest')
|
||||
const response = await this.client.get<{ result: BenchmarkResult | null }>(
|
||||
'/benchmark/results/latest'
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -472,9 +535,15 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async cleanupFailedEmbedJobs(): Promise<{ message: string; cleaned: number; filesDeleted: number } | undefined> {
|
||||
async cleanupFailedEmbedJobs(): Promise<
|
||||
{ message: string; cleaned: number; filesDeleted: number } | undefined
|
||||
> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.delete<{ message: string; cleaned: number; filesDeleted: number }>('/rag/failed-jobs')
|
||||
const response = await this.client.delete<{
|
||||
message: string
|
||||
cleaned: number
|
||||
filesDeleted: number
|
||||
}>('/rag/failed-jobs')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -495,7 +564,10 @@ class API {
|
|||
|
||||
async embedSingleRAGFile(source: string, force: boolean = false) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<{ message: string }>('/rag/files/embed', { source, force })
|
||||
const response = await this.client.post<{ message: string }>('/rag/files/embed', {
|
||||
source,
|
||||
force,
|
||||
})
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -509,7 +581,9 @@ class API {
|
|||
|
||||
async deleteRAGFile(source: string) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.delete<{ message: string }>('/rag/files', { data: { source } })
|
||||
const response = await this.client.delete<{ message: string }>('/rag/files', {
|
||||
data: { source },
|
||||
})
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -632,9 +706,7 @@ class API {
|
|||
|
||||
async listCuratedMapCollections() {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<CollectionWithStatus[]>(
|
||||
'/maps/curated-collections'
|
||||
)
|
||||
const response = await this.client.get<CollectionWithStatus[]>('/maps/curated-collections')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -663,26 +735,49 @@ class API {
|
|||
async listMapMarkers() {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<
|
||||
Array<{ id: number; name: string; longitude: number; latitude: number; color: string; notes: string | null; created_at: string }>
|
||||
Array<{
|
||||
id: number
|
||||
name: string
|
||||
longitude: number
|
||||
latitude: number
|
||||
color: string
|
||||
notes: string | null
|
||||
created_at: string
|
||||
}>
|
||||
>('/maps/markers')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async createMapMarker(data: { name: string; longitude: number; latitude: number; color?: string }) {
|
||||
async createMapMarker(data: {
|
||||
name: string
|
||||
longitude: number
|
||||
latitude: number
|
||||
color?: string
|
||||
}) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<
|
||||
{ id: number; name: string; longitude: number; latitude: number; color: string; notes: string | null; created_at: string }
|
||||
>('/maps/markers', data)
|
||||
const response = await this.client.post<{
|
||||
id: number
|
||||
name: string
|
||||
longitude: number
|
||||
latitude: number
|
||||
color: string
|
||||
notes: string | null
|
||||
created_at: string
|
||||
}>('/maps/markers', data)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async updateMapMarker(id: number, data: { name?: string; color?: string }) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.patch<
|
||||
{ id: number; name: string; longitude: number; latitude: number; color: string }
|
||||
>(`/maps/markers/${id}`, data)
|
||||
const response = await this.client.patch<{
|
||||
id: number
|
||||
name: string
|
||||
longitude: number
|
||||
latitude: number
|
||||
color: string
|
||||
}>(`/maps/markers/${id}`, data)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -715,9 +810,10 @@ class API {
|
|||
|
||||
async listCustomLibraries() {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<{ id: number; name: string; base_url: string; is_default: boolean }[]>(
|
||||
'/zim/custom-libraries'
|
||||
)
|
||||
const response =
|
||||
await this.client.get<
|
||||
{ id: number; name: string; base_url: string; is_default: boolean }[]
|
||||
>('/zim/custom-libraries')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -776,7 +872,9 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async cancelDownloadJob(jobId: string): Promise<{ success: boolean; message: string } | undefined> {
|
||||
async cancelDownloadJob(
|
||||
jobId: string
|
||||
): Promise<{ success: boolean; message: string } | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<{ success: boolean; message: string }>(
|
||||
`/downloads/jobs/${jobId}/cancel`
|
||||
|
|
@ -789,7 +887,7 @@ class API {
|
|||
return catchInternal(async () => {
|
||||
const response = await this.client.post<RunBenchmarkResponse>(
|
||||
`/benchmark/run${sync ? '?sync=true' : ''}`,
|
||||
{ benchmark_type: type },
|
||||
{ benchmark_type: type }
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
|
|
@ -806,17 +904,24 @@ class API {
|
|||
|
||||
async submitBenchmark(benchmark_id: string, anonymous: boolean) {
|
||||
try {
|
||||
const response = await this.client.post<SubmitBenchmarkResponse>('/benchmark/submit', { benchmark_id, anonymous })
|
||||
const response = await this.client.post<SubmitBenchmarkResponse>('/benchmark/submit', {
|
||||
benchmark_id,
|
||||
anonymous,
|
||||
})
|
||||
return response.data
|
||||
} catch (error: any) {
|
||||
// For 409 Conflict errors, throw a specific error that the UI can handle
|
||||
if (error.response?.status === 409) {
|
||||
const err = new Error(error.response?.data?.error || 'This benchmark has already been submitted to the repository')
|
||||
; (err as any).status = 409
|
||||
const err = new Error(
|
||||
error.response?.data?.error ||
|
||||
'This benchmark has already been submitted to the repository'
|
||||
)
|
||||
;(err as any).status = 409
|
||||
throw err
|
||||
}
|
||||
// For other errors, extract the message and throw
|
||||
const errorMessage = error.response?.data?.error || error.message || 'Failed to submit benchmark'
|
||||
const errorMessage =
|
||||
error.response?.data?.error || error.message || 'Failed to submit benchmark'
|
||||
throw new Error(errorMessage)
|
||||
}
|
||||
}
|
||||
|
|
@ -913,10 +1018,10 @@ class API {
|
|||
|
||||
async updateBuilderTag(benchmark_id: string, builder_tag: string) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<UpdateBuilderTagResponse>(
|
||||
'/benchmark/builder-tag',
|
||||
{ benchmark_id, builder_tag }
|
||||
)
|
||||
const response = await this.client.post<UpdateBuilderTagResponse>('/benchmark/builder-tag', {
|
||||
benchmark_id,
|
||||
builder_tag,
|
||||
})
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
@ -938,12 +1043,75 @@ class API {
|
|||
})()
|
||||
}
|
||||
|
||||
async listLocalLibraryFiles(): Promise<LocalLibraryListResponse | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<LocalLibraryListResponse>('/local-library/files')
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async uploadLocalLibraryFile(file: File): Promise<LocalLibraryListResponse | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const formData = new FormData()
|
||||
formData.append('file', file)
|
||||
const response = await this.client.post<LocalLibraryListResponse>(
|
||||
'/local-library/files',
|
||||
formData,
|
||||
{ headers: { 'Content-Type': 'multipart/form-data' } }
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async previewLocalLibraryFile(
|
||||
filename: string
|
||||
): Promise<LocalLibraryPreviewResponse | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<LocalLibraryPreviewResponse>(
|
||||
`/local-library/files/${encodeURIComponent(filename)}/preview`
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async renameLocalLibraryFile(
|
||||
filename: string,
|
||||
name: string
|
||||
): Promise<{ file: LocalLibraryFile } | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.patch<{ file: LocalLibraryFile }>(
|
||||
`/local-library/files/${encodeURIComponent(filename)}`,
|
||||
{ name }
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async deleteLocalLibraryFile(filename: string): Promise<{ message: string } | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.delete<{ message: string }>(
|
||||
`/local-library/files/${encodeURIComponent(filename)}`
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async indexLocalLibraryFile(
|
||||
filename: string
|
||||
): Promise<{ message: string; jobId?: string } | undefined> {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.post<{ message: string; jobId?: string }>(
|
||||
`/local-library/files/${encodeURIComponent(filename)}/index`
|
||||
)
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
||||
async getSetting(key: string) {
|
||||
return catchInternal(async () => {
|
||||
const response = await this.client.get<{ key: string; value: any }>(
|
||||
'/system/settings',
|
||||
{ params: { key } }
|
||||
)
|
||||
const response = await this.client.get<{ key: string; value: any }>('/system/settings', {
|
||||
params: { key },
|
||||
})
|
||||
return response.data
|
||||
})()
|
||||
}
|
||||
|
|
|
|||
|
|
@ -1,6 +1,7 @@
|
|||
import {
|
||||
IconBolt,
|
||||
IconHelp,
|
||||
IconLibrary,
|
||||
IconMapRoute,
|
||||
IconPlus,
|
||||
IconSettings,
|
||||
|
|
@ -28,6 +29,17 @@ const MAPS_ITEM = {
|
|||
poweredBy: null,
|
||||
}
|
||||
|
||||
const LOCAL_LIBRARY_ITEM = {
|
||||
label: 'Local Library',
|
||||
to: '/local-library',
|
||||
target: '',
|
||||
description: 'View PDFs and personal offline documents',
|
||||
icon: <IconLibrary size={48} />,
|
||||
installed: true,
|
||||
displayOrder: 5,
|
||||
poweredBy: null,
|
||||
}
|
||||
|
||||
// System items shown after all apps
|
||||
const SYSTEM_ITEMS = [
|
||||
{
|
||||
|
|
@ -90,14 +102,16 @@ export default function Home(props: {
|
|||
}
|
||||
}) {
|
||||
const items: DashboardItem[] = []
|
||||
const updateInfo = useUpdateAvailable();
|
||||
const updateInfo = useUpdateAvailable()
|
||||
const { aiAssistantName } = usePage<{ aiAssistantName: string }>().props
|
||||
|
||||
// Check if user has visited Easy Setup
|
||||
const { data: easySetupVisited } = useSystemSetting({
|
||||
key: 'ui.hasVisitedEasySetup'
|
||||
key: 'ui.hasVisitedEasySetup',
|
||||
})
|
||||
const shouldHighlightEasySetup = easySetupVisited?.value ? String(easySetupVisited.value) !== 'true' : false
|
||||
const shouldHighlightEasySetup = easySetupVisited?.value
|
||||
? String(easySetupVisited.value) !== 'true'
|
||||
: false
|
||||
|
||||
// Add installed services (non-dependency services only)
|
||||
props.system.services
|
||||
|
|
@ -105,7 +119,10 @@ export default function Home(props: {
|
|||
.forEach((service) => {
|
||||
items.push({
|
||||
// Inject custom AI Assistant name if this is the chat service
|
||||
label: service.service_name === SERVICE_NAMES.OLLAMA && aiAssistantName ? aiAssistantName : (service.friendly_name || service.service_name),
|
||||
label:
|
||||
service.service_name === SERVICE_NAMES.OLLAMA && aiAssistantName
|
||||
? aiAssistantName
|
||||
: service.friendly_name || service.service_name,
|
||||
to: service.ui_location ? getServiceLink(service.ui_location) : '#',
|
||||
target: '_blank',
|
||||
description:
|
||||
|
|
@ -124,6 +141,7 @@ export default function Home(props: {
|
|||
|
||||
// Add Maps as a Core Capability
|
||||
items.push(MAPS_ITEM)
|
||||
items.push(LOCAL_LIBRARY_ITEM)
|
||||
|
||||
// Add system items
|
||||
items.push(...SYSTEM_ITEMS)
|
||||
|
|
@ -134,24 +152,22 @@ export default function Home(props: {
|
|||
return (
|
||||
<AppLayout>
|
||||
<Head title="Command Center" />
|
||||
{
|
||||
updateInfo?.updateAvailable && (
|
||||
<div className='flex justify-center items-center p-4 w-full'>
|
||||
<Alert
|
||||
title="An update is available for Project N.O.M.A.D.!"
|
||||
type="info-inverted"
|
||||
variant="solid"
|
||||
className="w-full"
|
||||
buttonProps={{
|
||||
variant: 'primary',
|
||||
children: 'Go to Settings',
|
||||
icon: 'IconSettings',
|
||||
onClick: () => router.visit('/settings/update'),
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
{updateInfo?.updateAvailable && (
|
||||
<div className="flex justify-center items-center p-4 w-full">
|
||||
<Alert
|
||||
title="An update is available for Project N.O.M.A.D.!"
|
||||
type="info-inverted"
|
||||
variant="solid"
|
||||
className="w-full"
|
||||
buttonProps={{
|
||||
variant: 'primary',
|
||||
children: 'Go to Settings',
|
||||
icon: 'IconSettings',
|
||||
onClick: () => router.visit('/settings/update'),
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 gap-4 p-4">
|
||||
{items.map((item) => {
|
||||
const isEasySetup = item.label === 'Easy Setup'
|
||||
|
|
|
|||
|
|
@ -0,0 +1,252 @@
|
|||
import { Head } from '@inertiajs/react'
|
||||
import { useEffect, useMemo, useState } from 'react'
|
||||
import {
|
||||
IconBook,
|
||||
IconDownload,
|
||||
IconFileText,
|
||||
IconFileTypePdf,
|
||||
IconRefresh,
|
||||
IconSearch,
|
||||
IconUpload,
|
||||
} from '@tabler/icons-react'
|
||||
import AppLayout from '~/layouts/AppLayout'
|
||||
import StyledButton from '~/components/StyledButton'
|
||||
import Input from '~/components/inputs/Input'
|
||||
import api from '~/lib/api'
|
||||
import { formatBytes } from '~/lib/util'
|
||||
import type { LocalLibraryFile, LocalLibraryPreviewResponse } from '../../types/local_library'
|
||||
import { useNotifications } from '~/context/NotificationContext'
|
||||
|
||||
function fileIcon(type: LocalLibraryFile['type']) {
|
||||
if (type === 'pdf') return <IconFileTypePdf className="h-8 w-8" />
|
||||
if (type === 'epub') return <IconBook className="h-8 w-8" />
|
||||
return <IconFileText className="h-8 w-8" />
|
||||
}
|
||||
|
||||
export default function LocalLibraryPage() {
|
||||
const { addNotification } = useNotifications()
|
||||
const [files, setFiles] = useState<LocalLibraryFile[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [uploading, setUploading] = useState(false)
|
||||
const [query, setQuery] = useState('')
|
||||
const [selected, setSelected] = useState<LocalLibraryFile | null>(null)
|
||||
const [preview, setPreview] = useState<LocalLibraryPreviewResponse | null>(null)
|
||||
const [previewLoading, setPreviewLoading] = useState(false)
|
||||
|
||||
async function refresh() {
|
||||
setLoading(true)
|
||||
const result = await api.listLocalLibraryFiles()
|
||||
setFiles(result?.files ?? [])
|
||||
setLoading(false)
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
refresh()
|
||||
}, [])
|
||||
|
||||
const filteredFiles = useMemo(() => {
|
||||
const normalized = query.trim().toLowerCase()
|
||||
if (!normalized) return files
|
||||
return files.filter((file) => file.displayName.toLowerCase().includes(normalized))
|
||||
}, [files, query])
|
||||
|
||||
async function selectFile(file: LocalLibraryFile) {
|
||||
setSelected(file)
|
||||
setPreview(null)
|
||||
if (file.type === 'epub' || file.type === 'text') {
|
||||
setPreviewLoading(true)
|
||||
const result = await api.previewLocalLibraryFile(file.name)
|
||||
setPreview(result ?? null)
|
||||
setPreviewLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function uploadFile(file: File | undefined) {
|
||||
if (!file) return
|
||||
setUploading(true)
|
||||
const result = await api.uploadLocalLibraryFile(file)
|
||||
if (result?.files) {
|
||||
setFiles(result.files)
|
||||
addNotification({ message: 'File added to Local Library.', type: 'success' })
|
||||
}
|
||||
setUploading(false)
|
||||
}
|
||||
|
||||
async function deleteFile(file: LocalLibraryFile) {
|
||||
await api.deleteLocalLibraryFile(file.name)
|
||||
if (selected?.name === file.name) {
|
||||
setSelected(null)
|
||||
setPreview(null)
|
||||
}
|
||||
await refresh()
|
||||
}
|
||||
|
||||
async function indexFile(file: LocalLibraryFile) {
|
||||
const result = await api.indexLocalLibraryFile(file.name)
|
||||
addNotification({
|
||||
message: result?.message ?? 'Indexing queued for this file.',
|
||||
type: 'success',
|
||||
})
|
||||
}
|
||||
|
||||
return (
|
||||
<AppLayout>
|
||||
<Head title="Local Library | Project N.O.M.A.D." />
|
||||
<div className="flex h-[calc(100vh-5rem)] min-h-[720px] bg-surface-primary">
|
||||
<aside className="w-full max-w-sm border-r border-border-subtle bg-surface-secondary flex flex-col">
|
||||
<div className="p-4 border-b border-border-subtle">
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<h1 className="text-2xl font-semibold text-text-primary">Local Library</h1>
|
||||
<p className="text-sm text-text-muted">PDFs, eBooks, manuals, and field notes.</p>
|
||||
</div>
|
||||
<StyledButton variant="ghost" icon="IconRefresh" size="sm" onClick={refresh}>
|
||||
Refresh
|
||||
</StyledButton>
|
||||
</div>
|
||||
<label className="mt-4 flex cursor-pointer items-center justify-center gap-2 rounded-md border-2 border-dashed border-desert-green p-4 text-sm font-semibold text-desert-green hover:bg-desert-sand">
|
||||
<IconUpload className="h-5 w-5" />
|
||||
{uploading ? 'Uploading...' : 'Add Files'}
|
||||
<input
|
||||
type="file"
|
||||
className="hidden"
|
||||
accept=".pdf,.epub,.mobi,.azw,.azw3,.txt,.md,.rtf,.docx"
|
||||
onChange={(event) => uploadFile(event.target.files?.[0])}
|
||||
disabled={uploading}
|
||||
/>
|
||||
</label>
|
||||
<Input
|
||||
name="library-search"
|
||||
label=""
|
||||
placeholder="Search library"
|
||||
value={query}
|
||||
onChange={(event) => setQuery(event.target.value)}
|
||||
className="mt-4"
|
||||
leftIcon={<IconSearch className="h-5 w-5 text-text-muted" />}
|
||||
/>
|
||||
</div>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-3">
|
||||
{loading ? (
|
||||
<div className="flex h-full items-center justify-center text-text-muted">
|
||||
<IconRefresh className="mr-2 h-5 w-5 animate-spin" />
|
||||
Loading library
|
||||
</div>
|
||||
) : filteredFiles.length === 0 ? (
|
||||
<p className="p-4 text-sm text-text-muted">No local library files found.</p>
|
||||
) : (
|
||||
<div className="space-y-2">
|
||||
{filteredFiles.map((file) => (
|
||||
<button
|
||||
key={file.name}
|
||||
type="button"
|
||||
onClick={() => selectFile(file)}
|
||||
className={`w-full rounded border p-3 text-left transition-colors ${
|
||||
selected?.name === file.name
|
||||
? 'border-desert-green bg-white'
|
||||
: 'border-border-subtle bg-surface-primary hover:border-desert-green'
|
||||
}`}
|
||||
>
|
||||
<div className="flex items-start gap-3">
|
||||
<span className="text-desert-green">{fileIcon(file.type)}</span>
|
||||
<span className="min-w-0 flex-1">
|
||||
<span className="block truncate font-semibold text-text-primary">
|
||||
{file.displayName}
|
||||
</span>
|
||||
<span className="text-xs uppercase text-text-muted">{file.type}</span>
|
||||
<span className="ml-2 text-xs text-text-muted">
|
||||
{formatBytes(file.size)}
|
||||
</span>
|
||||
</span>
|
||||
</div>
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</aside>
|
||||
|
||||
<main className="flex min-w-0 flex-1 flex-col">
|
||||
{!selected ? (
|
||||
<div className="flex h-full items-center justify-center text-center">
|
||||
<div>
|
||||
<IconBook className="mx-auto mb-4 h-16 w-16 text-desert-green" />
|
||||
<h2 className="text-2xl font-semibold text-text-primary">Choose a file</h2>
|
||||
<p className="mt-2 text-text-muted">
|
||||
PDFs open here without leaving Command Center.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<div className="flex flex-wrap items-center justify-between gap-3 border-b border-border-subtle p-4">
|
||||
<div className="min-w-0">
|
||||
<h2 className="truncate text-xl font-semibold text-text-primary">
|
||||
{selected.displayName}
|
||||
</h2>
|
||||
<p className="text-sm text-text-muted">
|
||||
{selected.type.toUpperCase()} • {formatBytes(selected.size)}
|
||||
</p>
|
||||
</div>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{selected.canIndex && (
|
||||
<StyledButton
|
||||
variant="action"
|
||||
icon="IconBrain"
|
||||
onClick={() => indexFile(selected)}
|
||||
>
|
||||
Index for AI
|
||||
</StyledButton>
|
||||
)}
|
||||
<a href={selected.downloadUrl}>
|
||||
<StyledButton variant="secondary">
|
||||
<IconDownload className="mr-2 h-4 w-4" />
|
||||
Download
|
||||
</StyledButton>
|
||||
</a>
|
||||
<StyledButton
|
||||
variant="danger"
|
||||
icon="IconTrash"
|
||||
onClick={() => deleteFile(selected)}
|
||||
>
|
||||
Delete
|
||||
</StyledButton>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="min-h-0 flex-1 overflow-hidden bg-white">
|
||||
{selected.type === 'pdf' && selected.viewUrl ? (
|
||||
<iframe
|
||||
title={selected.displayName}
|
||||
src={selected.viewUrl}
|
||||
className="h-full w-full"
|
||||
/>
|
||||
) : selected.type === 'epub' || selected.type === 'text' ? (
|
||||
<article className="h-full overflow-y-auto px-8 py-6 text-text-primary">
|
||||
{previewLoading ? (
|
||||
<p className="text-text-muted">Loading preview...</p>
|
||||
) : (
|
||||
<div className="mx-auto max-w-4xl whitespace-pre-wrap leading-7">
|
||||
{preview?.text || 'No readable text preview was found.'}
|
||||
</div>
|
||||
)}
|
||||
</article>
|
||||
) : (
|
||||
<div className="flex h-full items-center justify-center text-center text-text-muted">
|
||||
<div>
|
||||
<IconFileText className="mx-auto mb-4 h-12 w-12" />
|
||||
<p>
|
||||
This file is stored locally and can be downloaded, but preview is not
|
||||
available.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</>
|
||||
)}
|
||||
</main>
|
||||
</div>
|
||||
</AppLayout>
|
||||
)
|
||||
}
|
||||
|
|
@ -21,12 +21,21 @@ import { formatBytes } from '~/lib/util'
|
|||
import useDebounce from '~/hooks/useDebounce'
|
||||
import ActiveModelDownloads from '~/components/ActiveModelDownloads'
|
||||
import { useSystemInfo } from '~/hooks/useSystemInfo'
|
||||
import type { MacAiProvider } from '../../../types/mac_ai'
|
||||
|
||||
export default function ModelsPage(props: {
|
||||
models: {
|
||||
availableModels: NomadOllamaModel[]
|
||||
installedModels: NomadInstalledModel[]
|
||||
settings: { chatSuggestionsEnabled: boolean; aiAssistantCustomName: string; remoteOllamaUrl: string; ollamaFlashAttention: boolean }
|
||||
settings: {
|
||||
chatSuggestionsEnabled: boolean
|
||||
aiAssistantCustomName: string
|
||||
remoteOllamaUrl: string
|
||||
aiProvider: MacAiProvider
|
||||
macNativeWorkerUrl: string
|
||||
macNativeModelRoot: string
|
||||
ollamaFlashAttention: boolean
|
||||
}
|
||||
}
|
||||
}) {
|
||||
const { aiAssistantName } = usePage<{ aiAssistantName: string }>().props
|
||||
|
|
@ -68,7 +77,9 @@ export default function ModelsPage(props: {
|
|||
message: `${aiAssistantName} is being reinstalled with GPU support. This page will reload shortly.`,
|
||||
type: 'success',
|
||||
})
|
||||
try { localStorage.removeItem('nomad:gpu-banner-dismissed') } catch {}
|
||||
try {
|
||||
localStorage.removeItem('nomad:gpu-banner-dismissed')
|
||||
} catch {}
|
||||
setTimeout(() => window.location.reload(), 5000)
|
||||
} catch (error) {
|
||||
addNotification({
|
||||
|
|
@ -84,9 +95,9 @@ export default function ModelsPage(props: {
|
|||
cancelText="Cancel"
|
||||
>
|
||||
<p className="text-text-primary">
|
||||
This will recreate the {aiAssistantName} container with GPU support enabled.
|
||||
Your downloaded models will be preserved. The service will be briefly
|
||||
unavailable during reinstall.
|
||||
This will recreate the {aiAssistantName} container with GPU support enabled. Your
|
||||
downloaded models will be preserved. The service will be briefly unavailable during
|
||||
reinstall.
|
||||
</p>
|
||||
</StyledModal>,
|
||||
'gpu-health-force-reinstall-modal'
|
||||
|
|
@ -102,9 +113,21 @@ export default function ModelsPage(props: {
|
|||
props.models.settings.aiAssistantCustomName
|
||||
)
|
||||
const [remoteOllamaUrl, setRemoteOllamaUrl] = useState(props.models.settings.remoteOllamaUrl)
|
||||
const [aiProvider, setAiProvider] = useState<MacAiProvider>(props.models.settings.aiProvider)
|
||||
const [macNativeWorkerUrl, setMacNativeWorkerUrl] = useState(
|
||||
props.models.settings.macNativeWorkerUrl
|
||||
)
|
||||
const [macNativeModelRoot, setMacNativeModelRoot] = useState(
|
||||
props.models.settings.macNativeModelRoot
|
||||
)
|
||||
const [remoteOllamaError, setRemoteOllamaError] = useState<string | null>(null)
|
||||
const [remoteOllamaSaving, setRemoteOllamaSaving] = useState(false)
|
||||
|
||||
const { data: macAiStatus, refetch: refetchMacAiStatus } = useQuery({
|
||||
queryKey: ['mac-ai', 'status'],
|
||||
queryFn: async () => api.getMacAiStatus(),
|
||||
})
|
||||
|
||||
async function handleSaveRemoteOllama() {
|
||||
setRemoteOllamaError(null)
|
||||
setRemoteOllamaSaving(true)
|
||||
|
|
@ -115,7 +138,8 @@ export default function ModelsPage(props: {
|
|||
router.reload()
|
||||
}
|
||||
} catch (error: any) {
|
||||
const msg = error?.response?.data?.message || error?.message || 'Failed to configure remote Ollama.'
|
||||
const msg =
|
||||
error?.response?.data?.message || error?.message || 'Failed to configure remote Ollama.'
|
||||
setRemoteOllamaError(msg)
|
||||
} finally {
|
||||
setRemoteOllamaSaving(false)
|
||||
|
|
@ -139,6 +163,31 @@ export default function ModelsPage(props: {
|
|||
}
|
||||
}
|
||||
|
||||
async function handleSaveMacAiProvider(providerOverride?: MacAiProvider) {
|
||||
const nextProvider = providerOverride ?? aiProvider
|
||||
setRemoteOllamaError(null)
|
||||
setRemoteOllamaSaving(true)
|
||||
try {
|
||||
const res = await api.configureMacAi({
|
||||
provider: nextProvider,
|
||||
workerUrl: macNativeWorkerUrl || null,
|
||||
modelRoot: macNativeModelRoot || null,
|
||||
})
|
||||
if (res?.success) {
|
||||
setAiProvider(nextProvider)
|
||||
addNotification({ message: res.message, type: 'success' })
|
||||
await refetchMacAiStatus()
|
||||
router.reload()
|
||||
}
|
||||
} catch (error: any) {
|
||||
setRemoteOllamaError(
|
||||
error?.response?.data?.message || error?.message || 'Failed to configure AI provider.'
|
||||
)
|
||||
} finally {
|
||||
setRemoteOllamaSaving(false)
|
||||
}
|
||||
}
|
||||
|
||||
const [query, setQuery] = useState('')
|
||||
const [queryUI, setQueryUI] = useState('')
|
||||
const [limit, setLimit] = useState(15)
|
||||
|
|
@ -150,7 +199,11 @@ export default function ModelsPage(props: {
|
|||
const forceRefreshRef = useRef(false)
|
||||
const [isForceRefreshing, setIsForceRefreshing] = useState(false)
|
||||
|
||||
const { data: availableModelData, isFetching, refetch } = useQuery({
|
||||
const {
|
||||
data: availableModelData,
|
||||
isFetching,
|
||||
refetch,
|
||||
} = useQuery({
|
||||
queryKey: ['ollama', 'availableModels', query, limit],
|
||||
queryFn: async () => {
|
||||
const force = forceRefreshRef.current
|
||||
|
|
@ -278,26 +331,28 @@ export default function ModelsPage(props: {
|
|||
className="!mt-6"
|
||||
/>
|
||||
)}
|
||||
{isInstalled && systemInfo?.gpuHealth?.status === 'passthrough_failed' && !gpuBannerDismissed && (
|
||||
<Alert
|
||||
type="warning"
|
||||
variant="bordered"
|
||||
title="GPU Not Accessible"
|
||||
message={`Your system has ${systemInfo?.gpuHealth?.gpuVendor === 'amd' ? 'an AMD' : 'an NVIDIA'} GPU, but ${aiAssistantName} can't access it. AI is running on CPU only, which is significantly slower.`}
|
||||
className="!mt-6"
|
||||
dismissible={true}
|
||||
onDismiss={handleDismissGpuBanner}
|
||||
buttonProps={{
|
||||
children: `Fix: Reinstall ${aiAssistantName}`,
|
||||
icon: 'IconRefresh',
|
||||
variant: 'action',
|
||||
size: 'sm',
|
||||
onClick: handleForceReinstallOllama,
|
||||
loading: reinstalling,
|
||||
disabled: reinstalling,
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
{isInstalled &&
|
||||
systemInfo?.gpuHealth?.status === 'passthrough_failed' &&
|
||||
!gpuBannerDismissed && (
|
||||
<Alert
|
||||
type="warning"
|
||||
variant="bordered"
|
||||
title="GPU Not Accessible"
|
||||
message={`Your system has ${systemInfo?.gpuHealth?.gpuVendor === 'amd' ? 'an AMD' : 'an NVIDIA'} GPU, but ${aiAssistantName} can't access it. AI is running on CPU only, which is significantly slower.`}
|
||||
className="!mt-6"
|
||||
dismissible={true}
|
||||
onDismiss={handleDismissGpuBanner}
|
||||
buttonProps={{
|
||||
children: `Fix: Reinstall ${aiAssistantName}`,
|
||||
icon: 'IconRefresh',
|
||||
variant: 'action',
|
||||
size: 'sm',
|
||||
onClick: handleForceReinstallOllama,
|
||||
loading: reinstalling,
|
||||
disabled: reinstalling,
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
|
||||
<StyledSectionHeader title="Settings" className="mt-8 mb-4" />
|
||||
<div className="bg-surface-primary rounded-lg border-2 border-border-subtle p-6">
|
||||
|
|
@ -323,7 +378,7 @@ export default function ModelsPage(props: {
|
|||
<Input
|
||||
name="aiAssistantCustomName"
|
||||
label="Assistant Name"
|
||||
helpText='Give your AI assistant a custom name that will be used in the chat interface and other areas of the application.'
|
||||
helpText="Give your AI assistant a custom name that will be used in the chat interface and other areas of the application."
|
||||
placeholder="AI Assistant"
|
||||
value={aiAssistantCustomName}
|
||||
onChange={(e) => setAiAssistantCustomName(e.target.value)}
|
||||
|
|
@ -397,9 +452,29 @@ export default function ModelsPage(props: {
|
|||
<StyledSectionHeader title="Remote Connection" className="mt-8 mb-4" />
|
||||
<div className="bg-surface-primary rounded-lg border-2 border-border-subtle p-6">
|
||||
<p className="text-sm text-text-secondary mb-4">
|
||||
Connect to any OpenAI-compatible API server — Ollama, LM Studio, llama.cpp, and others are all supported.
|
||||
For remote Ollama instances, the host must be started with <code className="bg-surface-secondary px-1 rounded">OLLAMA_HOST=0.0.0.0</code>.
|
||||
Connect to any OpenAI-compatible API server — Ollama, LM Studio, llama.cpp, and others
|
||||
are all supported. For remote Ollama instances, the host must be started with{' '}
|
||||
<code className="bg-surface-secondary px-1 rounded">OLLAMA_HOST=0.0.0.0</code>.
|
||||
</p>
|
||||
<div className="mb-5 grid grid-cols-1 md:grid-cols-4 gap-2">
|
||||
{(
|
||||
[
|
||||
['ollama', 'Ollama'],
|
||||
['remote', 'Remote API'],
|
||||
['native_mlx', 'Native MLX'],
|
||||
['native_coreml', 'Native Core ML'],
|
||||
] as Array<[MacAiProvider, string]>
|
||||
).map(([provider, label]) => (
|
||||
<StyledButton
|
||||
key={provider}
|
||||
variant={aiProvider === provider ? 'primary' : 'outline'}
|
||||
onClick={() => handleSaveMacAiProvider(provider)}
|
||||
disabled={remoteOllamaSaving}
|
||||
>
|
||||
{label}
|
||||
</StyledButton>
|
||||
))}
|
||||
</div>
|
||||
<div className="flex items-end gap-3">
|
||||
<div className="flex-1">
|
||||
<Input
|
||||
|
|
@ -437,6 +512,87 @@ export default function ModelsPage(props: {
|
|||
</StyledButton>
|
||||
)}
|
||||
</div>
|
||||
<div className="mt-6 border-t border-border-subtle pt-5">
|
||||
<h3 className="text-lg font-semibold text-text-primary">Native Mac Inference</h3>
|
||||
<p className="text-sm text-text-secondary mt-1 mb-4">
|
||||
On Apple Silicon, NOMAD can use a launchd-managed host worker for MLX and Core ML
|
||||
models. The worker runs on macOS so it can access Metal/Core ML acceleration while
|
||||
Command Center keeps using the same local OpenAI-compatible API shape.
|
||||
</p>
|
||||
<div className="grid grid-cols-1 lg:grid-cols-2 gap-4">
|
||||
<Input
|
||||
name="macNativeWorkerUrl"
|
||||
label="Native Worker URL"
|
||||
placeholder="http://host.docker.internal:8765"
|
||||
value={macNativeWorkerUrl}
|
||||
onChange={(e) => setMacNativeWorkerUrl(e.target.value)}
|
||||
/>
|
||||
<Input
|
||||
name="macNativeModelRoot"
|
||||
label="Model Folder"
|
||||
placeholder="~/Library/Application Support/Project N.O.M.A.D/mac-ai/models"
|
||||
value={macNativeModelRoot}
|
||||
onChange={(e) => setMacNativeModelRoot(e.target.value)}
|
||||
/>
|
||||
</div>
|
||||
<div className="mt-4 flex flex-wrap items-center gap-3">
|
||||
<StyledButton
|
||||
variant="primary"
|
||||
onClick={() => handleSaveMacAiProvider()}
|
||||
loading={remoteOllamaSaving}
|
||||
disabled={remoteOllamaSaving}
|
||||
>
|
||||
Save Native Settings
|
||||
</StyledButton>
|
||||
<StyledButton
|
||||
variant="ghost"
|
||||
icon="IconRefresh"
|
||||
onClick={() => refetchMacAiStatus()}
|
||||
>
|
||||
Check Worker
|
||||
</StyledButton>
|
||||
<span
|
||||
className={`text-sm font-semibold ${macAiStatus?.connected ? 'text-desert-green' : 'text-desert-red'}`}
|
||||
>
|
||||
{macAiStatus?.connected ? 'Native worker online' : 'Native worker offline'}
|
||||
</span>
|
||||
</div>
|
||||
{macAiStatus?.message && (
|
||||
<p className="mt-2 text-sm text-text-muted">{macAiStatus.message}</p>
|
||||
)}
|
||||
{macAiStatus?.models && macAiStatus.models.length > 0 && (
|
||||
<div className="mt-4 overflow-hidden rounded border border-border-subtle">
|
||||
<table className="min-w-full divide-y divide-border-subtle">
|
||||
<thead>
|
||||
<tr>
|
||||
<th className="px-4 py-2 text-left text-xs uppercase text-text-muted">
|
||||
Model
|
||||
</th>
|
||||
<th className="px-4 py-2 text-left text-xs uppercase text-text-muted">
|
||||
Backend
|
||||
</th>
|
||||
<th className="px-4 py-2 text-left text-xs uppercase text-text-muted">
|
||||
Chat
|
||||
</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody className="divide-y divide-border-subtle">
|
||||
{macAiStatus.models.map((model) => (
|
||||
<tr key={model.id}>
|
||||
<td className="px-4 py-2 text-sm text-text-primary">{model.name}</td>
|
||||
<td className="px-4 py-2 text-sm text-text-secondary">
|
||||
{model.backend.toUpperCase()}
|
||||
</td>
|
||||
<td className="px-4 py-2 text-sm text-text-secondary">
|
||||
{model.usableForChat ? 'Usable' : model.notes || 'Not a chat model'}
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<ActiveModelDownloads withHeader />
|
||||
|
|
@ -467,7 +623,7 @@ export default function ModelsPage(props: {
|
|||
onClick={handleForceRefresh}
|
||||
icon="IconRefresh"
|
||||
loading={isForceRefreshing}
|
||||
className='mt-1'
|
||||
className="mt-1"
|
||||
>
|
||||
Refresh Models
|
||||
</StyledButton>
|
||||
|
|
@ -536,7 +692,9 @@ export default function ModelsPage(props: {
|
|||
</span>
|
||||
</td>
|
||||
<td className="px-6 py-4 whitespace-nowrap">
|
||||
<span className="text-sm text-text-secondary">{tag.input || 'N/A'}</span>
|
||||
<span className="text-sm text-text-secondary">
|
||||
{tag.input || 'N/A'}
|
||||
</span>
|
||||
</td>
|
||||
<td className="px-6 py-4 whitespace-nowrap">
|
||||
<span className="text-sm text-text-secondary">
|
||||
|
|
@ -544,7 +702,9 @@ export default function ModelsPage(props: {
|
|||
</span>
|
||||
</td>
|
||||
<td className="px-6 py-4 whitespace-nowrap">
|
||||
<span className="text-sm text-text-secondary">{tag.size || 'N/A'}</span>
|
||||
<span className="text-sm text-text-secondary">
|
||||
{tag.size || 'N/A'}
|
||||
</span>
|
||||
</td>
|
||||
<td className="px-6 py-4 whitespace-nowrap">
|
||||
<StyledButton
|
||||
|
|
|
|||
|
|
@ -12,6 +12,8 @@ import DocsController from '#controllers/docs_controller'
|
|||
import DownloadsController from '#controllers/downloads_controller'
|
||||
import EasySetupController from '#controllers/easy_setup_controller'
|
||||
import HomeController from '#controllers/home_controller'
|
||||
import LocalLibraryController from '#controllers/local_library_controller'
|
||||
import MacAiController from '#controllers/mac_ai_controller'
|
||||
import MapsController from '#controllers/maps_controller'
|
||||
import OllamaController from '#controllers/ollama_controller'
|
||||
import RagController from '#controllers/rag_controller'
|
||||
|
|
@ -29,6 +31,7 @@ router.get('/home', [HomeController, 'home'])
|
|||
router.on('/about').renderInertia('about')
|
||||
router.get('/chat', [ChatsController, 'inertia'])
|
||||
router.get('/maps', [MapsController, 'index'])
|
||||
router.get('/local-library', [LocalLibraryController, 'page'])
|
||||
router.on('/knowledge-base').redirectToPath('/chat?knowledge_base=true') // redirect for legacy knowledge-base links
|
||||
|
||||
router.get('/easy-setup', [EasySetupController, 'index'])
|
||||
|
|
@ -111,6 +114,19 @@ router.get('/api/health', () => {
|
|||
return { status: 'ok' }
|
||||
})
|
||||
|
||||
router
|
||||
.group(() => {
|
||||
router.get('/files', [LocalLibraryController, 'list'])
|
||||
router.post('/files', [LocalLibraryController, 'upload'])
|
||||
router.get('/files/:filename/view', [LocalLibraryController, 'view'])
|
||||
router.get('/files/:filename/download', [LocalLibraryController, 'download'])
|
||||
router.get('/files/:filename/preview', [LocalLibraryController, 'preview'])
|
||||
router.patch('/files/:filename', [LocalLibraryController, 'rename'])
|
||||
router.delete('/files/:filename', [LocalLibraryController, 'delete'])
|
||||
router.post('/files/:filename/index', [LocalLibraryController, 'index'])
|
||||
})
|
||||
.prefix('/api/local-library')
|
||||
|
||||
router
|
||||
.group(() => {
|
||||
router.post('/chat', [OllamaController, 'chat'])
|
||||
|
|
@ -124,6 +140,13 @@ router
|
|||
})
|
||||
.prefix('/api/ollama')
|
||||
|
||||
router
|
||||
.group(() => {
|
||||
router.get('/status', [MacAiController, 'status'])
|
||||
router.post('/configure', [MacAiController, 'configure'])
|
||||
})
|
||||
.prefix('/api/mac-ai')
|
||||
|
||||
router
|
||||
.group(() => {
|
||||
router.get('/', [ChatsController, 'index'])
|
||||
|
|
|
|||
|
|
@ -0,0 +1,16 @@
|
|||
import * as assert from 'node:assert/strict'
|
||||
import { test } from 'node:test'
|
||||
|
||||
import { determineFileType } from '../../app/utils/fs.js'
|
||||
|
||||
test('local library file types include PDF, EPUB, and MOBI-style eBooks', () => {
|
||||
assert.equal(determineFileType('first-aid.pdf'), 'pdf')
|
||||
assert.equal(determineFileType('foraging.epub'), 'epub')
|
||||
assert.equal(determineFileType('manual.mobi'), 'mobi')
|
||||
assert.equal(determineFileType('manual.azw3'), 'mobi')
|
||||
})
|
||||
|
||||
test('local library unsupported binary formats remain unknown for RAG dispatch', () => {
|
||||
assert.equal(determineFileType('archive.exe'), 'unknown')
|
||||
assert.equal(determineFileType('disk.iso'), 'unknown')
|
||||
})
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
import * as assert from 'node:assert/strict'
|
||||
import { test } from 'node:test'
|
||||
|
||||
import { LocalLibraryService } from '../../app/services/local_library_service.js'
|
||||
|
||||
test('local library path resolution rejects traversal and nested paths', () => {
|
||||
const service = new LocalLibraryService()
|
||||
|
||||
assert.throws(() => service.resolveLibraryPath('../secret.pdf'), /invalid_filename/)
|
||||
assert.throws(() => service.resolveLibraryPath('manuals/field.pdf'), /invalid_filename/)
|
||||
assert.throws(() => service.resolveLibraryPath('manuals%2Ffield.pdf'), /invalid_filename/)
|
||||
assert.throws(() => service.resolveLibraryPath('..%2Fsecret.pdf'), /invalid_filename/)
|
||||
})
|
||||
|
||||
test('local library path resolution accepts a plain supported filename', () => {
|
||||
const service = new LocalLibraryService()
|
||||
const resolved = service.resolveLibraryPath('field-guide.pdf')
|
||||
|
||||
assert.match(resolved, /storage\/local_library\/field-guide\.pdf$/)
|
||||
})
|
||||
|
|
@ -0,0 +1,8 @@
|
|||
import * as assert from 'node:assert/strict'
|
||||
import { test } from 'node:test'
|
||||
|
||||
import { DEFAULT_MAC_NATIVE_WORKER_URL } from '../../app/services/mac_ai_service.js'
|
||||
|
||||
test('native mac ai worker default points at the macOS host from Docker', () => {
|
||||
assert.equal(DEFAULT_MAC_NATIVE_WORKER_URL, 'http://host.docker.internal:8765')
|
||||
})
|
||||
|
|
@ -1,22 +1,24 @@
|
|||
|
||||
export const KV_STORE_SCHEMA = {
|
||||
'chat.suggestionsEnabled': 'boolean',
|
||||
'chat.lastModel': 'string',
|
||||
'rag.docsEmbedded': 'boolean',
|
||||
'rag.defaultIngestPolicy': 'string',
|
||||
'system.updateAvailable': 'boolean',
|
||||
'system.latestVersion': 'string',
|
||||
'system.earlyAccess': 'boolean',
|
||||
'ui.hasVisitedEasySetup': 'boolean',
|
||||
'ui.theme': 'string',
|
||||
'ai.assistantCustomName': 'string',
|
||||
'gpu.type': 'string',
|
||||
'ai.remoteOllamaUrl': 'string',
|
||||
'ai.ollamaFlashAttention': 'boolean',
|
||||
'ai.amdGpuAcceleration': 'boolean',
|
||||
'ai.amdHsaOverride': 'string',
|
||||
'ai.autoFixGpuPassthrough': 'boolean',
|
||||
'gpu.autoRemediatedAt': 'string',
|
||||
'chat.suggestionsEnabled': 'boolean',
|
||||
'chat.lastModel': 'string',
|
||||
'rag.docsEmbedded': 'boolean',
|
||||
'rag.defaultIngestPolicy': 'string',
|
||||
'system.updateAvailable': 'boolean',
|
||||
'system.latestVersion': 'string',
|
||||
'system.earlyAccess': 'boolean',
|
||||
'ui.hasVisitedEasySetup': 'boolean',
|
||||
'ui.theme': 'string',
|
||||
'ai.assistantCustomName': 'string',
|
||||
'gpu.type': 'string',
|
||||
'ai.remoteOllamaUrl': 'string',
|
||||
'ai.provider': 'string',
|
||||
'ai.macNativeWorkerUrl': 'string',
|
||||
'ai.macNativeModelRoot': 'string',
|
||||
'ai.ollamaFlashAttention': 'boolean',
|
||||
'ai.amdGpuAcceleration': 'boolean',
|
||||
'ai.amdHsaOverride': 'string',
|
||||
'ai.autoFixGpuPassthrough': 'boolean',
|
||||
'gpu.autoRemediatedAt': 'string',
|
||||
} as const
|
||||
|
||||
type KVTagToType<T extends string> = T extends 'boolean' ? boolean : string
|
||||
|
|
|
|||
|
|
@ -0,0 +1,25 @@
|
|||
export type LocalLibraryFileType = 'pdf' | 'epub' | 'mobi' | 'text' | 'unknown'
|
||||
|
||||
export type LocalLibraryFile = {
|
||||
name: string
|
||||
displayName: string
|
||||
type: LocalLibraryFileType
|
||||
size: number
|
||||
modifiedTime: string
|
||||
viewUrl: string | null
|
||||
downloadUrl: string
|
||||
canPreview: boolean
|
||||
canIndex: boolean
|
||||
indexedSource: string
|
||||
}
|
||||
|
||||
export type LocalLibraryListResponse = {
|
||||
files: LocalLibraryFile[]
|
||||
}
|
||||
|
||||
export type LocalLibraryPreviewResponse = {
|
||||
name: string
|
||||
type: LocalLibraryFileType
|
||||
title: string
|
||||
text: string
|
||||
}
|
||||
|
|
@ -0,0 +1,20 @@
|
|||
export type MacAiProvider = 'ollama' | 'remote' | 'native_mlx' | 'native_coreml'
|
||||
|
||||
export type MacNativeModel = {
|
||||
id: string
|
||||
name: string
|
||||
backend: 'mlx' | 'coreml'
|
||||
path?: string
|
||||
usableForChat: boolean
|
||||
notes?: string
|
||||
}
|
||||
|
||||
export type MacNativeStatus = {
|
||||
configured: boolean
|
||||
connected: boolean
|
||||
url: string
|
||||
provider: MacAiProvider
|
||||
active: boolean
|
||||
models: MacNativeModel[]
|
||||
message?: string
|
||||
}
|
||||
|
|
@ -31,6 +31,7 @@ GREEN='\033[1;32m' # Light Green.
|
|||
WHIPTAIL_TITLE="Project N.O.M.A.D Installation"
|
||||
NOMAD_DIR="/opt/project-nomad"
|
||||
MANAGEMENT_COMPOSE_FILE_URL="https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/management_compose.yaml"
|
||||
MACOS_INSTALL_SCRIPT_URL="https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/macos/install_macos_nomad.sh"
|
||||
START_SCRIPT_URL="https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/start_nomad.sh"
|
||||
STOP_SCRIPT_URL="https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/stop_nomad.sh"
|
||||
UPDATE_SCRIPT_URL="https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/update_nomad.sh"
|
||||
|
|
@ -610,6 +611,16 @@ success_message() {
|
|||
###################################################################################################################################################################################################
|
||||
|
||||
# Pre-flight checks
|
||||
if [[ "$(uname -s)" == "Darwin" ]]; then
|
||||
echo -e "${GREEN}#${RESET} macOS detected. Switching to the macOS installer path.\\n"
|
||||
tmp_macos_installer="$(mktemp)"
|
||||
if ! curl -fsSL "$MACOS_INSTALL_SCRIPT_URL" -o "$tmp_macos_installer"; then
|
||||
echo -e "${RED}#${RESET} Failed to download the macOS installer. Please check your network connection."
|
||||
exit 1
|
||||
fi
|
||||
bash "$tmp_macos_installer"
|
||||
exit $?
|
||||
fi
|
||||
check_is_debian_based
|
||||
check_is_x86_64
|
||||
check_is_bash
|
||||
|
|
|
|||
|
|
@ -0,0 +1,74 @@
|
|||
#!/bin/bash
|
||||
set -euo pipefail
|
||||
|
||||
NOMAD_DIR="${NOMAD_DIR:-$HOME/.project-nomad}"
|
||||
NOMAD_PORT="${NOMAD_PORT:-8080}"
|
||||
COMPOSE_URL="${COMPOSE_URL:-https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/management_compose.yaml}"
|
||||
WORKER_URL="${WORKER_URL:-https://raw.githubusercontent.com/Crosstalk-Solutions/project-nomad/refs/heads/main/install/macos/nomad-mac-ai/worker.py}"
|
||||
|
||||
log() { printf '# %s\n' "$*"; }
|
||||
fail() { printf 'ERROR: %s\n' "$*" >&2; exit 1; }
|
||||
rand() { LC_ALL=C tr -dc 'A-Za-z0-9' < /dev/urandom | head -c "${1:-32}"; }
|
||||
|
||||
[[ "$(uname -s)" == "Darwin" ]] || fail "This installer is for macOS only."
|
||||
[[ "$(uname -m)" == "arm64" ]] || log "Apple Silicon was not detected. Native MLX/Core ML acceleration requires Apple Silicon."
|
||||
command -v docker >/dev/null 2>&1 || fail "Docker Desktop or Colima with Docker CLI is required."
|
||||
docker compose version >/dev/null 2>&1 || fail "Docker Compose v2 is required."
|
||||
command -v python3 >/dev/null 2>&1 || fail "python3 is required for the native Mac AI worker."
|
||||
|
||||
mkdir -p "$NOMAD_DIR/storage/logs" "$NOMAD_DIR/mac-ai/models" "$NOMAD_DIR/mac-ai/bin"
|
||||
log "Installing Project N.O.M.A.D. into $NOMAD_DIR"
|
||||
|
||||
curl -fsSL "$COMPOSE_URL" -o "$NOMAD_DIR/compose.yml"
|
||||
|
||||
app_key="$(rand)"
|
||||
db_root_password="$(rand)"
|
||||
db_user_password="$(rand)"
|
||||
|
||||
sed -i '' "s|/opt/project-nomad/storage|$NOMAD_DIR/storage|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|/opt/project-nomad/mysql|$NOMAD_DIR/mysql|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|/opt/project-nomad/redis|$NOMAD_DIR/redis|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|/opt/project-nomad:/opt/project-nomad|$NOMAD_DIR:/opt/project-nomad|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|URL=replaceme|URL=http://localhost:$NOMAD_PORT|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|APP_KEY=replaceme|APP_KEY=$app_key|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|DB_PASSWORD=replaceme|DB_PASSWORD=$db_user_password|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|MYSQL_ROOT_PASSWORD=replaceme|MYSQL_ROOT_PASSWORD=$db_root_password|g" "$NOMAD_DIR/compose.yml"
|
||||
sed -i '' "s|MYSQL_PASSWORD=replaceme|MYSQL_PASSWORD=$db_user_password|g" "$NOMAD_DIR/compose.yml"
|
||||
|
||||
curl -fsSL "$WORKER_URL" -o "$NOMAD_DIR/mac-ai/bin/worker.py"
|
||||
chmod +x "$NOMAD_DIR/mac-ai/bin/worker.py"
|
||||
|
||||
python3 -m venv "$NOMAD_DIR/mac-ai/venv"
|
||||
"$NOMAD_DIR/mac-ai/venv/bin/python" -m pip install --upgrade pip
|
||||
"$NOMAD_DIR/mac-ai/venv/bin/python" -m pip install mlx-lm coremltools
|
||||
|
||||
plist="$HOME/Library/LaunchAgents/us.projectnomad.mac-ai.plist"
|
||||
cat > "$plist" <<PLIST
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>Label</key><string>us.projectnomad.mac-ai</string>
|
||||
<key>ProgramArguments</key>
|
||||
<array>
|
||||
<string>$NOMAD_DIR/mac-ai/venv/bin/python</string>
|
||||
<string>$NOMAD_DIR/mac-ai/bin/worker.py</string>
|
||||
<string>--model-root</string>
|
||||
<string>$NOMAD_DIR/mac-ai/models</string>
|
||||
</array>
|
||||
<key>RunAtLoad</key><true/>
|
||||
<key>KeepAlive</key><true/>
|
||||
<key>StandardOutPath</key><string>$NOMAD_DIR/storage/logs/mac-ai.log</string>
|
||||
<key>StandardErrorPath</key><string>$NOMAD_DIR/storage/logs/mac-ai.err.log</string>
|
||||
</dict>
|
||||
</plist>
|
||||
PLIST
|
||||
|
||||
launchctl unload "$plist" >/dev/null 2>&1 || true
|
||||
launchctl load "$plist"
|
||||
|
||||
docker compose -p project-nomad -f "$NOMAD_DIR/compose.yml" up -d
|
||||
|
||||
log "Project N.O.M.A.D. is starting at http://localhost:$NOMAD_PORT"
|
||||
log "Native Mac AI worker is managed by launchd as us.projectnomad.mac-ai."
|
||||
log "Place MLX model folders or Core ML packages in $NOMAD_DIR/mac-ai/models."
|
||||
|
|
@ -0,0 +1,236 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Project N.O.M.A.D. native macOS inference worker.
|
||||
|
||||
This worker intentionally exposes a small OpenAI-compatible surface so the
|
||||
Command Center can use native MLX/Core ML models without running those runtimes
|
||||
inside Docker.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import traceback
|
||||
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
DEFAULT_MODEL_ROOT = Path.home() / ".project-nomad" / "mac-ai" / "models"
|
||||
|
||||
|
||||
def json_response(handler: BaseHTTPRequestHandler, status: int, payload: dict[str, Any]) -> None:
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
handler.send_response(status)
|
||||
handler.send_header("Content-Type", "application/json")
|
||||
handler.send_header("Content-Length", str(len(body)))
|
||||
handler.end_headers()
|
||||
handler.wfile.write(body)
|
||||
|
||||
|
||||
class ModelRegistry:
|
||||
def __init__(self, model_root: Path):
|
||||
self.model_root = model_root.expanduser()
|
||||
self.registry_path = self.model_root / "models.json"
|
||||
self.model_root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
def list_models(self) -> list[dict[str, Any]]:
|
||||
configured = self._configured_models()
|
||||
discovered = self._discovered_models()
|
||||
by_id: dict[str, dict[str, Any]] = {}
|
||||
for model in [*discovered, *configured]:
|
||||
by_id[model["id"]] = model
|
||||
return sorted(by_id.values(), key=lambda item: item["id"])
|
||||
|
||||
def find(self, model_id: str) -> dict[str, Any] | None:
|
||||
for model in self.list_models():
|
||||
if model["id"] == model_id:
|
||||
return model
|
||||
return None
|
||||
|
||||
def _configured_models(self) -> list[dict[str, Any]]:
|
||||
if not self.registry_path.exists():
|
||||
return []
|
||||
data = json.loads(self.registry_path.read_text("utf-8"))
|
||||
models = data.get("models", data if isinstance(data, list) else [])
|
||||
out: list[dict[str, Any]] = []
|
||||
for raw in models:
|
||||
if not isinstance(raw, dict) or not raw.get("id"):
|
||||
continue
|
||||
backend = raw.get("backend", "mlx")
|
||||
out.append(
|
||||
{
|
||||
"id": str(raw["id"]),
|
||||
"object": "model",
|
||||
"name": str(raw.get("name") or raw["id"]),
|
||||
"backend": "coreml" if backend == "coreml" else "mlx",
|
||||
"path": str(raw.get("path") or raw["id"]),
|
||||
"usable_for_chat": bool(raw.get("usable_for_chat", backend != "coreml")),
|
||||
"notes": raw.get("notes"),
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
def _discovered_models(self) -> list[dict[str, Any]]:
|
||||
out: list[dict[str, Any]] = []
|
||||
if not self.model_root.exists():
|
||||
return out
|
||||
for child in self.model_root.iterdir():
|
||||
if child.name == "models.json":
|
||||
continue
|
||||
if child.is_dir() and (child / "config.json").exists():
|
||||
out.append(
|
||||
{
|
||||
"id": child.name,
|
||||
"object": "model",
|
||||
"name": child.name,
|
||||
"backend": "mlx",
|
||||
"path": str(child),
|
||||
"usable_for_chat": True,
|
||||
}
|
||||
)
|
||||
elif child.suffix in {".mlmodel", ".mlpackage"}:
|
||||
out.append(
|
||||
{
|
||||
"id": child.name,
|
||||
"object": "model",
|
||||
"name": child.name,
|
||||
"backend": "coreml",
|
||||
"path": str(child),
|
||||
"usable_for_chat": False,
|
||||
"notes": "Core ML model is installed, but no chat adapter is configured.",
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
class MlxRuntime:
|
||||
def __init__(self):
|
||||
self._loaded: dict[str, Any] = {}
|
||||
|
||||
def generate(self, model_ref: str, messages: list[dict[str, str]], max_tokens: int, temperature: float) -> str:
|
||||
try:
|
||||
from mlx_lm import generate, load
|
||||
except Exception as exc: # pragma: no cover - host dependency
|
||||
raise RuntimeError("mlx-lm is not installed in the native worker venv.") from exc
|
||||
|
||||
if model_ref not in self._loaded:
|
||||
self._loaded[model_ref] = load(model_ref)
|
||||
model, tokenizer = self._loaded[model_ref]
|
||||
|
||||
prompt = self._prompt(tokenizer, messages)
|
||||
return generate(
|
||||
model,
|
||||
tokenizer,
|
||||
prompt=prompt,
|
||||
max_tokens=max_tokens,
|
||||
temp=temperature,
|
||||
verbose=False,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _prompt(tokenizer: Any, messages: list[dict[str, str]]) -> str:
|
||||
if hasattr(tokenizer, "apply_chat_template"):
|
||||
try:
|
||||
return tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
|
||||
except Exception:
|
||||
pass
|
||||
return "\n".join(f"{m.get('role', 'user')}: {m.get('content', '')}" for m in messages) + "\nassistant:"
|
||||
|
||||
|
||||
def make_handler(registry: ModelRegistry, mlx_runtime: MlxRuntime):
|
||||
class Handler(BaseHTTPRequestHandler):
|
||||
server_version = "NomadMacAI/1.0"
|
||||
|
||||
def do_GET(self) -> None: # noqa: N802
|
||||
if self.path == "/health":
|
||||
json_response(self, 200, {"status": "ok", "model_root": str(registry.model_root)})
|
||||
return
|
||||
if self.path == "/v1/models":
|
||||
json_response(self, 200, {"object": "list", "data": registry.list_models()})
|
||||
return
|
||||
json_response(self, 404, {"error": {"message": "Not found"}})
|
||||
|
||||
def do_POST(self) -> None: # noqa: N802
|
||||
if self.path != "/v1/chat/completions":
|
||||
json_response(self, 404, {"error": {"message": "Not found"}})
|
||||
return
|
||||
try:
|
||||
length = int(self.headers.get("Content-Length", "0"))
|
||||
payload = json.loads(self.rfile.read(length).decode("utf-8"))
|
||||
model_id = payload.get("model")
|
||||
model_info = registry.find(model_id)
|
||||
if not model_info:
|
||||
json_response(self, 404, {"error": {"message": f"Model not found: {model_id}"}})
|
||||
return
|
||||
if model_info["backend"] == "coreml" or not model_info.get("usable_for_chat", False):
|
||||
json_response(
|
||||
self,
|
||||
400,
|
||||
{"error": {"message": "This Core ML model is installed but is not configured as a chat model."}},
|
||||
)
|
||||
return
|
||||
text = mlx_runtime.generate(
|
||||
model_info.get("path") or model_id,
|
||||
payload.get("messages", []),
|
||||
int(payload.get("max_tokens") or payload.get("max_completion_tokens") or 512),
|
||||
float(payload.get("temperature", 0.7)),
|
||||
)
|
||||
if payload.get("stream"):
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "text/event-stream")
|
||||
self.send_header("Cache-Control", "no-cache")
|
||||
self.end_headers()
|
||||
chunk = {
|
||||
"id": "nomad-mac-ai",
|
||||
"object": "chat.completion.chunk",
|
||||
"model": model_id,
|
||||
"choices": [{"index": 0, "delta": {"content": text}, "finish_reason": None}],
|
||||
}
|
||||
self.wfile.write(f"data: {json.dumps(chunk)}\n\n".encode("utf-8"))
|
||||
done = {
|
||||
"id": "nomad-mac-ai",
|
||||
"object": "chat.completion.chunk",
|
||||
"model": model_id,
|
||||
"choices": [{"index": 0, "delta": {}, "finish_reason": "stop"}],
|
||||
}
|
||||
self.wfile.write(f"data: {json.dumps(done)}\n\ndata: [DONE]\n\n".encode("utf-8"))
|
||||
return
|
||||
json_response(
|
||||
self,
|
||||
200,
|
||||
{
|
||||
"id": "nomad-mac-ai",
|
||||
"object": "chat.completion",
|
||||
"model": model_id,
|
||||
"choices": [{"index": 0, "message": {"role": "assistant", "content": text}, "finish_reason": "stop"}],
|
||||
},
|
||||
)
|
||||
except Exception as exc:
|
||||
traceback.print_exc(file=sys.stderr)
|
||||
json_response(self, 500, {"error": {"message": str(exc)}})
|
||||
|
||||
def log_message(self, fmt: str, *args: Any) -> None:
|
||||
sys.stderr.write(f"[nomad-mac-ai] {self.address_string()} {fmt % args}\n")
|
||||
|
||||
return Handler
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--host", default=os.environ.get("NOMAD_MAC_AI_HOST", "127.0.0.1"))
|
||||
parser.add_argument("--port", type=int, default=int(os.environ.get("NOMAD_MAC_AI_PORT", "8765")))
|
||||
parser.add_argument("--model-root", default=os.environ.get("NOMAD_MAC_AI_MODEL_ROOT", str(DEFAULT_MODEL_ROOT)))
|
||||
args = parser.parse_args()
|
||||
|
||||
registry = ModelRegistry(Path(args.model_root))
|
||||
server = ThreadingHTTPServer((args.host, args.port), make_handler(registry, MlxRuntime()))
|
||||
print(f"nomad-mac-ai listening on http://{args.host}:{args.port}", flush=True)
|
||||
server.serve_forever()
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
Loading…
Reference in New Issue