RomanNum3ral
/
archinstall
mirror of https://github.com/archlinux/archinstall.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

PyParted and a large rewrite of the underlying partitioning (#1604) 

* Invert mypy files

* Add optional pre-commit hooks

* New profile structure

* Serialize profiles

* Use profile instead of classmethod

* Custom profile setup

* Separator between back

* Support profile import via url

* Move profiles module

* Refactor files

* Remove symlink

* Add user to docker group

* Update schema description

* Handle list services

* mypy fixes

* mypy fixes

* Rename profilesv2 to profiles

* flake8

* mypy again

* Support selecting DM

* Fix mypy

* Cleanup

* Update greeter setting

* Update schema

* Revert toml changes

* Poc external dependencies

* Dependency support

* New encryption menu

* flake8

* Mypy and flake8

* Unify lsblk command

* Update bootloader configuration

* Git hooks

* Fix import

* Pyparted

* Remove custom font setting

* flake8

* Remove default preview

* Manual partitioning menu

* Update structure

* Disk configuration

* Update filesystem

* luks2 encryption

* Everything works until installation

* Btrfsutil

* Btrfs handling

* Update btrfs

* Save encryption config

* Fix pipewire issue

* Update mypy version

* Update all pre-commit

* Update package versions

* Revert audio/pipewire

* Merge master PRs

* Add master changes

* Merge master changes

* Small renaming

* Pull master changes

* Reset disk enc after disk config change

* Generate locals

* Update naming

* Fix imports

* Fix broken sync

* Fix pre selection on table menu

* Profile menu

* Update profile

* Fix post_install

* Added python-pyparted to PKGBUILD, this requires [testing] to be enabled in order to run makepkg. Package still works via python -m build etc.

* Swaped around some setuptools logic in pyproject

Since we define `package-data` and `packages` there should be no need for:
```
[tool.setuptools.packages.find]
where = ["archinstall", "archinstall.*"]
```

* Removed pyproject collisions. Duplicate definitions.

* Made sure pyproject.toml includes languages

* Add example and update README

* Fix pyproject issues

* Generate locale

* Refactor imports

* Simplify imports

* Add profile description and package examples

* Align code

* Fix mypy

* Simplify imports

* Fix saving config

* Fix wrong luks merge

* Refactor installation

* Fix cdrom device loading

* Fix wrongly merged code

* Fix imports and greeter

* Don't terminate on partprobe error

* Use specific path on partprobe from luks

* Update archinstall/lib/disk/device_model.py

Co-authored-by: codefiles <11915375+codefiles@users.noreply.github.com>

* Update archinstall/lib/disk/device_model.py

Co-authored-by: codefiles <11915375+codefiles@users.noreply.github.com>

* Update github workflow to test archinstall installation

* Update sway merge

* Generate locales

* Update workflow

---------

Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
Co-authored-by: Anton Hvornum <anton@hvornum.se>
Co-authored-by: Anton Hvornum <anton.feeds+github@gmail.com>
Co-authored-by: codefiles <11915375+codefiles@users.noreply.github.com>
This commit is contained in:
Daniel Girtler 2023-04-19 20:55:42 +10:00 committed by GitHub
parent 5253e57e9f
commit 00b0ae7ba4
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
201 changed files with 13468 additions and 10351 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

12
.github/workflows/iso-build.yaml vendored
View File

@ -32,17 +32,7 @@ jobs:
- run: cat /etc/os-release
- run: pacman-key --init
- run: pacman --noconfirm -Sy archlinux-keyring
- run: mkdir -p /tmp/archlive/airootfs/root/archinstall-git; cp -r . /tmp/archlive/airootfs/root/archinstall-git
- run: echo "pip uninstall archinstall -y; cd archinstall-git; rm -rf dist; python -m build -n; pip install dist/archinstall*.whl" > /tmp/archlive/airootfs/root/.zprofile
- run: echo "echo \"This is an unofficial ISO for development and testing of archinstall. No support will be provided.\"" >> /tmp/archlive/airootfs/root/.zprofile
- run: echo "echo \"This ISO was built from Git SHA $GITHUB_SHA\"" >> /tmp/archlive/airootfs/root/.zprofile
- run: echo "echo \"Type archinstall to launch the installer.\"" >> /tmp/archlive/airootfs/root/.zprofile
- run: cat /tmp/archlive/airootfs/root/.zprofile
- run: pacman -Sy; pacman --noconfirm -S git archiso
- run: cp -r /usr/share/archiso/configs/releng/* /tmp/archlive
- run: echo -e "git\npython\npython-pip\npython-build\npython-flit\npython-setuptools\npython-wheel" >> /tmp/archlive/packages.x86_64
- run: find /tmp/archlive
- run: cd /tmp/archlive; mkarchiso -v -w work/ -o out/ ./
- run: ./build_iso.sh
- uses: actions/upload-artifact@v3
with:
name: Arch Live ISO

6
.github/workflows/mypy.yaml vendored
View File

@ -15,8 +15,4 @@ jobs:
# one day this will be enabled
# run: mypy --strict --module archinstall || exit 0
- name: run mypy
run: mypy --follow-imports=silent archinstall/lib/menu/abstract_menu.py archinstall/lib/menu/global_menu.py
archinstall/lib/models/network_configuration.py archinstall/lib/menu/list_manager.py archinstall/lib/user_interaction/network_conf.py archinstall/lib/models/users.py
archinstall/lib/disk/blockdevice.py archinstall/lib/user_interaction/subvolume_config.py archinstall/lib/disk/btrfs/btrfs_helpers.py
archinstall/lib/translationhandler.py archinstall/lib/disk/diskinfo.py archinstall/lib/menu/table_selection_menu.py archinstall/lib/hsm
archinstall/lib/disk/encryption.py archinstall/lib/models/disk_encryption.py
run: mypy --config-file mypy.ini

29
.github/workflows/python-build.yml vendored
View File

@ -7,20 +7,37 @@ on: [ push, pull_request ]
jobs:
deploy:
runs-on: ubuntu-latest
container:
image: archlinux:latest
options: --privileged
steps:
- uses: actions/checkout@v3
- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.x'
- name: Install dependencies
python-version: '3.10'
- name: Prepare arch
run: |
pacman-key --init
pacman --noconfirm -Sy archlinux-keyring
pacman --noconfirm -Sy python-pyparted pkgconfig gcc
- name: Install build dependencies
run: |
python -m pip install --upgrade pip
pip install build twine
- name: Build archinstall
pip install --upgrade build twine wheel setuptools installer
pip uninstall archinstall -y
- name: Install package dependencies
run: |
python -m build . --wheel
pip install --upgrade simple-term-menu pyparted
- name: Build archinstall
run: python -m build --wheel --no-isolation
- name: Install archinstall
run: python -m installer dist/*.whl
- name: Run archinstall
run: |
python -V
archinstall -v
- uses: actions/upload-artifact@v3
with:
name: archinstall
path: dist/*
path: dist/*

3
.gitignore vendored
View File

@ -33,4 +33,5 @@ venv
**/cmd_history.txt
**/*.*~
/*.sig
/*.json
/*.json
requirements.txt

42
.pre-commit-config.yaml Normal file
View File

@ -0,0 +1,42 @@
default_stages: ['commit']
repos:
- repo: https://github.com/pycqa/autoflake
rev: v2.0.2
hooks:
- id: autoflake
args: [
'--in-place',
'--remove-all-unused-imports',
'--ignore-init-module-imports'
]
files: \.py$
require_serial: true
fail_fast: true
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.4.0
hooks:
# general hooks:
- id: check-added-large-files # Prevent giant files from being committed
args: ['--maxkb=5000']
- id: check-merge-conflict # Check for files that contain merge conflict strings
- id: check-symlinks # Checks for symlinks which do not point to anything
- id: check-yaml # Attempts to load all yaml files to verify syntax
- id: destroyed-symlinks # Detects symlinks which are changed to regular files
- id: detect-private-key # Checks for the existence of private keys
- id: end-of-file-fixer # Makes sure files end in a newline and only a newline
- id: trailing-whitespace # Trims trailing whitespace
# Python specific hooks:
- id: check-ast # Simply check whether files parse as valid python
- id: check-docstring-first # Checks for a common error of placing code before the docstring
- repo: https://github.com/pycqa/flake8
rev: 6.0.0
hooks:
- id: flake8
args: [--config=.flake8]
fail_fast: true
- repo: https://github.com/pre-commit/mirrors-mypy
rev: v1.1.1
hooks:
- id: mypy
args: [--config=mypy.ini]
fail_fast: true

2
PKGBUILD
View File

@ -13,6 +13,7 @@ license=(GPL3)
depends=(
'python'
'systemd'
'python-pyparted'
)
makedepends=(
'python-setuptools'
@ -48,7 +49,6 @@ prepare() {
# use real directories for examples and profiles, as symlinks do not work
rm -fv $pkgname/{examples,profiles}
mv -v examples profiles $pkgname/
}
build() {

181
README.md
View File

@ -26,44 +26,60 @@ Assuming you are on an Arch Linux live-ISO:
# archinstall
#### Advanced
Some additional options that are not needed by most users are hidden behind the `--advanced` flag.
## Running from a declarative configuration file or URL
Prerequisites:
1. Edit the [configuration file](https://github.com/archlinux/archinstall/blob/master/examples/config-sample.json) according to your requirements.
`archinstall` can be run with a JSON configuration file. There are 2 different configuration files to consider,
the `user_configuration.json` contains all general installation configuration, whereas the `user_credentials.json`
contains the sensitive user configuration such as user password, root password and encryption password.
Assuming you are on a Arch Linux live-ISO and booted into EFI mode.
An example of the user configuration file can be found here
[configuration file](https://github.com/archlinux/archinstall/blob/master/examples/config-sample.json)
and example of the credentials configuration here
[credentials file](https://github.com/archlinux/archinstall/blob/master/examples/creds-sample.json).
# archinstall --config <path to user config file or URL> --disk-layout <path to disk layout config file or URL> --creds <path to user credentials config file or URL>
**HINT:** The configuration files can be auto-generated by starting `archisntall`, configuring all desired menu
points and then going to `Save configuration`.
To load the configuration file into `archinstall` run the following command
```
archinstall --config <path to user config file or URL> --creds <path to user credentials config file or URL>
```
# Available Languages
Archinstall is available in different languages which have been contributed and are maintained by the community.
Archinstall is available in different languages which have been contributed and are maintained by the community.
Current translations are listed below and vary in the amount of translations per language
```
English
Deutsch
Español
Français
Arabic
Brazilian Portuguese
Czech
Dutch
French
Georgian
German
Indonesian
Italiano
Nederlands
Polskie
Português do Brasil
Português
Svenska
Türkçe
čeština
Русский
اردو
Ελληνικά
தமிழ்
Italian
Korean
Modern Greek
Polish
Portuguese
Russian
Spanish
Swedish
Tamil
Turkish
Ukrainian
Urdu
```
Any contributions to the translations are more than welcome, and to get started please follow [the guide](https://github.com/archlinux/archinstall/blob/master/archinstall/locales/README.md)
Any contributions to the translations are more than welcome,
to get started please follow [the guide](https://github.com/archlinux/archinstall/blob/master/archinstall/locales/README.md)
# Help?
# Help or Issues
Submit an issue here on GitHub, or submit a post in the discord help channel.<br>
When doing so, attach the `/var/log/archinstall/install.log` to the issue ticket. This helps us help you!
@ -86,73 +102,111 @@ Therefore, Archinstall will try its best to not introduce any breaking changes e
# Scripting your own installation
You could just copy [guided.py](https://github.com/archlinux/archinstall/blob/master/examples/guided.py) as a starting point.
## Scripting interactive installation
However, assuming you're building your own ISO and want to create an automated installation process, or you want to install virtual machines onto local disk images, here is a [minimal example](https://github.com/archlinux/archinstall/blob/master/examples/minimal.py) of how to install using archinstall as a Python library:<br>
There are some examples in the `examples/` directory that should serve as a starting point.
The following is a small example of how to script your own *interative* installation:
```python
import archinstall, getpass
from pathlib import Path
# Select a harddrive and a disk password
harddrive = archinstall.select_disk(archinstall.all_blockdevices(partitions=False))
disk_password = getpass.getpass(prompt='Disk password (won\'t echo): ')
from archinstall import Installer, ProfileConfiguration, profile_handler, User
from archinstall.default_profiles.minimal import MinimalProfile
from archinstall.lib.disk.device_model import FilesystemType
from archinstall.lib.disk.encryption_menu import DiskEncryptionMenu
from archinstall.lib.disk.filesystem import FilesystemHandler
from archinstall.lib.user_interaction.disk_conf import select_disk_config
# We disable safety precautions in the library that protects the partitions
harddrive.keep_partitions = False
fs_type = FilesystemType('ext4')
# First, we configure the basic filesystem layout
with archinstall.Filesystem(harddrive, archinstall.GPT) as fs:
# We create a filesystem layout that will use the entire drive
# (this is a helper function, you can partition manually as well)
fs.use_entire_disk(root_filesystem_type='btrfs')
# Select a device to use for the installation
disk_config = select_disk_config()
boot = fs.find_partition('/boot')
root = fs.find_partition('/')
# Optional: ask for disk encryption configuration
data_store = {}
disk_encryption = DiskEncryptionMenu(disk_config.device_modifications, data_store).run()
boot.format('vfat')
# initiate file handler with the disk config and the optional disk encryption config
fs_handler = FilesystemHandler(disk_config, disk_encryption)
# Set the flag for encrypted to allow for encryption and then encrypt
root.encrypted = True
root.encrypt(password=disk_password)
# perform all file operations
# WARNING: this will potentially format the filesystem and delete all data
fs_handler.perform_filesystem_operations()
with archinstall.luks2(root, 'luksloop', disk_password) as unlocked_root:
unlocked_root.format(root.filesystem)
unlocked_root.mount('/mnt')
mountpoint = Path('/tmp')
boot.mount('/mnt/boot')
with Installer(
mountpoint,
disk_config,
disk_encryption=disk_encryption,
kernels=['linux']
) as installation:
installation.mount_ordered_layout()
installation.minimal_installation(hostname='minimal-arch')
installation.add_additional_packages(['nano', 'wget', 'git'])
with archinstall.Installer('/mnt') as installation:
if installation.minimal_installation(hostname='minimal-arch'):
installation.add_bootloader()
# Optionally, install a profile of choice.
# In this case, we install a minimal profile that is empty
profile_config = ProfileConfiguration(MinimalProfile())
profile_handler.install_profile_config(installation, profile_config)
installation.add_additional_packages(['nano', 'wget', 'git'])
# Optionally, install a profile of choice.
# In this case, we install a minimal profile that is empty
installation.install_profile('minimal')
user = User('devel', 'devel', False)
installation.create_users(user)
installation.user_set_pw('root', 'airoot')
user = User('archinstall', 'password', True)
installation.create_users(user)
```
This installer will perform the following:
* Prompt the user to select a disk and disk-password
* Proceed to wipe the selected disk with a `GPT` partition table on a UEFI system and MBR on a BIOS system.
* Sets up a default 100% used disk with encryption.
* Prompt the user to configurate the disk partitioning
* Prompt the user to setup disk encryption
* Create a file handler instance for the configured disk and the optional disk encryption
* Perform the disk operations (WARNING: this will potentially format the disks and erase all data)
* Installs a basic instance of Arch Linux *(base base-devel linux linux-firmware btrfs-progs efibootmgr)*
* Installs and configures a bootloader to partition 0 on uefi. On BIOS, it sets the root to partition 0.
* Install additional packages *(nano, wget, git)*
* Create a new user
> **Creating your own ISO with this script on it:** Follow [ArchISO](https://wiki.archlinux.org/index.php/archiso)'s guide on how to create your own ISO.
## Script non-interactive automated installation
For an example of a fully scripted, automated installation please see the example
[full_automated_installation.py](https://github.com/archlinux/archinstall/blob/master/examples/full_automated_installation.py)
## Unattended installation based on MAC address
Archinstall comes with an [unattended](https://github.com/archlinux/archinstall/blob/master/examples/unattended.py) example which will look for a matching profile for the machine it is being run on, based on any local MAC address.
For instance, if the machine that [unattended](https://github.com/archlinux/archinstall/blob/master/examples/unattended.py) is run on has the MAC address `52:54:00:12:34:56` it will look for a profile called [profiles/52-54-00-12-34-56.py](https://github.com/archlinux/archinstall/blob/master/profiles/52-54-00-12-34-56.py).
Archinstall comes with an [unattended](https://github.com/archlinux/archinstall/blob/master/examples/mac_address_installation.py)
example which will look for a matching profile for the machine it is being run on, based on any local MAC address.
For instance, if the machine the code is executed on has the MAC address `52:54:00:12:34:56` it will look for a profile called
[52-54-00-12-34-56.py](https://github.com/archlinux/archinstall/default_profiles/tailored.py).
If it's found, the unattended installation will commence and source that profile as its installation procedure.
# Profiles
`archinstall` ships with a set of pre-defined profiles that can be chosen during the installation process.
| *Desktop* | *Server* |
|---------------|------------|
| Awesome | Cockpit |
| Bspwm | Docker |
| Budgie | Lighttpd |
| Cinnamon | Mariadb |
| Cutefish | Nginx |
| Deepin | Postgresql |
| Enlightenment | Tomcat |
| Gnome | httpd |
| Kde | sshd |
| Lxqt | |
| Mate | |
| Qtile | |
| Sway | |
| Xfce4 | |
| i3-wm | |
The definitions of the profiles and what packages they will install can be seen directly in the menu or
[default profiles](https://github.com/archlinux/archinstall/default_profiles)
# Testing
## Using a Live ISO Image
@ -167,8 +221,7 @@ you can replace the version of archinstall with a new version and run that with
4. Now clone the latest repository with `git clone https://github.com/archlinux/archinstall`
5. Enter the repository with `cd archinstall`
*At this stage, you can choose to check out a feature branch for instance with `git checkout v2.3.1-rc1`*
6. Build the project and install it using `python setup.py install`
*If you get a 'No Module named setuptools' error, run `pacman -S python-setuptools`*
6. Build the project and install it using `pip install`
After this, running archinstall with `python -m archinstall` will run against whatever branch you chose in step 5.

143
archinstall/__init__.py
View File

@ -1,8 +1,12 @@
"""Arch Linux installer - guided, templates etc."""
import typing
import importlib
from argparse import ArgumentParser, Namespace
from .lib.disk import *
from .lib import disk
from .lib import menu
from .lib import models as models
from .lib import packages
from .lib.exceptions import *
from .lib.general import *
from .lib.hardware import *
@ -10,41 +14,20 @@ from .lib.installer import __packages__, Installer, accessibility_tools_in_use
from .lib.locale_helpers import *
from .lib.luks import *
from .lib.mirrors import *
from .lib.models.network_configuration import NetworkConfigurationHandler
from .lib.models.users import User
from .lib.networking import *
from .lib.output import *
from .lib.models.dataclasses import (
VersionDef,
PackageSearchResult,
PackageSearch,
LocalPackage
)
from .lib.packages.packages import (
group_search,
package_search,
find_package,
find_packages,
installed_package,
validate_package_list,
)
from .lib.profiles import *
from archinstall.lib.profile.profiles_handler import ProfileHandler, profile_handler
from .lib.profile.profile_menu import ProfileConfiguration
from .lib.services import *
from .lib.storage import *
from .lib.systemd import *
from .lib.user_interaction import *
from .lib.menu import Menu
from .lib.menu.list_manager import ListManager
from .lib.menu.text_input import TextInput
from .lib.menu.global_menu import GlobalMenu
from .lib.menu.abstract_menu import (
Selector,
AbstractMenu
)
from .lib.global_menu import GlobalMenu
from .lib.translationhandler import TranslationHandler, DeferredTranslation
from .lib.plugins import plugins, load_plugin # This initiates the plugin loading ceremony
from .lib.plugins import plugins, load_plugin # This initiates the plugin loading ceremony
from .lib.configuration import *
from .lib.udev import udevadm_info
parser = ArgumentParser()
__version__ = "2.5.4"
@ -66,8 +49,6 @@ def define_arguments():
parser.add_argument("-v", "--version", action="version", version="%(prog)s " + __version__)
parser.add_argument("--config", nargs="?", help="JSON configuration file or URL")
parser.add_argument("--creds", nargs="?", help="JSON credentials configuration file")
parser.add_argument("--disk_layouts","--disk_layout","--disk-layouts","--disk-layout",nargs="?",
help="JSON disk layout file")
parser.add_argument("--silent", action="store_true",
help="WARNING: Disables all prompts for input and confirmation. If no configuration is provided, this is ignored")
parser.add_argument("--dry-run", "--dry_run", action="store_true",
@ -79,6 +60,7 @@ def define_arguments():
parser.add_argument("--no-pkg-lookups", action="store_true", default=False, help="Disabled package validation specifically prior to starting installation.")
parser.add_argument("--plugin", nargs="?", type=str)
def parse_unspecified_argument_list(unknowns :list, multiple :bool = False, error :bool = False) -> dict:
"""We accept arguments not defined to the parser. (arguments "ad hoc").
Internally argparse return to us a list of words so we have to parse its contents, manually.
@ -129,7 +111,8 @@ def parse_unspecified_argument_list(unknowns :list, multiple :bool = False, erro
print(f" We ignore the entry {element} as it isn't related to any argument")
return config
def cleanup_empty_args(args :typing.Union[Namespace, dict]) -> dict:
def cleanup_empty_args(args: Union[Namespace, dict]) -> dict:
"""
Takes arguments (dictionary or argparse Namespace) and removes any
None values. This ensures clean mergers during dict.update(args)
@ -161,7 +144,7 @@ def get_arguments() -> Dict[str, Any]:
3) Amend
Change whatever is needed on the configuration dictionary (it could be done in post_process_arguments but this ougth to be left to changes anywhere else in the code, not in the arguments dictionary
"""
config = {}
config: Dict[str, Any] = {}
args, unknowns = parser.parse_known_args()
# preprocess the JSON files.
# TODO Expand the url access to the other JSON file arguments ?
@ -174,15 +157,15 @@ def get_arguments() -> Dict[str, Any]:
exit(1)
# load the parameters. first the known, then the unknowns
args = cleanup_empty_args(args)
config.update(args)
clean_args = cleanup_empty_args(args)
config.update(clean_args)
config.update(parse_unspecified_argument_list(unknowns))
# amend the parameters (check internal consistency)
# Installation can't be silent if config is not passed
if args.get('config') is None:
if clean_args.get('config') is None:
config["silent"] = False
else:
config["silent"] = args.get('silent')
config["silent"] = clean_args.get('silent')
# avoiding a compatibility issue
if 'dry-run' in config:
@ -190,29 +173,24 @@ def get_arguments() -> Dict[str, Any]:
return config
def load_config():
"""
refine and set some arguments. Formerly at the scripts
"""
from .lib.models import NetworkConfiguration
arguments.setdefault('sys-language', 'en_US')
arguments.setdefault('sys-encoding', 'utf-8')
if (archinstall_lang := arguments.get('archinstall-language', None)) is not None:
arguments['archinstall-language'] = TranslationHandler().get_language_by_name(archinstall_lang)
if arguments.get('harddrives', None) is not None:
if type(arguments['harddrives']) is str:
arguments['harddrives'] = arguments['harddrives'].split(',')
arguments['harddrives'] = [BlockDevice(BlockDev) for BlockDev in arguments['harddrives']]
# Temporarily disabling keep_partitions if config file is loaded
# Temporary workaround to make Desktop Environments work
if disk_config := arguments.get('disk_config', {}):
arguments['disk_config'] = disk.DiskLayoutConfiguration.parse_arg(disk_config)
if arguments.get('profile', None) is not None:
if type(arguments.get('profile', None)) is dict:
arguments['profile'] = Profile(None, arguments.get('profile', None)['path'])
else:
arguments['profile'] = Profile(None, arguments.get('profile', None))
storage['_desktop_profile'] = arguments.get('desktop-environment', None)
if profile_config := arguments.get('profile_config', None):
arguments['profile_config'] = ProfileConfiguration.parse_arg(profile_config)
if arguments.get('mirror-region', None) is not None:
if type(arguments.get('mirror-region', None)) is dict:
@ -221,12 +199,6 @@ def load_config():
selected_region = arguments.get('mirror-region', None)
arguments['mirror-region'] = {selected_region: list_mirrors()[selected_region]}
arguments.setdefault('sys-language', 'en_US')
arguments.setdefault('sys-encoding', 'utf-8')
if arguments.get('gfx_driver', None) is not None:
storage['gfx_driver_packages'] = AVAILABLE_GFX_DRIVERS.get(arguments.get('gfx_driver', None), None)
if arguments.get('servers', None) is not None:
storage['_selected_servers'] = arguments.get('servers', None)
@ -240,10 +212,13 @@ def load_config():
superusers = arguments.get('!superusers', None)
arguments['!users'] = User.parse_arguments(users, superusers)
if arguments.get('disk_encryption', None) is not None and arguments.get('disk_layouts', None) is not None:
if arguments.get('bootloader', None) is not None:
arguments['bootloader'] = Bootloader.from_arg(arguments['bootloader'])
if arguments.get('disk_encryption', None) is not None and disk_config is not None:
password = arguments.get('encryption_password', '')
arguments['disk_encryption'] = DiskEncryption.parse_arg(
arguments['disk_layouts'],
arguments['disk_encryption'] = disk.DiskEncryption.parse_arg(
arguments['disk_config'],
arguments['disk_encryption'],
password
)
@ -251,8 +226,8 @@ def load_config():
def post_process_arguments(arguments):
storage['arguments'] = arguments
if arguments.get('mount_point'):
storage['MOUNT_POINT'] = arguments['mount_point']
if mountpoint := arguments.get('mount_point', None):
storage['MOUNT_POINT'] = Path(mountpoint)
if arguments.get('debug', False):
log(f"Warning: --debug mode will write certain credentials to {storage['LOG_PATH']}/{storage['LOG_FILE']}!", fg="red", level=logging.WARNING)
@ -260,53 +235,31 @@ def post_process_arguments(arguments):
if arguments.get('plugin', None):
load_plugin(arguments['plugin'])
if arguments.get('disk_layouts', None) is not None:
layout_storage = {}
if not json_stream_to_structure('--disk_layouts',arguments['disk_layouts'],layout_storage):
exit(1)
else:
if arguments.get('harddrives') is None:
arguments['harddrives'] = [disk for disk in layout_storage]
# backward compatibility. Change partition.format for partition.wipe
for disk in layout_storage:
for i, partition in enumerate(layout_storage[disk].get('partitions',[])):
if 'format' in partition:
partition['wipe'] = partition['format']
del partition['format']
elif 'btrfs' in partition:
partition['btrfs']['subvolumes'] = Subvolume.parse_arguments(partition['btrfs']['subvolumes'])
arguments['disk_layouts'] = layout_storage
load_config()
define_arguments()
arguments = get_arguments()
arguments: Dict[str, Any] = get_arguments()
post_process_arguments(arguments)
# @archinstall.plugin decorator hook to programmatically add
# plugins in runtime. Useful in profiles and other things.
# plugins in runtime. Useful in profiles_bck and other things.
def plugin(f, *args, **kwargs):
plugins[f.__name__] = f
def run_as_a_module():
"""
Since we're running this as a 'python -m archinstall' module OR
a nuitka3 compiled version of the project.
This function and the file __main__ acts as a entry point.
This can either be run as the compiled and installed application: python setup.py install
OR straight as a module: python -m archinstall
In any case we will be attempting to load the provided script to be run from the scripts/ folder
"""
script = arguments.get('script', None)
# Add another path for finding profiles, so that list_profiles() in Script() can find guided.py, unattended.py etc.
storage['PROFILE_PATH'].append(os.path.abspath(f'{os.path.dirname(__file__)}/examples'))
try:
script = Script(arguments.get('script', None))
except ProfileNotFound as err:
print(f"Couldn't find file: {err}")
sys.exit(1)
if script is None:
print('No script to run provided')
os.chdir(os.path.abspath(os.path.dirname(__file__)))
# Remove the example directory from the PROFILE_PATH, to avoid guided.py etc shows up in user input questions.
storage['PROFILE_PATH'].pop()
script.execute()
mod_name = f'archinstall.scripts.{script}'
# by loading the module we'll automatically run the script
importlib.import_module(mod_name)

6
archinstall/__main__.py
View File

@ -5,9 +5,13 @@ import pathlib
# Load .git version before the builtin version
if pathlib.Path('./archinstall/__init__.py').absolute().exists():
spec = importlib.util.spec_from_file_location("archinstall", "./archinstall/__init__.py")
if spec is None or spec.loader is None:
raise ValueError('Could not retrieve spec from file: archinstall/__init__.py')
archinstall = importlib.util.module_from_spec(spec)
sys.modules["archinstall"] = archinstall
spec.loader.exec_module(sys.modules["archinstall"])
spec.loader.exec_module(archinstall)
else:
import archinstall

0
examples/__init__.py → archinstall/default_profiles/__init__.py
View File

0
profiles/__init__.py → archinstall/default_profiles/applications/__init__.py
View File

40
archinstall/default_profiles/applications/pipewire.py Normal file
View File

@ -0,0 +1,40 @@
from typing import List, Union, Any, TYPE_CHECKING
import archinstall
from archinstall.default_profiles.profile import Profile, ProfileType
from archinstall.lib.models import User
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class PipewireProfile(Profile):
def __init__(self):
super().__init__('Pipewire', ProfileType.Application)
@property
def packages(self) -> List[str]:
return [
'pipewire',
'pipewire-alsa',
'pipewire-jack',
'pipewire-pulse',
'gst-plugin-pipewire',
'libpulse',
'wireplumber'
]
def _enable_pipewire_for_all(self, install_session: 'Installer'):
users: Union[User, List[User]] = archinstall.arguments.get('!users', None)
if not isinstance(users, list):
users = [users]
for user in users:
install_session.arch_chroot('systemctl enable --user pipewire-pulse.service', run_as=user.username)
def install(self, install_session: 'Installer'):
super().install(install_session)
install_session.add_additional_packages(self.packages)
self._enable_pipewire_for_all(install_session)

218
archinstall/default_profiles/custom.py Normal file
View File

@ -0,0 +1,218 @@
from typing import List, Dict, Optional, TYPE_CHECKING, Any
from ..lib import menu
from archinstall.lib.output import log, FormattedOutput
from archinstall.lib.profile.profiles_handler import profile_handler
from archinstall.default_profiles.profile import Profile, ProfileType, SelectResult, ProfileInfo, TProfile
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class CustomProfileList(menu.ListManager):
def __init__(self, prompt: str, profiles: List[TProfile]):
self._actions = [
str(_('Add profile')),
str(_('Edit profile')),
str(_('Delete profile'))
]
super().__init__(prompt, profiles, [self._actions[0]], self._actions[1:])
def reformat(self, data: List[TProfile]) -> Dict[str, Optional[TProfile]]:
table = FormattedOutput.as_table(data)
rows = table.split('\n')
# these are the header rows of the table and do not map to any profile obviously
# we're adding 2 spaces as prefix because the menu selector '> ' will be put before
# the selectable rows so the header has to be aligned
display_data: Dict[str, Optional[TProfile]] = {f' {rows[0]}': None, f' {rows[1]}': None}
for row, profile in zip(rows[2:], data):
row = row.replace('|', '\\|')
display_data[row] = profile
return display_data
def selected_action_display(self, profile: TProfile) -> str:
return profile.name
def handle_action(
self,
action: str,
entry: Optional['CustomTypeProfile'],
data: List['CustomTypeProfile']
) -> List['CustomTypeProfile']:
if action == self._actions[0]: # add
new_profile = self._add_profile()
if new_profile is not None:
# in case a profile with the same name as an existing profile
# was created we'll replace the existing one
data = [d for d in data if d.name != new_profile.name]
data += [new_profile]
elif entry is not None:
if action == self._actions[1]: # edit
new_profile = self._add_profile(entry)
if new_profile is not None:
# we'll remove the original profile and add the modified version
data = [d for d in data if d.name != entry.name and d.name != new_profile.name]
data += [new_profile]
elif action == self._actions[2]: # delete
data = [d for d in data if d != entry]
return data
def _is_new_profile_name(self, name: str) -> bool:
existing_profile = profile_handler.get_profile_by_name(name)
if existing_profile is not None and existing_profile.profile_type != ProfileType.CustomType:
return False
return True
def _add_profile(self, editing: Optional['CustomTypeProfile'] = None) -> Optional['CustomTypeProfile']:
name_prompt = '\n\n' + str(_('Profile name: '))
while True:
profile_name = menu.TextInput(name_prompt, editing.name if editing else '').run().strip()
if not profile_name:
return None
if not self._is_new_profile_name(profile_name):
error_prompt = str(_("The profile name you entered is already in use. Try again"))
print(error_prompt)
else:
break
packages_prompt = str(_('Packages to be install with this profile (space separated, leave blank to skip): '))
edit_packages = ' '.join(editing.packages) if editing else ''
packages = menu.TextInput(packages_prompt, edit_packages).run().strip()
services_prompt = str(_('Services to be enabled with this profile (space separated, leave blank to skip): '))
edit_services = ' '.join(editing.services) if editing else ''
services = menu.TextInput(services_prompt, edit_services).run().strip()
choice = menu.Menu(
str(_('Should this profile be enabled for installation?')),
menu.Menu.yes_no(),
skip=False,
default_option=menu.Menu.no(),
clear_screen=False,
show_search_hint=False
).run()
enable_profile = True if choice.value == menu.Menu.yes() else False
profile = CustomTypeProfile(
profile_name,
enabled=enable_profile,
packages=packages.split(' '),
services=services.split(' ')
)
return profile
# TODO
# Still needs some ironing out
class CustomProfile():
def __init__(self):
super().__init__(
'Custom',
ProfileType.Custom,
description=str(_('Create your own'))
)
def json(self) -> Dict[str, Any]:
data: Dict[str, Any] = {'main': self.name, 'gfx_driver': self.gfx_driver, 'custom': []}
for profile in self._current_selection:
data['custom'].append({
'name': profile.name,
'packages': profile.packages,
'services': profile.services,
'enabled': profile.custom_enabled
})
return data
def do_on_select(self) -> SelectResult:
custom_profile_list = CustomProfileList('', profile_handler.get_custom_profiles())
custom_profiles = custom_profile_list.run()
# we'll first remove existing custom default_profiles with
# the same name and then add the new ones this
# will avoid errors of default_profiles with duplicate naming
profile_handler.remove_custom_profiles(custom_profiles)
profile_handler.add_custom_profiles(custom_profiles)
self.set_current_selection(custom_profiles)
if custom_profile_list.is_last_choice_cancel():
return SelectResult.SameSelection
enabled_profiles = [p for p in self._current_selection if p.custom_enabled]
# in case we only created inactive default_profiles we wanna store them but
# we want to reset the original setting
if not enabled_profiles:
return SelectResult.ResetCurrent
return SelectResult.NewSelection
def post_install(self, install_session: 'Installer'):
for profile in self._current_selection:
profile.post_install(install_session)
def install(self, install_session: 'Installer'):
driver_packages = self.gfx_driver_packages()
install_session.add_additional_packages(driver_packages)
for profile in self._current_selection:
if profile.custom_enabled:
log(f'Installing custom profile {profile.name}...')
install_session.add_additional_packages(profile.packages)
install_session.enable_service(profile.services)
profile.install(install_session)
def info(self) -> Optional[ProfileInfo]:
enabled_profiles = [p for p in self._current_selection if p.custom_enabled]
if enabled_profiles:
details = ', '.join([p.name for p in enabled_profiles])
gfx_driver = self.gfx_driver
return ProfileInfo(self.name, details, gfx_driver)
return None
def reset(self):
for profile in self._current_selection:
profile.set_enabled(False)
self.gfx_driver = None
class CustomTypeProfile(Profile):
def __init__(
self,
name: str,
enabled: bool = False,
packages: List[str] = [],
services: List[str] = []
):
super().__init__(
name,
ProfileType.CustomType,
packages=packages,
services=services,
support_gfx_driver=True
)
self.custom_enabled = enabled
def json(self) -> Dict[str, Any]:
return {
'name': self.name,
'packages': self.packages,
'services': self.services,
'enabled': self.custom_enabled
}

87
archinstall/default_profiles/desktop.py Normal file
View File

@ -0,0 +1,87 @@
from typing import Any, TYPE_CHECKING, List, Optional, Dict
from archinstall.lib import menu
from archinstall.lib.output import log
from archinstall.lib.profile.profiles_handler import profile_handler
from archinstall.default_profiles.profile import Profile, ProfileType, SelectResult, GreeterType
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class DesktopProfile(Profile):
def __init__(self, current_selection: List[Profile] = []):
super().__init__(
'Desktop',
ProfileType.Desktop,
description=str(_('Provides a selection of desktop environments and tiling window managers, e.g. gnome, kde, sway')),
current_selection=current_selection,
support_greeter=True
)
@property
def packages(self) -> List[str]:
return [
'nano',
'vim',
'openssh',
'htop',
'wget',
'iwd',
'wireless_tools',
'wpa_supplicant',
'smartmontools',
'xdg-utils'
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
combined_greeters: Dict[GreeterType, int] = {}
for profile in self.current_selection:
if profile.default_greeter_type:
combined_greeters.setdefault(profile.default_greeter_type, 0)
combined_greeters[profile.default_greeter_type] += 1
if len(combined_greeters) >= 1:
return list(combined_greeters)[0]
return None
def _do_on_select_profiles(self):
for profile in self.current_selection:
profile.do_on_select()
def do_on_select(self) -> SelectResult:
choice = profile_handler.select_profile(
profile_handler.get_desktop_profiles(),
self._current_selection,
title=str(_('Select your desired desktop environment')),
multi=True
)
match choice.type_:
case menu.MenuSelectionType.Selection:
self.set_current_selection(choice.value) # type: ignore
self._do_on_select_profiles()
return SelectResult.NewSelection
case menu.MenuSelectionType.Skip:
return SelectResult.SameSelection
case menu.MenuSelectionType.Reset:
return SelectResult.ResetCurrent
def post_install(self, install_session: 'Installer'):
for profile in self._current_selection:
profile.post_install(install_session)
def install(self, install_session: 'Installer'):
# Install common packages for all desktop environments
install_session.add_additional_packages(self.packages)
for profile in self._current_selection:
log(f'Installing profile {profile.name}...')
install_session.add_additional_packages(profile.packages)
install_session.enable_service(profile.services)
profile.install(install_session)

0
profiles/applications/__init__.py → archinstall/default_profiles/desktops/__init__.py
View File

36
archinstall/default_profiles/desktops/awesome.py Normal file
View File

@ -0,0 +1,36 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class AwesomeProfile(XorgProfile):
def __init__(self):
super().__init__('Awesome', ProfileType.WindowMgr, description='')
@property
def packages(self) -> List[str]:
return ['alacritty']
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()
def install(self, install_session: 'Installer'):
super().install(install_session)
# TODO: Copy a full configuration to ~/.config/awesome/rc.lua instead.
with open(f"{install_session.target}/etc/xdg/awesome/rc.lua", 'r') as fh:
awesome_lua = fh.read()
# Replace xterm with alacritty for a smoother experience.
awesome_lua = awesome_lua.replace('"xterm"', '"alacritty"')
with open(f"{install_session.target}/etc/xdg/awesome/rc.lua", 'w') as fh:
fh.write(awesome_lua)
# TODO: Configure the right-click-menu to contain the above packages that were installed. (as a user config)

30
archinstall/default_profiles/desktops/bspwm.py Normal file
View File

@ -0,0 +1,30 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class BspwmProfile(XorgProfile):
def __init__(self):
super().__init__('Bspwm', ProfileType.WindowMgr, description='')
@property
def packages(self) -> List[str]:
return [
'bspwm',
'sxhkd',
'dmenu',
'xdo',
'rxvt-unicode'
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

30
archinstall/default_profiles/desktops/budgie.py Normal file
View File

@ -0,0 +1,30 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class BudgieProfile(XorgProfile):
def __init__(self):
super().__init__('Budgie', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"arc-gtk-theme",
"budgie",
"mate-terminal",
"nemo",
"papirus-icon-theme",
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

31
archinstall/default_profiles/desktops/cinnamon.py Normal file
View File

@ -0,0 +1,31 @@
from typing import Optional, List, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class CinnamonProfile(XorgProfile):
def __init__(self):
super().__init__('Cinnamon', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"cinnamon",
"system-config-printer",
"gnome-keyring",
"gnome-terminal",
"blueberry",
"metacity"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

31
archinstall/default_profiles/desktops/cutefish.py Normal file
View File

@ -0,0 +1,31 @@
from typing import Optional, List, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class CutefishProfile(XorgProfile):
def __init__(self):
super().__init__('Cutefish', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"cutefish",
"noto-fonts"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Sddm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()
def install(self, install_session: 'Installer'):
super().install(install_session)

28
archinstall/default_profiles/desktops/deepin.py Normal file
View File

@ -0,0 +1,28 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class DeepinProfile(XorgProfile):
def __init__(self):
super().__init__('Deepin', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"deepin",
"deepin-terminal",
"deepin-editor"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

27
archinstall/default_profiles/desktops/enlightenment.py Normal file
View File

@ -0,0 +1,27 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class EnlighenmentProfile(XorgProfile):
def __init__(self):
super().__init__('Enlightenment', ProfileType.WindowMgr, description='')
@property
def packages(self) -> List[str]:
return [
"enlightenment",
"terminology"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

27
archinstall/default_profiles/desktops/gnome.py Normal file
View File

@ -0,0 +1,27 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class GnomeProfile(XorgProfile):
def __init__(self):
super().__init__('Gnome', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
'gnome',
'gnome-tweaks'
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Gdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

33
archinstall/default_profiles/desktops/i3.py Normal file
View File

@ -0,0 +1,33 @@
from typing import Optional, List, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class I3wmProfile(XorgProfile):
def __init__(self):
super().__init__('i3-wm', ProfileType.WindowMgr, description='')
@property
def packages(self) -> List[str]:
return [
'i3-wm',
'i3lock',
'i3status',
'i3blocks',
'xterm',
'lightdm-gtk-greeter',
'lightdm',
'dmenu',
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

32
archinstall/default_profiles/desktops/kde.py Normal file
View File

@ -0,0 +1,32 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class KdeProfile(XorgProfile):
def __init__(self):
super().__init__('Kde', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"plasma-meta",
"konsole",
"kwrite",
"dolphin",
"ark",
"plasma-wayland-session",
"egl-wayland"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Sddm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

35
archinstall/default_profiles/desktops/lxqt.py Normal file
View File

@ -0,0 +1,35 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class LxqtProfile(XorgProfile):
def __init__(self):
super().__init__('Lxqt', ProfileType.DesktopEnv, description='')
# NOTE: SDDM is the only officially supported greeter for LXQt, so unlike other DEs, lightdm is not used here.
# LXQt works with lightdm, but since this is not supported, we will not default to this.
# https://github.com/lxqt/lxqt/issues/795
@property
def packages(self) -> List[str]:
return [
"lxqt",
"breeze-icons",
"oxygen-icons",
"xdg-utils",
"ttf-freefont",
"leafpad",
"slock",
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Sddm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

27
archinstall/default_profiles/desktops/mate.py Normal file
View File

@ -0,0 +1,27 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class MateProfile(XorgProfile):
def __init__(self):
super().__init__('Mate', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"mate",
"mate-extra"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

27
archinstall/default_profiles/desktops/qtile.py Normal file
View File

@ -0,0 +1,27 @@
from typing import Optional, List, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class QtileProfile(XorgProfile):
def __init__(self):
super().__init__('Qtile', ProfileType.WindowMgr, description='')
@property
def packages(self) -> List[str]:
return [
'qtile',
'alacritty'
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

66
archinstall/default_profiles/desktops/sway.py Normal file
View File

@ -0,0 +1,66 @@
from typing import List, Optional, TYPE_CHECKING, Any
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
from archinstall.lib.menu import Menu
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class SwayProfile(XorgProfile):
def __init__(self):
super().__init__(
'Sway',
ProfileType.WindowMgr,
description=''
)
self._control_preference = []
@property
def packages(self) -> List[str]:
return [
"sway",
"swaybg",
"swaylock",
"swayidle",
"waybar",
"dmenu",
"brightnessctl",
"grim",
"slurp",
"pavucontrol",
"foot",
"xorg-xwayland"
] + self._control_preference
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
@property
def services(self) -> List[str]:
if "seatd" in self._control_preference:
return ['seatd']
elif "polkit" in self._control_preference:
return ['polkit']
return []
def _get_system_privelege_control_preference(self):
# need to activate seat service and add to seat group
title = str(_('Sway needs access to your seat (collection of hardware devices i.e. keyboard, mouse, etc)'))
title += str(_('\n\nChoose an option to give Sway access to your hardware'))
choice = Menu(title, ["polkit", "seatd"], skip=False).run()
self._control_preference = [choice.value]
def do_on_select(self):
self._get_system_privelege_control_preference()
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()
def install(self, install_session: 'Installer'):
super().install(install_session)

30
archinstall/default_profiles/desktops/xfce4.py Normal file
View File

@ -0,0 +1,30 @@
from typing import List, Optional, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType, GreeterType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
_: Any
class Xfce4Profile(XorgProfile):
def __init__(self):
super().__init__('Xfce4', ProfileType.DesktopEnv, description='')
@property
def packages(self) -> List[str]:
return [
"xfce4",
"xfce4-goodies",
"pavucontrol",
"gvfs",
"xarchiver"
]
@property
def default_greeter_type(self) -> Optional[GreeterType]:
return GreeterType.Lightdm
def preview_text(self) -> Optional[str]:
text = str(_('Environment type: {}')).format(self.profile_type.value)
return text + '\n' + self.packages_text()

15
archinstall/default_profiles/minimal.py Normal file
View File

@ -0,0 +1,15 @@
from typing import Any, TYPE_CHECKING
from archinstall.default_profiles.profile import Profile, ProfileType
if TYPE_CHECKING:
_: Any
class MinimalProfile(Profile):
def __init__(self):
super().__init__(
'Minimal',
ProfileType.Minimal,
description=str(_('A very basic installation that allows you to customize Arch Linux as you see fit.'))
)

206
archinstall/default_profiles/profile.py Normal file
View File

@ -0,0 +1,206 @@
from __future__ import annotations
from dataclasses import dataclass
from enum import Enum, auto
from typing import List, Optional, Any, Dict, TYPE_CHECKING, TypeVar
from archinstall.lib.output import FormattedOutput
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
TProfile = TypeVar('TProfile', bound='Profile')
class ProfileType(Enum):
# top level default_profiles
Server = 'Server'
Desktop = 'Desktop'
Xorg = 'Xorg'
Minimal = 'Minimal'
Custom = 'Custom'
# detailed selection default_profiles
ServerType = 'ServerType'
WindowMgr = 'Window Manager'
DesktopEnv = 'Desktop Environment'
CustomType = 'CustomType'
# special things
Tailored = 'Tailored'
Application = 'Application'
class GreeterType(Enum):
Lightdm = 'lightdm'
Sddm = 'sddm'
Gdm = 'gdm'
class SelectResult(Enum):
NewSelection = auto()
SameSelection = auto()
ResetCurrent = auto()
@dataclass
class ProfileInfo:
name: str
details: Optional[str]
gfx_driver: Optional[str] = None
greeter: Optional[str] = None
@property
def absolute_name(self) -> str:
if self.details is not None:
return self.details
return self.name
class Profile:
def __init__(
self,
name: str,
profile_type: ProfileType,
description: str = '',
current_selection: List[TProfile] = [],
packages: List[str] = [],
services: List[str] = [],
support_gfx_driver: bool = False,
support_greeter: bool = False
):
self.name = name
self.description = description
self.profile_type = profile_type
self._support_gfx_driver = support_gfx_driver
self._support_greeter = support_greeter
# self.gfx_driver: Optional[str] = None
self._current_selection = current_selection
self._packages = packages
self._services = services
# Only used for custom default_profiles
self.custom_enabled = False
@property
def current_selection(self) -> List[TProfile]:
return self._current_selection
@property
def packages(self) -> List[str]:
"""
Returns a list of packages that should be installed when
this profile is among the choosen ones
"""
return self._packages
@property
def services(self) -> List[str]:
"""
Returns a list of services that should be enabled when
this profile is among the chosen ones
"""
return self._services
@property
def default_greeter_type(self) -> Optional[GreeterType]:
"""
Setting a default greeter type for a desktop profile
"""
return None
def install(self, install_session: 'Installer'):
"""
Performs installation steps when this profile was selected
"""
def post_install(self, install_session: 'Installer'):
"""
Hook that will be called when the installation process is
finished and custom installation steps for specific default_profiles
are needed
"""
def json(self) -> Dict:
"""
Returns a json representation of the profile
"""
return {}
def do_on_select(self) -> SelectResult:
"""
Hook that will be called when a profile is selected
"""
return SelectResult.NewSelection
def current_selection_names(self) -> List[str]:
if self._current_selection:
return [s.name for s in self._current_selection]
return []
def reset(self):
self.set_current_selection([])
def set_current_selection(self, current_selection: List[TProfile]):
self._current_selection = current_selection
def is_top_level_profile(self) -> bool:
top_levels = [ProfileType.Desktop, ProfileType.Server, ProfileType.Xorg, ProfileType.Minimal, ProfileType.Custom]
return self.profile_type in top_levels
def is_desktop_profile(self) -> bool:
return self.profile_type == ProfileType.Desktop
def is_server_type_profile(self) -> bool:
return self.profile_type == ProfileType.ServerType
def is_desktop_type_profile(self) -> bool:
return self.profile_type == ProfileType.DesktopEnv or self.profile_type == ProfileType.WindowMgr
def is_xorg_type_profile(self) -> bool:
return self.profile_type == ProfileType.Xorg
def is_tailored(self) -> bool:
return self.profile_type == ProfileType.Tailored
def is_custom_type_profile(self) -> bool:
return self.profile_type == ProfileType.CustomType
def is_graphic_driver_supported(self) -> bool:
if not self._current_selection:
return self._support_gfx_driver
else:
if any([p._support_gfx_driver for p in self._current_selection]):
return True
return False
def is_greeter_supported(self) -> bool:
return self._support_greeter
def preview_text(self) -> Optional[str]:
"""
Used for preview text in profiles_bck. If a description is set for a
profile it will automatically display that one in the preivew.
If no preview or a different text should be displayed just
"""
if self.description:
return self.description
return None
def packages_text(self) -> str:
text = str(_('Installed packages')) + ':\n'
nr_packages = len(self.packages)
if nr_packages <= 5:
col = 1
elif nr_packages <= 10:
col = 2
elif nr_packages <= 15:
col = 3
else:
col = 4
text += FormattedOutput.as_columns(self.packages, col)
return text

57
archinstall/default_profiles/server.py Normal file
View File

@ -0,0 +1,57 @@
import logging
from typing import Any, TYPE_CHECKING, List
from archinstall.lib.output import log
from archinstall.lib.menu import MenuSelectionType
from archinstall.lib.profile.profiles_handler import profile_handler
from archinstall.default_profiles.profile import ProfileType, Profile, SelectResult, TProfile
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class ServerProfile(Profile):
def __init__(self, current_value: List[TProfile] = []):
super().__init__(
'Server',
ProfileType.Server,
description=str(_('Provides a selection of various server packages to install and enable, e.g. httpd, nginx, mariadb')),
current_selection=current_value
)
def do_on_select(self) -> SelectResult:
available_servers = profile_handler.get_server_profiles()
choice = profile_handler.select_profile(
available_servers,
self._current_selection,
title=str(_('Choose which servers to install, if none then a minimal installation will be done')),
multi=True
)
match choice.type_:
case MenuSelectionType.Selection:
self.set_current_selection(choice.value) # type: ignore
return SelectResult.NewSelection
case MenuSelectionType.Skip:
return SelectResult.SameSelection
case MenuSelectionType.Reset:
return SelectResult.ResetCurrent
def post_install(self, install_session: 'Installer'):
for profile in self._current_selection:
profile.post_install(install_session)
def install(self, install_session: 'Installer'):
server_info = self.current_selection_names()
details = ', '.join(server_info)
log(f'Now installing the selected servers: {details}', level=logging.INFO)
for server in self._current_selection:
log(f'Installing {server.name}...', level=logging.INFO)
install_session.add_additional_packages(server.packages)
install_session.enable_service(server.services)
server.install(install_session)
log('If your selections included multiple servers with the same port, you may have to reconfigure them.', fg="yellow", level=logging.INFO)

0
archinstall/default_profiles/servers/__init__.py Normal file
View File

19
archinstall/default_profiles/servers/cockpit.py Normal file
View File

@ -0,0 +1,19 @@
from typing import List
from archinstall.default_profiles.profile import Profile, ProfileType
class CockpitProfile(Profile):
def __init__(self):
super().__init__(
'Cockpit',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['cockpit', 'udisks2', 'packagekit']
@property
def services(self) -> List[str]:
return ['cockpit.socket']

33
archinstall/default_profiles/servers/docker.py Normal file
View File

@ -0,0 +1,33 @@
from typing import List, Union, TYPE_CHECKING
import archinstall
from archinstall.default_profiles.profile import Profile, ProfileType
from archinstall.lib.models import User
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
class DockerProfile(Profile):
def __init__(self):
super().__init__(
'Docker',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['docker']
@property
def services(self) -> List[str]:
return ['docker']
def post_install(self, install_session: 'Installer'):
users: Union[User, List[User]] = archinstall.arguments.get('!users', None)
if not isinstance(users, list):
users = [users]
for user in users:
install_session.arch_chroot(f'usermod -a -G docker {user.username}')

19
archinstall/default_profiles/servers/httpd.py Normal file
View File

@ -0,0 +1,19 @@
from typing import List
from archinstall.default_profiles.profile import Profile, ProfileType
class HttpdProfile(Profile):
def __init__(self):
super().__init__(
'httpd',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['apache']
@property
def services(self) -> List[str]:
return ['httpd']

19
archinstall/default_profiles/servers/lighttpd.py Normal file
View File

@ -0,0 +1,19 @@
from typing import List
from archinstall.default_profiles.profile import Profile, ProfileType
class LighttpdProfile(Profile):
def __init__(self):
super().__init__(
'Lighttpd',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['lighttpd']
@property
def services(self) -> List[str]:
return ['lighttpd']

25
archinstall/default_profiles/servers/mariadb.py Normal file
View File

@ -0,0 +1,25 @@
from typing import List, TYPE_CHECKING
from archinstall.default_profiles.profile import Profile, ProfileType
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
class MariadbProfile(Profile):
def __init__(self):
super().__init__(
'Mariadb',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['mariadb']
@property
def services(self) -> List[str]:
return ['mariadb']
def post_install(self, install_session: 'Installer'):
install_session.arch_chroot('mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql')

19
archinstall/default_profiles/servers/nginx.py Normal file
View File

@ -0,0 +1,19 @@
from typing import List
from archinstall.default_profiles.profile import Profile, ProfileType
class NginxProfile(Profile):
def __init__(self):
super().__init__(
'Nginx',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['nginx']
@property
def services(self) -> List[str]:
return ['nginx']

26
archinstall/default_profiles/servers/postgresql.py Normal file
View File

@ -0,0 +1,26 @@
from typing import List, TYPE_CHECKING
from archinstall.default_profiles.profile import Profile, ProfileType
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
class PostgresqlProfile(Profile):
def __init__(self):
super().__init__(
'Postgresql',
ProfileType.ServerType,
''
)
@property
def packages(self) -> List[str]:
return ['postgresql']
@property
def services(self) -> List[str]:
return ['postgresql']
def post_install(self, install_session: 'Installer'):
install_session.arch_chroot("initdb -D /var/lib/postgres/data", run_as='postgres')

19
archinstall/default_profiles/servers/sshd.py Normal file
View File

@ -0,0 +1,19 @@
from typing import List
from archinstall.default_profiles.profile import Profile, ProfileType
class SshdProfile(Profile):
def __init__(self):
super().__init__(
'sshd',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['openssh']
@property
def services(self) -> List[str]:
return ['sshd']

19
archinstall/default_profiles/servers/tomcat.py Normal file
View File

@ -0,0 +1,19 @@
from typing import List
from archinstall.default_profiles.profile import Profile, ProfileType
class TomcatProfile(Profile):
def __init__(self):
super().__init__(
'Tomcat',
ProfileType.ServerType
)
@property
def packages(self) -> List[str]:
return ['tomcat10']
@property
def services(self) -> List[str]:
return ['tomcat10']

21
archinstall/default_profiles/tailored.py Normal file
View File

@ -0,0 +1,21 @@
from typing import List, Any, TYPE_CHECKING
from archinstall.default_profiles.profile import ProfileType
from archinstall.default_profiles.xorg import XorgProfile
if TYPE_CHECKING:
from archinstall.lib.installer import Installer
_: Any
class TailoredProfile(XorgProfile):
def __init__(self):
super().__init__('52-54-00-12-34-56', ProfileType.Tailored, description='')
@property
def packages(self) -> List[str]:
return ['nano', 'wget', 'git']
def install(self, install_session: 'Installer'):
super().install(install_session)
# do whatever you like here :)

21
archinstall/default_profiles/xorg.py Normal file
View File

@ -0,0 +1,21 @@
from typing import Any, TYPE_CHECKING
from archinstall.default_profiles.profile import Profile, ProfileType
if TYPE_CHECKING:
_: Any
class XorgProfile(Profile):
def __init__(
self,
name: str = 'Xorg',
profile_type: ProfileType = ProfileType.Xorg,
description: str = str(_('Installs a minimal system as well as xorg and graphics drivers.')),
):
super().__init__(
name,
profile_type,
description=description,
support_gfx_driver=True
)

67
archinstall/lib/configuration.py
View File

@ -2,27 +2,15 @@ import os
import json
import stat
import logging
import pathlib
from typing import Optional, Dict
from pathlib import Path
from typing import Optional, Dict, Any, TYPE_CHECKING
from .hsm.fido import Fido2
from .models.disk_encryption import DiskEncryption
from .storage import storage
from .general import JSON, UNSAFE_JSON
from .output import log
from .exceptions import RequirementError
def configuration_sanity_check():
disk_encryption: DiskEncryption = storage['arguments'].get('disk_encryption')
if disk_encryption is not None and disk_encryption.hsm_device:
if not Fido2.get_fido2_devices():
raise RequirementError(
f"In order to use HSM to pair with the disk encryption,"
+ f" one needs to be accessible through /dev/hidraw* and support"
+ f" the FIDO2 protocol. You can check this by running"
+ f" 'systemd-cryptenroll --fido2-device=list'."
)
if TYPE_CHECKING:
_: Any
class ConfigurationOutput:
@ -35,13 +23,11 @@ class ConfigurationOutput:
:type config: Dict
"""
self._config = config
self._user_credentials = {}
self._disk_layout = None
self._user_config = {}
self._default_save_path = pathlib.Path(storage.get('LOG_PATH', '.'))
self._user_credentials: Dict[str, Any] = {}
self._user_config: Dict[str, Any] = {}
self._default_save_path = Path(storage.get('LOG_PATH', '.'))
self._user_config_file = 'user_configuration.json'
self._user_creds_file = "user_credentials.json"
self._disk_layout_file = "user_disk_layout.json"
self._sensitive = ['!users']
self._ignore = ['abort', 'install', 'config', 'creds', 'dry_run']
@ -56,23 +42,18 @@ class ConfigurationOutput:
def user_configuration_file(self):
return self._user_config_file
@property
def disk_layout_file(self):
return self._disk_layout_file
def _process_config(self):
for key in self._config:
if key in self._sensitive:
self._user_credentials[key] = self._config[key]
elif key == 'disk_layouts':
self._disk_layout = self._config[key]
elif key in self._ignore:
pass
else:
self._user_config[key] = self._config[key]
if key == 'disk_encryption' and self._config[key]: # special handling for encryption password
self._user_credentials['encryption_password'] = self._config[key].encryption_password
# special handling for encryption password
if key == 'disk_encryption' and self._config[key] is not None:
self._user_credentials['encryption_password'] = self._config[key].encryption_password
def user_config_to_json(self) -> str:
return json.dumps({
@ -81,11 +62,6 @@ class ConfigurationOutput:
'version': storage['__version__']
}, indent=4, sort_keys=True, cls=JSON)
def disk_layout_to_json(self) -> Optional[str]:
if self._disk_layout:
return json.dumps(self._disk_layout, indent=4, sort_keys=True, cls=JSON)
return None
def user_credentials_to_json(self) -> Optional[str]:
if self._user_credentials:
return json.dumps(self._user_credentials, indent=4, sort_keys=True, cls=UNSAFE_JSON)
@ -96,15 +72,11 @@ class ConfigurationOutput:
log(" -- Chosen configuration --", level=logging.DEBUG)
user_conig = self.user_config_to_json()
disk_layout = self.disk_layout_to_json()
log(user_conig, level=logging.INFO)
if disk_layout:
log(disk_layout, level=logging.INFO)
print()
def _is_valid_path(self, dest_path :pathlib.Path) -> bool:
def _is_valid_path(self, dest_path: Path) -> bool:
if (not dest_path.exists()) or not (dest_path.is_dir()):
log(
'Destination directory {} does not exist or is not a directory,\n Configuration files can not be saved'.format(dest_path.resolve()),
@ -113,7 +85,7 @@ class ConfigurationOutput:
return False
return True
def save_user_config(self, dest_path :pathlib.Path = None):
def save_user_config(self, dest_path: Path):
if self._is_valid_path(dest_path):
target = dest_path / self._user_config_file
@ -122,7 +94,7 @@ class ConfigurationOutput:
os.chmod(str(dest_path / self._user_config_file), stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP)
def save_user_creds(self, dest_path :pathlib.Path = None):
def save_user_creds(self, dest_path: Path):
if self._is_valid_path(dest_path):
if user_creds := self.user_credentials_to_json():
target = dest_path / self._user_creds_file
@ -132,21 +104,10 @@ class ConfigurationOutput:
os.chmod(str(target), stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP)
def save_disk_layout(self, dest_path :pathlib.Path = None):
if self._is_valid_path(dest_path):
if disk_layout := self.disk_layout_to_json():
target = dest_path / self._disk_layout_file
with target.open('w') as config_file:
config_file.write(disk_layout)
os.chmod(str(target), stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP)
def save(self, dest_path :pathlib.Path = None):
def save(self, dest_path: Optional[Path] = None):
if not dest_path:
dest_path = self._default_save_path
if self._is_valid_path(dest_path):
self.save_user_config(dest_path)
self.save_user_creds(dest_path)
self.save_disk_layout(dest_path)

47
archinstall/lib/disk/__init__.py
View File

@ -1,7 +1,40 @@
from .btrfs import *
from .helpers import *
from .blockdevice import BlockDevice
from .filesystem import Filesystem, MBR, GPT
from .partition import *
from .user_guides import *
from .validators import *
from .device_handler import device_handler, disk_layouts
from .fido import Fido2
from .filesystem import FilesystemHandler
from .subvolume_menu import SubvolumeMenu
from .partitioning_menu import (
manual_partitioning,
PartitioningList
)
from .device_model import (
_DeviceInfo,
BDevice,
DiskLayoutType,
DiskLayoutConfiguration,
PartitionTable,
Unit,
Size,
SubvolumeModification,
DeviceGeometry,
PartitionType,
PartitionFlag,
FilesystemType,
ModificationStatus,
PartitionModification,
DeviceModification,
EncryptionType,
DiskEncryption,
Fido2Device,
LsblkInfo,
CleanType,
get_lsblk_info,
get_all_lsblk_info,
get_lsblk_by_mountpoint
)
from .encryption_menu import (
select_encryption_type,
select_encrypted_password,
select_hsm,
select_partitions_to_encrypt,
DiskEncryptionMenu,
)

301
archinstall/lib/disk/blockdevice.py
View File

@ -1,301 +0,0 @@
from __future__ import annotations
import json
import logging
import time
from collections import OrderedDict
from dataclasses import dataclass
from typing import Optional, Dict, Any, Iterator, List, TYPE_CHECKING
from ..exceptions import DiskError, SysCallError
from ..output import log
from ..general import SysCommand
from ..storage import storage
if TYPE_CHECKING:
from .partition import Partition
_: Any
@dataclass
class BlockSizeInfo:
start: str
end: str
size: str
@dataclass
class BlockInfo:
pttype: str
ptuuid: str
size: int
tran: Optional[str]
rota: bool
free_space: Optional[List[BlockSizeInfo]]
class BlockDevice:
def __init__(self, path :str, info :Optional[Dict[str, Any]] = None):
if not info:
from .helpers import all_blockdevices
# If we don't give any information, we need to auto-fill it.
# Otherwise any subsequent usage will break.
self.info = all_blockdevices(partitions=False)[path].info
else:
self.info = info
self._path = path
self.keep_partitions = True
self._block_info = self._fetch_information()
self._partitions: Dict[str, 'Partition'] = {}
self._load_partitions()
# TODO: Currently disk encryption is a BIT misleading.
# It's actually partition-encryption, but for future-proofing this
# I'm placing the encryption password on a BlockDevice level.
def __repr__(self, *args :str, **kwargs :str) -> str:
return self._str_repr
@property
def path(self) -> str:
return self._path
@property
def _str_repr(self) -> str:
return f"BlockDevice({self._device_or_backfile}, size={self.size}GB, free_space={self._safe_free_space()}, bus_type={self.bus_type})"
def as_json(self) -> Dict[str, Any]:
return {
str(_('Device')): self._device_or_backfile,
str(_('Size')): f'{self.size}GB',
str(_('Free space')): f'{self._safe_free_space()}',
str(_('Bus-type')): f'{self.bus_type}'
}
def __iter__(self) -> Iterator['Partition']:
for partition in self.partitions:
yield self.partitions[partition]
def __getitem__(self, key :str, *args :str, **kwargs :str) -> Any:
if hasattr(self, key):
return getattr(self, key)
if self.info and key in self.info:
return self.info[key]
raise KeyError(f'{self.info} does not contain information: "{key}"')
def __lt__(self, left_comparitor :'BlockDevice') -> bool:
return self._path < left_comparitor.path
def json(self) -> str:
"""
json() has precedence over __dump__, so this is a way
to give less/partial information for user readability.
"""
return self._path
def __dump__(self) -> Dict[str, Dict[str, Any]]:
return {
self._path: {
'partuuid': self.uuid,
'wipe': self.info.get('wipe', None),
'partitions': [part.__dump__() for part in self.partitions.values()]
}
}
def _call_lsblk(self, path: str) -> Dict[str, Any]:
output = SysCommand(f'lsblk --json -b -o+SIZE,PTTYPE,ROTA,TRAN,PTUUID {self._path}').decode('UTF-8')
if output:
lsblk_info = json.loads(output)
return lsblk_info
raise DiskError(f'Failed to read disk "{self.path}" with lsblk')
def _load_partitions(self):
from .partition import Partition
self._partitions.clear()
lsblk_info = self._call_lsblk(self._path)
device = lsblk_info['blockdevices'][0]
self._partitions.clear()
if children := device.get('children', None):
root = f'/dev/{device["name"]}'
for child in children:
part_id = child['name'].removeprefix(device['name'])
self._partitions[part_id] = Partition(root + part_id, block_device=self, part_id=part_id)
def _get_free_space(self) -> Optional[List[BlockSizeInfo]]:
# NOTE: parted -s will default to `cancel` on prompt, skipping any partition
# that is "outside" the disk. in /dev/sr0 this is usually the case with Archiso,
# so the free will ignore the ESP partition and just give the "free" space.
# Doesn't harm us, but worth noting in case something weird happens.
try:
output = SysCommand(f"parted -s --machine {self._path} print free").decode('utf-8')
if output:
free_lines = [line for line in output.split('\n') if 'free' in line]
sizes = []
for free_space in free_lines:
_, start, end, size, *_ = free_space.strip('\r\n;').split(':')
sizes.append(BlockSizeInfo(start, end, size))
return sizes
except SysCallError as error:
log(f"Could not get free space on {self._path}: {error}", level=logging.DEBUG)
return None
def _fetch_information(self) -> BlockInfo:
lsblk_info = self._call_lsblk(self._path)
device = lsblk_info['blockdevices'][0]
free_space = self._get_free_space()
return BlockInfo(
pttype=device['pttype'],
ptuuid=device['ptuuid'],
size=device['size'],
tran=device['tran'],
rota=device['rota'],
free_space=free_space
)
@property
def _device_or_backfile(self) -> Optional[str]:
"""
Returns the actual device-endpoint of the BlockDevice.
If it's a loop-back-device it returns the back-file,
For other types it return self.device
"""
if self.info.get('type') == 'loop':
return self.info['back-file']
else:
return self.device
@property
def mountpoint(self) -> None:
"""
A dummy function to enable transparent comparisons of mountpoints.
As blockdevices can't be mounted directly, this will always be None
"""
return None
@property
def device(self) -> Optional[str]:
"""
Returns the device file of the BlockDevice.
If it's a loop-back-device it returns the /dev/X device,
If it's a ATA-drive it returns the /dev/X device
And if it's a crypto-device it returns the parent device
"""
if "DEVTYPE" not in self.info:
raise DiskError(f'Could not locate backplane info for "{self._path}"')
if self.info['DEVTYPE'] in ['disk','loop']:
return self._path
elif self.info['DEVTYPE'][:4] == 'raid':
# This should catch /dev/md## raid devices
return self._path
elif self.info['DEVTYPE'] == 'crypt':
if 'pkname' not in self.info:
raise DiskError(f'A crypt device ({self._path}) without a parent kernel device name.')
return f"/dev/{self.info['pkname']}"
else:
log(f"Unknown blockdevice type for {self._path}: {self.info['DEVTYPE']}", level=logging.DEBUG)
return None
@property
def partition_type(self) -> str:
return self._block_info.pttype
@property
def uuid(self) -> str:
return self._block_info.ptuuid
@property
def size(self) -> float:
from .helpers import convert_size_to_gb
return convert_size_to_gb(self._block_info.size)
@property
def bus_type(self) -> Optional[str]:
return self._block_info.tran
@property
def spinning(self) -> bool:
return self._block_info.rota
@property
def partitions(self) -> Dict[str, 'Partition']:
return OrderedDict(sorted(self._partitions.items()))
@property
def partition(self) -> List['Partition']:
return list(self.partitions.values())
@property
def first_free_sector(self) -> str:
if block_size := self._largest_free_space():
return block_size.start
else:
return '512MB'
@property
def first_end_sector(self) -> str:
if block_size := self._largest_free_space():
return block_size.end
else:
return f"{self.size}GB"
def _safe_free_space(self) -> str:
if self._block_info.free_space:
sizes = [free_space.size for free_space in self._block_info.free_space]
return '+'.join(sizes)
return '?'
def _largest_free_space(self) -> Optional[BlockSizeInfo]:
if self._block_info.free_space:
sorted_sizes = sorted(self._block_info.free_space, key=lambda x: x.size, reverse=True)
return sorted_sizes[0]
return None
def _partprobe(self) -> bool:
return SysCommand(['partprobe', self._path]).exit_code == 0
def flush_cache(self) -> None:
self._load_partitions()
def get_partition(self, uuid :Optional[str] = None, partuuid :Optional[str] = None) -> Partition:
if not uuid and not partuuid:
raise ValueError(f"BlockDevice.get_partition() requires either a UUID or a PARTUUID for lookups.")
log(f"Retrieving partition PARTUUID={partuuid} or UUID={uuid}", level=logging.DEBUG, fg="gray")
for count in range(storage.get('DISK_RETRY_ATTEMPTS', 5)):
for partition_index, partition in self.partitions.items():
try:
if uuid and partition.uuid and partition.uuid.lower() == uuid.lower():
log(f"Matched UUID={uuid} against {partition.uuid}", level=logging.DEBUG, fg="gray")
return partition
elif partuuid and partition.part_uuid and partition.part_uuid.lower() == partuuid.lower():
log(f"Matched PARTUUID={partuuid} against {partition.part_uuid}", level=logging.DEBUG, fg="gray")
return partition
except DiskError as error:
# Most likely a blockdevice that doesn't support or use UUID's
# (like Microsoft recovery partition)
log(f"Could not get UUID/PARTUUID of {partition}: {error}", level=logging.DEBUG, fg="gray")
pass
log(f"uuid {uuid} or {partuuid} not found. Waiting {storage.get('DISK_TIMEOUTS', 1) * count}s for next attempt",level=logging.DEBUG)
self.flush_cache()
time.sleep(storage.get('DISK_TIMEOUTS', 1) * count)
log(f"Could not find {uuid}/{partuuid} in disk after 5 retries", level=logging.INFO)
log(f"Cache: {self._partitions}")
log(f"Partitions: {self.partitions.items()}")
raise DiskError(f"Partition {uuid}/{partuuid} was never found on {self} despite several attempts.")

56
archinstall/lib/disk/btrfs/__init__.py
View File

@ -1,56 +0,0 @@
from __future__ import annotations
import pathlib
import glob
import logging
from typing import Union, Dict, TYPE_CHECKING
# https://stackoverflow.com/a/39757388/929999
if TYPE_CHECKING:
from ...installer import Installer
from .btrfs_helpers import (
subvolume_info_from_path as subvolume_info_from_path,
find_parent_subvolume as find_parent_subvolume,
setup_subvolumes as setup_subvolumes,
mount_subvolume as mount_subvolume
)
from .btrfssubvolumeinfo import BtrfsSubvolumeInfo as BtrfsSubvolume
from .btrfspartition import BTRFSPartition as BTRFSPartition
from ...exceptions import DiskError, Deprecated
from ...general import SysCommand
from ...output import log
def create_subvolume(installation: Installer, subvolume_location :Union[pathlib.Path, str]) -> bool:
"""
This function uses btrfs to create a subvolume.
@installation: archinstall.Installer instance
@subvolume_location: a localized string or path inside the installation / or /boot for instance without specifying /mnt/boot
"""
installation_mountpoint = installation.target
if type(installation_mountpoint) == str:
installation_mountpoint = pathlib.Path(installation_mountpoint)
# Set up the required physical structure
if type(subvolume_location) == str:
subvolume_location = pathlib.Path(subvolume_location)
target = installation_mountpoint / subvolume_location.relative_to(subvolume_location.anchor)
# Difference from mount_subvolume:
# We only check if the parent exists, since we'll run in to "target path already exists" otherwise
if not target.parent.exists():
target.parent.mkdir(parents=True)
if glob.glob(str(target / '*')):
raise DiskError(f"Cannot create subvolume at {target} because it contains data (non-empty folder target)")
# Remove the target if it exists
if target.exists():
target.rmdir()
log(f"Creating a subvolume on {target}", level=logging.INFO)
if (cmd := SysCommand(f"btrfs subvolume create {target}")).exit_code != 0:
raise DiskError(f"Could not create a subvolume at {target}: {cmd}")

136
archinstall/lib/disk/btrfs/btrfs_helpers.py
View File

@ -1,136 +0,0 @@
import logging
import re
from pathlib import Path
from typing import Optional, Dict, Any, TYPE_CHECKING
from ...models.subvolume import Subvolume
from ...exceptions import SysCallError, DiskError
from ...general import SysCommand
from ...output import log
from ...plugins import plugins
from ..helpers import get_mount_info
from .btrfssubvolumeinfo import BtrfsSubvolumeInfo
if TYPE_CHECKING:
from .btrfspartition import BTRFSPartition
from ...installer import Installer
class fstab_btrfs_compression_plugin():
def __init__(self, partition_dict):
self.partition_dict = partition_dict
def on_genfstab(self, installation):
with open(f"{installation.target}/etc/fstab", 'r') as fh:
fstab = fh.read()
# Replace the {installation}/etc/fstab with entries
# using the compress=zstd where the mountpoint has compression set.
with open(f"{installation.target}/etc/fstab", 'w') as fh:
for line in fstab.split('\n'):
# So first we grab the mount options by using subvol=.*? as a locator.
# And we also grab the mountpoint for the entry, for instance /var/log
if (subvoldef := re.findall(',.*?subvol=.*?[\t ]', line)) and (mountpoint := re.findall('[\t ]/.*?[\t ]', line)):
for subvolume in self.partition_dict.get('btrfs', {}).get('subvolumes', []):
# We then locate the correct subvolume and check if it's compressed
if subvolume.compress and subvolume.mountpoint == mountpoint[0].strip():
# We then sneak in the compress=zstd option if it doesn't already exist:
# We skip entries where compression is already defined
if ',compress=zstd,' not in line:
line = line.replace(subvoldef[0], f",compress=zstd{subvoldef[0]}")
break
fh.write(f"{line}\n")
return True
def mount_subvolume(installation: 'Installer', device: 'BTRFSPartition', subvolume: Subvolume):
# we normalize the subvolume name (getting rid of slash at the start if exists.
# In our implementation has no semantic load.
# Every subvolume is created from the top of the hierarchy- and simplifies its further use
name = subvolume.name.lstrip('/')
mountpoint = Path(subvolume.mountpoint)
installation_target = Path(installation.target)
mountpoint = installation_target / mountpoint.relative_to(mountpoint.anchor)
mountpoint.mkdir(parents=True, exist_ok=True)
mount_options = subvolume.options + [f'subvol={name}']
log(f"Mounting subvolume {name} on {device} to {mountpoint}", level=logging.INFO, fg="gray")
SysCommand(f"mount {device.path} {mountpoint} -o {','.join(mount_options)}")
def setup_subvolumes(installation: 'Installer', partition_dict: Dict[str, Any]):
log(f"Setting up subvolumes: {partition_dict['btrfs']['subvolumes']}", level=logging.INFO, fg="gray")
for subvolume in partition_dict['btrfs']['subvolumes']:
# we normalize the subvolume name (getting rid of slash at the start if exists. In our implementation has no semantic load.
# Every subvolume is created from the top of the hierarchy- and simplifies its further use
name = subvolume.name.lstrip('/')
# We create the subvolume using the BTRFSPartition instance.
# That way we ensure not only easy access, but also accurate mount locations etc.
partition_dict['device_instance'].create_subvolume(name, installation=installation)
# Make the nodatacow processing now
# It will be the main cause of creation of subvolumes which are not to be mounted
# it is not an options which can be established by subvolume (but for whole file systems), and can be
# set up via a simple attribute change in a directory (if empty). And here the directories are brand new
if subvolume.nodatacow:
if (cmd := SysCommand(f"chattr +C {installation.target}/{name}")).exit_code != 0:
raise DiskError(f"Could not set nodatacow attribute at {installation.target}/{name}: {cmd}")
# Make the compress processing now
# it is not an options which can be established by subvolume (but for whole file systems), and can be
# set up via a simple attribute change in a directory (if empty). And here the directories are brand new
# in this way only zstd compression is activaded
# TODO WARNING it is not clear if it should be a standard feature, so it might need to be deactivated
if subvolume.compress:
if not any(['compress' in filesystem_option for filesystem_option in partition_dict.get('filesystem', {}).get('mount_options', [])]):
if (cmd := SysCommand(f"chattr +c {installation.target}/{name}")).exit_code != 0:
raise DiskError(f"Could not set compress attribute at {installation.target}/{name}: {cmd}")
if 'fstab_btrfs_compression_plugin' not in plugins:
plugins['fstab_btrfs_compression_plugin'] = fstab_btrfs_compression_plugin(partition_dict)
def subvolume_info_from_path(path: Path) -> Optional[BtrfsSubvolumeInfo]:
try:
subvolume_name = ''
result = {}
for index, line in enumerate(SysCommand(f"btrfs subvolume show {path}")):
if index == 0:
subvolume_name = line.strip().decode('UTF-8')
continue
if b':' in line:
key, value = line.strip().decode('UTF-8').split(':', 1)
# A bit of a hack, until I figure out how @dataclass
# allows for hooking in a pre-processor to do this we have to do it here:
result[key.lower().replace(' ', '_').replace('(s)', 's')] = value.strip()
return BtrfsSubvolumeInfo(**{'full_path' : path, 'name' : subvolume_name, **result}) # type: ignore
except SysCallError as error:
log(f"Could not retrieve subvolume information from {path}: {error}", level=logging.WARNING, fg="orange")
return None
def find_parent_subvolume(path: Path, filters=[]) -> Optional[BtrfsSubvolumeInfo]:
# A root path cannot have a parent
if str(path) == '/':
return None
if found_mount := get_mount_info(str(path.parent), traverse=True, ignore=filters):
if not (subvolume := subvolume_info_from_path(found_mount['target'])):
if found_mount['target'] == '/':
return None
return find_parent_subvolume(path.parent, filters=[*filters, found_mount['target']])
return subvolume
return None

109
archinstall/lib/disk/btrfs/btrfspartition.py
View File

@ -1,109 +0,0 @@
import glob
import pathlib
import logging
from typing import Optional, TYPE_CHECKING
from ...exceptions import DiskError
from ...storage import storage
from ...output import log
from ...general import SysCommand
from ..partition import Partition
from ..helpers import findmnt
from .btrfs_helpers import (
subvolume_info_from_path
)
if TYPE_CHECKING:
from ...installer import Installer
from .btrfssubvolumeinfo import BtrfsSubvolumeInfo
class BTRFSPartition(Partition):
def __init__(self, *args, **kwargs):
Partition.__init__(self, *args, **kwargs)
@property
def subvolumes(self):
for filesystem in findmnt(pathlib.Path(self.path), recurse=True).get('filesystems', []):
if '[' in filesystem.get('source', ''):
yield subvolume_info_from_path(filesystem['target'])
def iterate_children(struct):
for c in struct.get('children', []):
if '[' in child.get('source', ''):
yield subvolume_info_from_path(c['target'])
for sub_child in iterate_children(c):
yield sub_child
for child in iterate_children(filesystem):
yield child
def create_subvolume(self, subvolume :pathlib.Path, installation :Optional['Installer'] = None) -> 'BtrfsSubvolumeInfo':
"""
Subvolumes have to be created within a mountpoint.
This means we need to get the current installation target.
After we get it, we need to verify it is a btrfs subvolume filesystem.
Finally, the destination must be empty.
"""
# Allow users to override the installation session
if not installation:
installation = storage.get('installation_session')
# Determain if the path given, is an absolute path or a relative path.
# We do this by checking if the path contains a known mountpoint.
if str(subvolume)[0] == '/':
if filesystems := findmnt(subvolume, traverse=True).get('filesystems'):
if (target := filesystems[0].get('target')) and target != '/' and str(subvolume).startswith(target):
# Path starts with a known mountpoint which isn't /
# Which means it's an absolute path to a mounted location.
pass
else:
# Since it's not an absolute position with a known start.
# We omit the anchor ('/' basically) and make sure it's appendable
# to the installation.target later
subvolume = subvolume.relative_to(subvolume.anchor)
# else: We don't need to do anything about relative paths, they should be appendable to installation.target as-is.
# If the subvolume is not absolute, then we do two checks:
# 1. Check if the partition itself is mounted somewhere, and use that as a root
# 2. Use an active Installer().target as the root, assuming it's filesystem is btrfs
# If both above fail, we need to warn the user that such setup is not supported.
if str(subvolume)[0] != '/':
if self.mountpoint is None and installation is None:
raise DiskError("When creating a subvolume on BTRFSPartition()'s, you need to either initiate a archinstall.Installer() or give absolute paths when creating the subvoulme.")
elif self.mountpoint:
subvolume = self.mountpoint / subvolume
elif installation:
ongoing_installation_destination = installation.target
if type(ongoing_installation_destination) == str:
ongoing_installation_destination = pathlib.Path(ongoing_installation_destination)
subvolume = ongoing_installation_destination / subvolume
subvolume.parent.mkdir(parents=True, exist_ok=True)
# <!--
# We perform one more check from the given absolute position.
# And we traverse backwards in order to locate any if possible subvolumes above
# our new btrfs subvolume. This is because it needs to be mounted under it to properly
# function.
# if btrfs_parent := find_parent_subvolume(subvolume):
# print('Found parent:', btrfs_parent)
# -->
log(f'Attempting to create subvolume at {subvolume}', level=logging.DEBUG, fg="grey")
if glob.glob(str(subvolume / '*')):
raise DiskError(f"Cannot create subvolume at {subvolume} because it contains data (non-empty folder target is not supported by BTRFS)")
# Ideally we would like to check if the destination is already a subvolume.
# But then we would need the mount-point at this stage as well.
# So we'll comment out this check:
# elif subvolinfo := subvolume_info_from_path(subvolume):
# raise DiskError(f"Destination {subvolume} is already a subvolume: {subvolinfo}")
# And deal with it here:
SysCommand(f"btrfs subvolume create {subvolume}")
return subvolume_info_from_path(subvolume)

192
archinstall/lib/disk/btrfs/btrfssubvolumeinfo.py
View File

@ -1,192 +0,0 @@
import pathlib
import datetime
import logging
import string
import random
import shutil
from dataclasses import dataclass
from typing import Optional, List# , TYPE_CHECKING
from functools import cached_property
# if TYPE_CHECKING:
# from ..blockdevice import BlockDevice
from ...exceptions import DiskError
from ...general import SysCommand
from ...output import log
from ...storage import storage
@dataclass
class BtrfsSubvolumeInfo:
full_path :pathlib.Path
name :str
uuid :str
parent_uuid :str
creation_time :datetime.datetime
subvolume_id :int
generation :int
gen_at_creation :int
parent_id :int
top_level_id :int
send_transid :int
send_time :datetime.datetime
receive_transid :int
received_uuid :Optional[str] = None
flags :Optional[str] = None
receive_time :Optional[datetime.datetime] = None
snapshots :Optional[List] = None
def __post_init__(self):
self.full_path = pathlib.Path(self.full_path)
# Convert "-" entries to `None`
if self.parent_uuid == "-":
self.parent_uuid = None
if self.received_uuid == "-":
self.received_uuid = None
if self.flags == "-":
self.flags = None
if self.receive_time == "-":
self.receive_time = None
if self.snapshots == "":
self.snapshots = []
# Convert timestamps into datetime workable objects (and preserve timezone by using ISO formats)
self.creation_time = datetime.datetime.fromisoformat(self.convert_to_ISO_format(self.creation_time))
self.send_time = datetime.datetime.fromisoformat(self.convert_to_ISO_format(self.send_time))
if self.receive_time:
self.receive_time = datetime.datetime.fromisoformat(self.convert_to_ISO_format(self.receive_time))
@property
def parent_subvolume(self):
from .btrfs_helpers import find_parent_subvolume
return find_parent_subvolume(self.full_path)
@property
def root(self) -> bool:
from .btrfs_helpers import subvolume_info_from_path
# TODO: Make this function traverse storage['MOUNT_POINT'] and find the first
# occurrence of a mountpoint that is a btrfs volume instead of lazy assume / is a subvolume.
# It would also be nice if it could use findmnt(self.full_path) and traverse backwards
# finding the last occurrence of a subvolume which 'self' belongs to.
if volume := subvolume_info_from_path(storage['MOUNT_POINT']):
return self.full_path == volume.full_path
return False
@cached_property
def partition(self):
from ..helpers import findmnt, get_parent_of_partition, all_blockdevices
from ..partition import Partition
from ..blockdevice import BlockDevice
from ..mapperdev import MapperDev
from .btrfspartition import BTRFSPartition
from .btrfs_helpers import subvolume_info_from_path
try:
# If the subvolume is mounted, it's pretty trivial to lookup the partition (parent) device.
if filesystem := findmnt(self.full_path).get('filesystems', []):
if source := filesystem[0].get('source', None):
# Strip away subvolume definitions from findmnt
if '[' in source:
source = source[:source.find('[')]
if filesystem[0].get('fstype', '') == 'btrfs':
return BTRFSPartition(source, BlockDevice(get_parent_of_partition(pathlib.Path(source))))
elif filesystem[0].get('source', '').startswith('/dev/mapper'):
return MapperDev(source)
else:
return Partition(source, BlockDevice(get_parent_of_partition(pathlib.Path(source))))
except DiskError:
# Subvolume has never been mounted, we have no reliable way of finding where it is.
# But we have the UUID of the partition, and can begin looking for it by mounting
# all blockdevices that we can reliably support.. This is taxing tho and won't cover all devices.
log(f"Looking up {self}, this might take time.", fg="orange", level=logging.WARNING)
for blockdevice, instance in all_blockdevices(mappers=True, partitions=True, error=True).items():
if type(instance) in (Partition, MapperDev):
we_mounted_it = False
detection_mountpoint = instance.mountpoint
if not detection_mountpoint:
if type(instance) == Partition and instance.encrypted:
# TODO: Perhaps support unlocking encrypted volumes?
# This will cause a lot of potential user interactions tho.
log(f"Ignoring {blockdevice} because it's encrypted.", fg="gray", level=logging.DEBUG)
continue
detection_mountpoint = pathlib.Path(f"/tmp/{''.join([random.choice(string.ascii_letters) for x in range(20)])}")
detection_mountpoint.mkdir(parents=True, exist_ok=True)
instance.mount(str(detection_mountpoint))
we_mounted_it = True
if (filesystem := findmnt(detection_mountpoint)) and (filesystem := filesystem.get('filesystems', [])):
if subvolume := subvolume_info_from_path(filesystem[0]['target']):
if subvolume.uuid == self.uuid:
# The top level subvolume matched of ourselves,
# which means the instance we're iterating has the subvol we're looking for.
log(f"Found the subvolume on device {instance}", level=logging.DEBUG, fg="gray")
return instance
def iterate_children(struct):
for child in struct.get('children', []):
if '[' in child.get('source', ''):
yield subvolume_info_from_path(child['target'])
for sub_child in iterate_children(child):
yield sub_child
for child in iterate_children(filesystem[0]):
if child.uuid == self.uuid:
# We found a child within the instance that has the subvol we're looking for.
log(f"Found the subvolume on device {instance}", level=logging.DEBUG, fg="gray")
return instance
if we_mounted_it:
instance.unmount()
shutil.rmtree(detection_mountpoint)
@cached_property
def mount_options(self) -> Optional[List[str]]:
from ..helpers import findmnt
if filesystem := findmnt(self.full_path).get('filesystems', []):
return filesystem[0].get('options').split(',')
def convert_to_ISO_format(self, time_string):
time_string_almost_done = time_string.replace(' ', 'T', 1).replace(' ', '')
iso_string = f"{time_string_almost_done[:-2]}:{time_string_almost_done[-2:]}"
return iso_string
def mount(self, mountpoint :pathlib.Path, options=None, include_previously_known_options=True):
from ..helpers import findmnt
try:
if mnt_info := findmnt(pathlib.Path(mountpoint), traverse=False):
log(f"Unmounting {mountpoint} as it was already mounted using {mnt_info}")
SysCommand(f"umount {mountpoint}")
except DiskError:
# No previously mounted device at the mountpoint
pass
if not options:
options = []
try:
if include_previously_known_options and (cached_options := self.mount_options):
options += cached_options
except DiskError:
pass
if not any('subvol=' in x for x in options):
options += f'subvol={self.name}'
SysCommand(f"mount {self.partition.path} {mountpoint} -o {','.join(options)}")
log(f"{self} has successfully been mounted to {mountpoint}", level=logging.INFO, fg="gray")
def unmount(self, recurse :bool = True):
SysCommand(f"umount {'-R' if recurse else ''} {self.full_path}")
log(f"Successfully unmounted {self}", level=logging.INFO, fg="gray")

599
archinstall/lib/disk/device_handler.py Normal file
View File

@ -0,0 +1,599 @@
from __future__ import annotations
import json
import logging
import os
import time
from pathlib import Path
from typing import List, Dict, Any, Optional, TYPE_CHECKING
from parted import ( # type: ignore
Disk, Geometry, FileSystem,
PartitionException, DiskLabelException,
getAllDevices, freshDisk, Partition,
)
from .device_model import (
DeviceModification, PartitionModification,
BDevice, _DeviceInfo, _PartitionInfo,
FilesystemType, Unit, PartitionTable,
ModificationStatus, get_lsblk_info, LsblkInfo,
_BtrfsSubvolumeInfo, get_all_lsblk_info, DiskEncryption
)
from ..exceptions import DiskError, UnknownFilesystemFormat
from ..general import SysCommand, SysCallError, JSON
from ..luks import Luks2
from ..output import log
from ..utils.util import is_subpath
if TYPE_CHECKING:
_: Any
class DeviceHandler(object):
_TMP_BTRFS_MOUNT = Path('/mnt/arch_btrfs')
def __init__(self):
self._devices: Dict[Path, BDevice] = {}
self.load_devices()
@property
def devices(self) -> List[BDevice]:
return list(self._devices.values())
def load_devices(self):
block_devices = {}
for device in getAllDevices():
try:
disk = Disk(device)
except DiskLabelException as error:
if 'unrecognised disk label' in getattr(error, 'message', str(error)):
disk = freshDisk(device, PartitionTable.GPT.value)
else:
log(f'Unable to get disk from device: {device}', level=logging.DEBUG)
continue
device_info = _DeviceInfo.from_disk(disk)
partition_infos = []
for partition in disk.partitions:
lsblk_info = get_lsblk_info(partition.path)
fs_type = self._determine_fs_type(partition, lsblk_info)
subvol_infos = []
if fs_type == FilesystemType.Btrfs:
subvol_infos = self.get_btrfs_info(partition.path)
partition_infos.append(
_PartitionInfo.from_partition(
partition,
fs_type,
lsblk_info.partuuid,
lsblk_info.mountpoints,
subvol_infos
)
)
block_device = BDevice(disk, device_info, partition_infos)
block_devices[block_device.device_info.path] = block_device
self._devices = block_devices
def _determine_fs_type(
self,
partition: Partition,
lsblk_info: Optional[LsblkInfo] = None
) -> Optional[FilesystemType]:
try:
if partition.fileSystem:
return FilesystemType(partition.fileSystem.type)
elif lsblk_info is not None:
return FilesystemType(lsblk_info.fstype) if lsblk_info.fstype else None
return None
except ValueError:
log(f'Could not determine the filesystem: {partition.fileSystem}', level=logging.DEBUG)
return None
def get_device(self, path: Path) -> Optional[BDevice]:
return self._devices.get(path, None)
def get_device_by_partition_path(self, partition_path: Path) -> Optional[BDevice]:
partition = self.find_partition(partition_path)
if partition:
return partition.disk.device
return None
def find_partition(self, path: Path) -> Optional[_PartitionInfo]:
for device in self._devices.values():
part = next(filter(lambda x: str(x.path) == str(path), device.partition_infos), None)
if part is not None:
return part
return None
def get_uuid_for_path(self, path: Path) -> Optional[str]:
partition = self.find_partition(path)
return partition.partuuid if partition else None
def get_btrfs_info(self, dev_path: Path) -> List[_BtrfsSubvolumeInfo]:
lsblk_info = get_lsblk_info(dev_path)
subvol_infos: List[_BtrfsSubvolumeInfo] = []
if not lsblk_info.mountpoint:
self.mount(dev_path, self._TMP_BTRFS_MOUNT, create_target_mountpoint=True)
mountpoint = self._TMP_BTRFS_MOUNT
else:
# when multiple subvolumes are mounted then the lsblk output may look like
# "mountpoint": "/mnt/archinstall/.snapshots"
# "mountpoints": ["/mnt/archinstall/.snapshots", "/mnt/archinstall/home", ..]
# so we'll determine the minimum common path and assume that's the root
path_strings = [str(m) for m in lsblk_info.mountpoints]
common_prefix = os.path.commonprefix(path_strings)
mountpoint = Path(common_prefix)
try:
result = SysCommand(f'btrfs subvolume list {mountpoint}')
except SysCallError as err:
log(f'Failed to read btrfs subvolume information: {err}', level=logging.DEBUG)
return subvol_infos
if result.exit_code == 0:
try:
if decoded := result.decode('utf-8'):
# ID 256 gen 16 top level 5 path @
for line in decoded.splitlines():
# expected output format:
# ID 257 gen 8 top level 5 path @home
name = Path(line.split(' ')[-1])
sub_vol_mountpoint = lsblk_info.btrfs_subvol_info.get(name, None)
subvol_infos.append(_BtrfsSubvolumeInfo(name, sub_vol_mountpoint))
except json.decoder.JSONDecodeError as err:
log(f"Could not decode lsblk JSON: {result}", fg="red", level=logging.ERROR)
raise err
if not lsblk_info.mountpoint:
self.umount(dev_path)
return subvol_infos
def _perform_formatting(
self,
fs_type: FilesystemType,
path: Path,
additional_parted_options: List[str] = []
):
options = []
command = ''
match fs_type:
case FilesystemType.Btrfs:
options += ['-f']
command += 'mkfs.btrfs'
case FilesystemType.Fat16:
options += ['-F16']
command += 'mkfs.fat'
case FilesystemType.Fat32:
options += ['-F32']
command += 'mkfs.fat'
case FilesystemType.Ext2:
options += ['-F']
command += 'mkfs.ext2'
case FilesystemType.Ext3:
options += ['-F']
command += 'mkfs.ext3'
case FilesystemType.Ext4:
options += ['-F']
command += 'mkfs.ext4'
case FilesystemType.Xfs:
options += ['-f']
command += 'mkfs.xfs'
case FilesystemType.F2fs:
options += ['-f']
command += 'mkfs.f2fs'
case FilesystemType.Ntfs:
options += ['-f', '-Q']
command += 'mkfs.ntfs'
case FilesystemType.Reiserfs:
command += 'mkfs.reiserfs'
case _:
raise UnknownFilesystemFormat(f'Filetype "{fs_type.value}" is not supported')
options += additional_parted_options
options_str = ' '.join(options)
log(f'Formatting filesystem: /usr/bin/{command} {options_str} {path}')
try:
if (handle := SysCommand(f"/usr/bin/{command} {options_str} {path}")).exit_code != 0:
mkfs_error = handle.decode()
raise DiskError(f'Could not format {path} with {fs_type.value}: {mkfs_error}')
except SysCallError as error:
msg = f'Could not format {path} with {fs_type.value}: {error.message}'
log(msg, fg='red')
raise DiskError(msg) from error
def _perform_enc_formatting(
self,
dev_path: Path,
mapper_name: Optional[str],
fs_type: FilesystemType,
enc_conf: DiskEncryption
):
luks_handler = Luks2(
dev_path,
mapper_name=mapper_name,
password=enc_conf.encryption_password
)
key_file = luks_handler.encrypt()
log(f'Unlocking luks2 device: {dev_path}', level=logging.DEBUG)
luks_handler.unlock(key_file=key_file)
if not luks_handler.mapper_dev:
raise DiskError('Failed to unlock luks device')
log(f'luks2 formatting mapper dev: {luks_handler.mapper_dev}', level=logging.INFO)
self._perform_formatting(fs_type, luks_handler.mapper_dev)
log(f'luks2 locking device: {dev_path}', level=logging.INFO)
luks_handler.lock()
def format(
self,
modification: DeviceModification,
enc_conf: Optional['DiskEncryption'] = None
):
"""
Format can be given an overriding path, for instance /dev/null to test
the formatting functionality and in essence the support for the given filesystem.
"""
# verify that all partitions have a path set (which implies that they have been created)
missing_path = next(filter(lambda x: x.dev_path is None, modification.partitions), None)
if missing_path is not None:
raise ValueError('When formatting, all partitions must have a path set')
# crypto luks is not known to parted and can therefore not
# be used as a filesystem type in that sense;
invalid_fs_type = next(filter(lambda x: x.fs_type is FilesystemType.Crypto_luks, modification.partitions), None)
if invalid_fs_type is not None:
raise ValueError('Crypto luks cannot be set as a filesystem type')
# make sure all devices are unmounted
self._umount_all_existing(modification)
for part_mod in modification.partitions:
# partition will be encrypted
if enc_conf is not None and part_mod in enc_conf.partitions:
self._perform_enc_formatting(
part_mod.real_dev_path,
part_mod.mapper_name,
part_mod.fs_type,
enc_conf
)
else:
self._perform_formatting(part_mod.fs_type, part_mod.real_dev_path)
def _perform_partitioning(
self,
part_mod: PartitionModification,
block_device: BDevice,
disk: Disk,
requires_delete: bool
):
# when we require a delete and the partition to be (re)created
# already exists then we have to delete it first
if requires_delete and part_mod.status in [ModificationStatus.Modify, ModificationStatus.Delete]:
log(f'Delete existing partition: {part_mod.real_dev_path}', level=logging.INFO)
part_info = self.find_partition(part_mod.real_dev_path)
if not part_info:
raise DiskError(f'No partition for dev path found: {part_mod.real_dev_path}')
disk.deletePartition(part_info.partition)
disk.commit()
if part_mod.status == ModificationStatus.Delete:
return
start_sector = part_mod.start.convert(
Unit.sectors,
block_device.device_info.sector_size
)
length_sector = part_mod.length.convert(
Unit.sectors,
block_device.device_info.sector_size
)
geometry = Geometry(
device=block_device.disk.device,
start=start_sector.value,
length=length_sector.value
)
filesystem = FileSystem(type=part_mod.fs_type.value, geometry=geometry)
partition = Partition(
disk=disk,
type=part_mod.type.get_partition_code(),
fs=filesystem,
geometry=geometry
)
for flag in part_mod.flags:
partition.setFlag(flag.value)
log(f'\tType: {part_mod.type.value}', level=logging.DEBUG)
log(f'\tFilesystem: {part_mod.fs_type.value}', level=logging.DEBUG)
log(f'\tGeometry: {start_sector.value} start sector, {length_sector.value} length', level=logging.DEBUG)
try:
disk.addPartition(partition=partition, constraint=disk.device.optimalAlignedConstraint)
disk.commit()
# the creation will take a bit of time
time.sleep(3)
# the partition has a real path now as it was created
part_mod.dev_path = Path(partition.path)
info = get_lsblk_info(part_mod.dev_path)
if not info.partuuid:
raise DiskError(f'Unable to determine new partition uuid: {part_mod.dev_path}')
part_mod.partuuid = info.partuuid
part_mod.uuid = info.uuid
except PartitionException as ex:
raise DiskError(f'Unable to add partition, most likely due to overlapping sectors: {ex}') from ex
def create_btrfs_volumes(
self,
part_mod: PartitionModification,
enc_conf: Optional['DiskEncryption'] = None
):
log(f'Creating subvolumes: {part_mod.real_dev_path}', level=logging.INFO)
luks_handler = None
# unlock the partition first if it's encrypted
if enc_conf is not None and part_mod in enc_conf.partitions:
if not part_mod.mapper_name:
raise ValueError('No device path specified for modification')
luks_handler = self.unlock_luks2_dev(
part_mod.real_dev_path,
part_mod.mapper_name,
enc_conf.encryption_password
)
if not luks_handler.mapper_dev:
raise DiskError('Failed to unlock luks device')
self.mount(luks_handler.mapper_dev, self._TMP_BTRFS_MOUNT, create_target_mountpoint=True)
else:
self.mount(part_mod.real_dev_path, self._TMP_BTRFS_MOUNT, create_target_mountpoint=True)
for sub_vol in part_mod.btrfs_subvols:
log(f'Creating subvolume: {sub_vol.name}', level=logging.DEBUG)
if luks_handler is not None:
subvol_path = self._TMP_BTRFS_MOUNT / sub_vol.name
else:
subvol_path = self._TMP_BTRFS_MOUNT / sub_vol.name
SysCommand(f"btrfs subvolume create {subvol_path}")
if sub_vol.nodatacow:
if (result := SysCommand(f'chattr +C {subvol_path}')).exit_code != 0:
raise DiskError(f'Could not set nodatacow attribute at {subvol_path}: {result.decode()}')
if sub_vol.compress:
if (result := SysCommand(f'chattr +c {subvol_path}')).exit_code != 0:
raise DiskError(f'Could not set compress attribute at {subvol_path}: {result}')
if luks_handler is not None and luks_handler.mapper_dev is not None:
self.umount(luks_handler.mapper_dev)
luks_handler.lock()
else:
self.umount(part_mod.real_dev_path)
def unlock_luks2_dev(self, dev_path: Path, mapper_name: str, enc_password: str) -> Luks2:
luks_handler = Luks2(dev_path, mapper_name=mapper_name, password=enc_password)
if not luks_handler.is_unlocked():
luks_handler.unlock()
if not luks_handler.is_unlocked():
raise DiskError(f'Failed to unlock luks2 device: {dev_path}')
return luks_handler
def _umount_all_existing(self, modification: DeviceModification):
log(f'Unmounting all partitions: {modification.device_path}', level=logging.INFO)
existing_partitions = self._devices[modification.device_path].partition_infos
for partition in existing_partitions:
log(f'Unmounting: {partition.path}', level=logging.DEBUG)
# un-mount for existing encrypted partitions
if partition.fs_type == FilesystemType.Crypto_luks:
Luks2(partition.path).lock()
else:
self.umount(partition.path, recursive=True)
def partition(
self,
modification: DeviceModification,
partition_table: Optional[PartitionTable] = None
):
"""
Create a partition table on the block device and create all partitions.
"""
if modification.wipe:
if partition_table is None:
raise ValueError('Modification is marked as wipe but no partitioning table was provided')
if partition_table.MBR and len(modification.partitions) > 3:
raise DiskError('Too many partitions on disk, MBR disks can only have 3 primary partitions')
# make sure all devices are unmounted
self._umount_all_existing(modification)
# WARNING: the entire device will be wiped and all data lost
if modification.wipe:
self.wipe_dev(modification.device)
part_table = partition_table.value if partition_table else None
disk = freshDisk(modification.device.disk.device, part_table)
else:
log(f'Use existing device: {modification.device_path}')
disk = modification.device.disk
log(f'Creating partitions: {modification.device_path}')
# TODO sort by delete first
for part_mod in modification.partitions:
# don't touch existing partitions
if part_mod.exists():
continue
# if the entire disk got nuked then we don't have to delete
# any existing partitions anymore because they're all gone already
requires_delete = modification.wipe is False
self._perform_partitioning(part_mod, modification.device, disk, requires_delete=requires_delete)
self.partprobe(modification.device.device_info.path)
def mount(
self,
dev_path: Path,
target_mountpoint: Path,
mount_fs: Optional[str] = None,
create_target_mountpoint: bool = True,
options: List[str] = []
):
if create_target_mountpoint and not target_mountpoint.exists():
target_mountpoint.mkdir(parents=True, exist_ok=True)
if not target_mountpoint.exists():
raise ValueError('Target mountpoint does not exist')
lsblk_info = get_lsblk_info(dev_path)
if target_mountpoint in lsblk_info.mountpoints:
log(f'Device already mounted at {target_mountpoint}')
return
str_options = ','.join(options)
str_options = f'-o {str_options}' if str_options else ''
mount_fs = f'-t {mount_fs}' if mount_fs else ''
command = f'mount {mount_fs} {str_options} {dev_path} {target_mountpoint}'
log(f'Mounting {dev_path}: command', level=logging.DEBUG)
try:
result = SysCommand(command)
if result.exit_code != 0:
raise DiskError(f'Could not mount {dev_path}: {command}\n{result.decode()}')
except SysCallError as err:
raise DiskError(f'Could not mount {dev_path}: {command}\n{err.message}')
def umount(self, mountpoint: Path, recursive: bool = False):
try:
lsblk_info = get_lsblk_info(mountpoint)
except SysCallError as ex:
# this could happen if before partitioning the device contained 3 partitions
# and after partitioning only 2 partitions were created, then the modifications object
# will have a reference to /dev/sX3 which is being tried to umount here now
if 'not a block device' in ex.message:
return
raise ex
if len(lsblk_info.mountpoints) > 0:
log(f'Partition {mountpoint} is currently mounted at: {[str(m) for m in lsblk_info.mountpoints]}', level=logging.DEBUG)
for mountpoint in lsblk_info.mountpoints:
log(f'Unmounting mountpoint: {mountpoint}', level=logging.DEBUG)
command = 'umount'
if recursive:
command += ' -R'
SysCommand(f'{command} {mountpoint}')
def detect_pre_mounted_mods(self, base_mountpoint: Path) -> List[DeviceModification]:
part_mods: Dict[Path, List[PartitionModification]] = {}
for device in self.devices:
for part_info in device.partition_infos:
for mountpoint in part_info.mountpoints:
if is_subpath(mountpoint, base_mountpoint):
path = Path(part_info.disk.device.path)
part_mods.setdefault(path, [])
part_mods[path].append(PartitionModification.from_existing_partition(part_info))
break
device_mods: List[DeviceModification] = []
for device_path, mods in part_mods.items():
device_mod = DeviceModification(self._devices[device_path], False, mods)
device_mods.append(device_mod)
return device_mods
def partprobe(self, path: Optional[Path] = None):
if path is not None:
command = f'partprobe {path}'
else:
command = 'partprobe'
try:
result = SysCommand(command)
if result.exit_code != 0:
log(f'Error calling partprobe: {result.decode()}', level=logging.DEBUG)
raise DiskError(f'Could not perform partprobe on {path}: {result.decode()}')
except SysCallError as error:
log(f"partprobe experienced an error with {path}: {error}", level=logging.DEBUG)
def _wipe(self, dev_path: Path):
"""
Wipe a device (partition or otherwise) of meta-data, be it file system, LVM, etc.
@param dev_path: Device path of the partition to be wiped.
@type dev_path: str
"""
with open(dev_path, 'wb') as p:
p.write(bytearray(1024))
def wipe_dev(self, block_device: BDevice):
"""
Wipe the block device of meta-data, be it file system, LVM, etc.
This is not intended to be secure, but rather to ensure that
auto-discovery tools don't recognize anything here.
"""
log(f'Wiping partitions and metadata: {block_device.device_info.path}')
for partition in block_device.partition_infos:
self._wipe(partition.path)
self._wipe(block_device.device_info.path)
device_handler = DeviceHandler()
def disk_layouts() -> str:
try:
lsblk_info = get_all_lsblk_info()
return json.dumps(lsblk_info, indent=4, sort_keys=True, cls=JSON)
except SysCallError as err:
log(f"Could not return disk layouts: {err}", level=logging.WARNING, fg="yellow")
return ''
except json.decoder.JSONDecodeError as err:
log(f"Could not return disk layouts: {err}", level=logging.WARNING, fg="yellow")
return ''

1033
archinstall/lib/disk/device_model.py Normal file
View File

File diff suppressed because it is too large Load Diff

40
archinstall/lib/disk/diskinfo.py
View File

@ -1,40 +0,0 @@
import dataclasses
import json
from dataclasses import dataclass, field
from typing import Optional, List
from ..general import SysCommand
from ..exceptions import DiskError
@dataclass
class LsblkInfo:
size: int = 0
log_sec: int = 0
pttype: Optional[str] = None
rota: bool = False
tran: Optional[str] = None
ptuuid: Optional[str] = None
partuuid: Optional[str] = None
uuid: Optional[str] = None
fstype: Optional[str] = None
type: Optional[str] = None
mountpoints: List[str] = field(default_factory=list)
def get_lsblk_info(dev_path: str) -> LsblkInfo:
fields = [f.name for f in dataclasses.fields(LsblkInfo)]
lsblk_fields = ','.join([f.upper().replace('_', '-') for f in fields])
output = SysCommand(f'lsblk --json -b -o+{lsblk_fields} {dev_path}').decode('UTF-8')
if output:
block_devices = json.loads(output)
info = block_devices['blockdevices'][0]
lsblk_info = LsblkInfo()
for f in fields:
setattr(lsblk_info, f, info[f.replace('_', '-')])
return lsblk_info
raise DiskError(f'Failed to read disk "{dev_path}" with lsblk')

48
archinstall/lib/disk/dmcryptdev.py
View File

@ -1,48 +0,0 @@
import pathlib
import logging
import json
from dataclasses import dataclass
from typing import Optional
from ..exceptions import SysCallError
from ..general import SysCommand
from ..output import log
from .mapperdev import MapperDev
@dataclass
class DMCryptDev:
dev_path :pathlib.Path
@property
def name(self):
with open(f"/sys/devices/virtual/block/{pathlib.Path(self.path).name}/dm/name", "r") as fh:
return fh.read().strip()
@property
def path(self):
return f"/dev/mapper/{self.dev_path}"
@property
def blockdev(self):
pass
@property
def MapperDev(self):
return MapperDev(mappername=self.name)
@property
def mountpoint(self) -> Optional[str]:
try:
data = json.loads(SysCommand(f"findmnt --json -R {self.dev_path}").decode())
for filesystem in data['filesystems']:
return filesystem.get('target')
except SysCallError as error:
# Not mounted anywhere most likely
log(f"Could not locate mount information for {self.dev_path}: {error}", level=logging.WARNING, fg="yellow")
pass
return None
@property
def filesystem(self) -> Optional[str]:
return self.MapperDev.filesystem

91
archinstall/lib/disk/encryption.py → archinstall/lib/disk/encryption_menu.py
View File

@ -1,30 +1,44 @@
from pathlib import Path
from typing import Dict, Optional, Any, TYPE_CHECKING, List
from ..menu.abstract_menu import Selector, AbstractSubMenu
from ..menu.menu import MenuSelectionType
from ..menu.table_selection_menu import TableMenu
from ..models.disk_encryption import EncryptionType, DiskEncryption
from ..user_interaction.partitioning_conf import current_partition_layout
from ..disk import (
DeviceModification,
PartitionModification,
DiskEncryption,
EncryptionType
)
from ..menu import (
Selector,
AbstractSubMenu,
MenuSelectionType,
TableMenu
)
from ..user_interaction.utils import get_password
from ..menu import Menu
from ..general import secret
from ..hsm.fido import Fido2Device, Fido2
from .fido import Fido2Device, Fido2
from ..output import FormattedOutput
if TYPE_CHECKING:
_: Any
class DiskEncryptionMenu(AbstractSubMenu):
def __init__(self, data_store: Dict[str, Any], preset: Optional[DiskEncryption], disk_layouts: Dict[str, Any]):
def __init__(
self,
mods: List[DeviceModification],
data_store: Dict[str, Any],
preset: Optional[DiskEncryption] = None
):
if preset:
self._preset = preset
else:
self._preset = DiskEncryption()
self._disk_layouts = disk_layouts
self._modifications = mods
super().__init__(data_store=data_store)
def _setup_selection_menu_options(self):
def setup_selection_menu_options(self):
self._menu_options['encryption_password'] = \
Selector(
_('Encryption password'),
@ -45,8 +59,8 @@ class DiskEncryptionMenu(AbstractSubMenu):
self._menu_options['partitions'] = \
Selector(
_('Partitions'),
func=lambda preset: select_partitions_to_encrypt(self._disk_layouts, preset),
display_func=lambda x: f'{sum([len(y) for y in x.values()])} {_("Partitions")}' if x else None,
func=lambda preset: select_partitions_to_encrypt(self._modifications.device_modifications, preset),
display_func=lambda x: f'{len(x)} {_("Partitions")}' if x else None,
dependencies=['encryption_password'],
default=self._preset.partitions,
preview_func=self._prev_disk_layouts,
@ -84,24 +98,18 @@ class DiskEncryptionMenu(AbstractSubMenu):
return None
def _prev_disk_layouts(self) -> Optional[str]:
selector = self._menu_options['partitions']
if selector.has_selection():
partitions: Dict[str, Any] = selector.current_selection
all_partitions = []
for parts in partitions.values():
all_partitions += parts
partitions: Optional[List[PartitionModification]] = self._menu_options['partitions'].current_selection
if partitions:
output = str(_('Partitions to be encrypted')) + '\n'
output += current_partition_layout(all_partitions, with_title=False)
output += FormattedOutput.as_table(partitions)
return output.rstrip()
return None
def select_encryption_type(preset: EncryptionType) -> Optional[EncryptionType]:
title = str(_('Select disk encryption option'))
options = [
# _type_to_text(EncryptionType.FullDiskEncryption),
EncryptionType.type_to_text(EncryptionType.Partition)
]
@ -137,38 +145,35 @@ def select_hsm(preset: Optional[Fido2Device] = None) -> Optional[Fido2Device]:
return None
def select_partitions_to_encrypt(disk_layouts: Dict[str, Any], preset: Dict[str, Any]) -> Dict[str, Any]:
# If no partitions was marked as encrypted, but a password was supplied and we have some disks to format..
# Then we need to identify which partitions to encrypt. This will default to / (root).
all_partitions = []
for blockdevice in disk_layouts.values():
if partitions := blockdevice.get('partitions'):
partitions = [p for p in partitions if p['mountpoint'] != '/boot']
all_partitions += partitions
def select_partitions_to_encrypt(
modification: List[DeviceModification],
preset: List[PartitionModification]
) -> List[PartitionModification]:
partitions: List[PartitionModification] = []
if all_partitions:
# do not allow encrypting the boot partition
for mod in modification:
partitions += list(filter(lambda x: x.mountpoint != Path('/boot'), mod.partitions))
# do not allow encrypting existing partitions that are not marked as wipe
avail_partitions = list(filter(lambda x: not x.exists(), partitions))
if avail_partitions:
title = str(_('Select which partitions to encrypt'))
partition_table = current_partition_layout(all_partitions, with_title=False).strip()
partition_table = FormattedOutput.as_table(avail_partitions)
choice = TableMenu(
title,
table_data=(all_partitions, partition_table),
table_data=(avail_partitions, partition_table),
preset=preset,
multi=True
).run()
match choice.type_:
case MenuSelectionType.Reset:
return {}
return []
case MenuSelectionType.Skip:
return preset
case MenuSelectionType.Selection:
selections: List[Any] = choice.value # type: ignore
partitions = {}
for path, device in disk_layouts.items():
for part in selections:
if part in device.get('partitions', []):
partitions.setdefault(path, []).append(part)
return partitions
return {}
return choice.multi_value
return []

61
archinstall/lib/hsm/fido.py → archinstall/lib/disk/fido.py
View File

@ -2,36 +2,11 @@ from __future__ import annotations
import getpass
import logging
from typing import List
from dataclasses import dataclass
from pathlib import Path
from typing import List, Dict
from .device_model import PartitionModification, Fido2Device
from ..general import SysCommand, SysCommandWorker, clear_vt100_escape_codes
from ..disk.partition import Partition
from ..general import log
@dataclass
class Fido2Device:
path: Path
manufacturer: str
product: str
def json(self) -> Dict[str, str]:
return {
'path': str(self.path),
'manufacturer': self.manufacturer,
'product': self.product
}
@classmethod
def parse_arg(cls, arg: Dict[str, str]) -> 'Fido2Device':
return Fido2Device(
Path(arg['path']),
arg['manufacturer'],
arg['product']
)
from ..output import log
class Fido2:
@ -92,18 +67,28 @@ class Fido2:
return cls._fido2_devices
@classmethod
def fido2_enroll(cls, hsm_device: Fido2Device, partition :Partition, password :str):
worker = SysCommandWorker(f"systemd-cryptenroll --fido2-device={hsm_device.path} {partition.real_device}", peek_output=True)
def fido2_enroll(
cls,
hsm_device: Fido2Device,
part_mod: PartitionModification,
password: str
):
worker = SysCommandWorker(f"systemd-cryptenroll --fido2-device={hsm_device.path} {part_mod.dev_path}", peek_output=True)
pw_inputted = False
pin_inputted = False
while worker.is_alive():
if pw_inputted is False and bytes(f"please enter current passphrase for disk {partition.real_device}", 'UTF-8') in worker._trace_log.lower():
worker.write(bytes(password, 'UTF-8'))
pw_inputted = True
if pw_inputted is False:
if bytes(f"please enter current passphrase for disk {part_mod.dev_path}", 'UTF-8') in worker._trace_log.lower():
worker.write(bytes(password, 'UTF-8'))
pw_inputted = True
elif pin_inputted is False:
if bytes(f"please enter security token pin", 'UTF-8') in worker._trace_log.lower():
worker.write(bytes(getpass.getpass(" "), 'UTF-8'))
pin_inputted = True
elif pin_inputted is False and bytes(f"please enter security token pin", 'UTF-8') in worker._trace_log.lower():
worker.write(bytes(getpass.getpass(" "), 'UTF-8'))
pin_inputted = True
log(f"You might need to touch the FIDO2 device to unlock it if no prompt comes up after 3 seconds.", level=logging.INFO, fg="yellow")
log(
f"You might need to touch the FIDO2 device to unlock it if no prompt comes up after 3 seconds.",
level=logging.INFO,
fg="yellow"
)

349
archinstall/lib/disk/filesystem.py
View File

@ -1,301 +1,98 @@
from __future__ import annotations
import time
import logging
import json
import pathlib
from typing import Optional, Dict, Any, TYPE_CHECKING
# https://stackoverflow.com/a/39757388/929999
from ..models.disk_encryption import DiskEncryption
import signal
import sys
import time
from typing import Any, Optional, TYPE_CHECKING
from .device_model import DiskLayoutConfiguration, DiskLayoutType, PartitionTable, FilesystemType, DiskEncryption
from .device_handler import device_handler
from ..hardware import has_uefi
from ..output import log
from ..menu import Menu
if TYPE_CHECKING:
from .blockdevice import BlockDevice
_: Any
from .partition import Partition
from .validators import valid_fs_type
from ..exceptions import DiskError, SysCallError
from ..general import SysCommand
from ..output import log
from ..storage import storage
GPT = 0b00000001
MBR = 0b00000010
# A sane default is 5MiB, that allows for plenty of buffer for GRUB on MBR
# but also 4MiB for memory cards for instance. And another 1MiB to avoid issues.
# (we've been pestered by disk issues since the start, so please let this be here for a few versions)
DEFAULT_PARTITION_START = '5MiB'
class Filesystem:
# TODO:
# When instance of a HDD is selected, check all usages and gracefully unmount them
# as well as close any crypto handles.
def __init__(self, blockdevice :BlockDevice, mode :int):
self.blockdevice = blockdevice
self.mode = mode
def __enter__(self, *args :str, **kwargs :str) -> 'Filesystem':
return self
def __repr__(self) -> str:
return f"Filesystem(blockdevice={self.blockdevice}, mode={self.mode})"
def __exit__(self, *args :str, **kwargs :str) -> bool:
# TODO: https://stackoverflow.com/questions/28157929/how-to-safely-handle-an-exception-inside-a-context-manager
if len(args) >= 2 and args[1]:
raise args[1]
SysCommand('sync')
return True
def partuuid_to_index(self, uuid :str) -> Optional[int]:
for i in range(storage['DISK_RETRY_ATTEMPTS']):
self.partprobe()
time.sleep(max(0.1, storage['DISK_TIMEOUTS'] * i))
# We'll use unreliable lbslk to grab children under the /dev/<device>
output = json.loads(SysCommand(f"lsblk --json {self.blockdevice.device}").decode('UTF-8'))
for device in output['blockdevices']:
for index, partition in enumerate(device.get('children', [])):
# But we'll use blkid to reliably grab the PARTUUID for that child device (partition)
partition_uuid = SysCommand(f"blkid -s PARTUUID -o value /dev/{partition.get('name')}").decode().strip()
if partition_uuid.lower() == uuid.lower():
return index
raise DiskError(f"Failed to convert PARTUUID {uuid} to a partition index number on blockdevice {self.blockdevice.device}")
def load_layout(self, layout :Dict[str, Any]) -> None:
from ..luks import luks2
from .btrfs import BTRFSPartition
# If the layout tells us to wipe the drive, we do so
if layout.get('wipe', False):
if self.mode == GPT:
if not self.parted_mklabel(self.blockdevice.device, "gpt"):
raise KeyError(f"Could not create a GPT label on {self}")
elif self.mode == MBR:
if not self.parted_mklabel(self.blockdevice.device, "msdos"):
raise KeyError(f"Could not create a MS-DOS label on {self}")
self.blockdevice.flush_cache()
time.sleep(3)
prev_partition = None
# We then iterate the partitions in order
for partition in layout.get('partitions', []):
# We don't want to re-add an existing partition (those containing a UUID already)
if partition.get('wipe', False) and not partition.get('PARTUUID', None):
start = partition.get('start') or (
prev_partition and f'{prev_partition["device_instance"].end_sectors}s' or DEFAULT_PARTITION_START)
partition['device_instance'] = self.add_partition(partition.get('type', 'primary'),
start=start,
end=partition.get('size', '100%'),
partition_format=partition.get('filesystem', {}).get('format', 'btrfs'),
skip_mklabel=layout.get('wipe', False) is not False)
elif (partition_uuid := partition.get('PARTUUID')):
# We try to deal with both UUID and PARTUUID of a partition when it's being re-used.
# We should re-name or separate this logi based on partition.get('PARTUUID') and partition.get('UUID')
# but for now, lets just attempt to deal with both.
try:
partition['device_instance'] = self.blockdevice.get_partition(uuid=partition_uuid)
except DiskError:
partition['device_instance'] = self.blockdevice.get_partition(partuuid=partition_uuid)
log(_("Re-using partition instance: {}").format(partition['device_instance']), level=logging.DEBUG, fg="gray")
else:
log(f"{self}.load_layout() doesn't know how to work without 'wipe' being set or UUID ({partition.get('PARTUUID')}) was given and found.", fg="yellow", level=logging.WARNING)
continue
if partition.get('filesystem', {}).get('format', False):
# needed for backward compatibility with the introduction of the new "format_options"
format_options = partition.get('options',[]) + partition.get('filesystem',{}).get('format_options',[])
disk_encryption: DiskEncryption = storage['arguments'].get('disk_encryption')
if disk_encryption and partition in disk_encryption.all_partitions:
if not partition['device_instance']:
raise DiskError(f"Internal error caused us to loose the partition. Please report this issue upstream!")
if partition.get('mountpoint',None):
loopdev = f"{storage.get('ENC_IDENTIFIER', 'ai')}{pathlib.Path(partition['mountpoint']).name}loop"
else:
loopdev = f"{storage.get('ENC_IDENTIFIER', 'ai')}{pathlib.Path(partition['device_instance'].path).name}"
partition['device_instance'].encrypt(password=disk_encryption.encryption_password)
# Immediately unlock the encrypted device to format the inner volume
with luks2(partition['device_instance'], loopdev, disk_encryption.encryption_password, auto_unmount=True) as unlocked_device:
if not partition.get('wipe'):
if storage['arguments'] == 'silent':
raise ValueError(f"Missing fs-type to format on newly created encrypted partition {partition['device_instance']}")
else:
if not partition.get('filesystem'):
partition['filesystem'] = {}
if not partition['filesystem'].get('format', False):
while True:
partition['filesystem']['format'] = input(f"Enter a valid fs-type for newly encrypted partition {partition['filesystem']['format']}: ").strip()
if not partition['filesystem']['format'] or valid_fs_type(partition['filesystem']['format']) is False:
log(_("You need to enter a valid fs-type in order to continue. See `man parted` for valid fs-type's."))
continue
break
unlocked_device.format(partition['filesystem']['format'], options=format_options)
elif partition.get('wipe', False):
if not partition['device_instance']:
raise DiskError(f"Internal error caused us to loose the partition. Please report this issue upstream!")
partition['device_instance'].format(partition['filesystem']['format'], options=format_options)
if partition['filesystem']['format'] == 'btrfs':
# We upgrade the device instance to a BTRFSPartition if we format it as such.
# This is so that we can gain access to more features than otherwise available in Partition()
partition['device_instance'] = BTRFSPartition(
partition['device_instance'].path,
block_device=partition['device_instance'].block_device,
encrypted=False,
filesystem='btrfs',
autodetect_filesystem=False
)
if partition.get('boot', False):
log(f"Marking partition {partition['device_instance']} as bootable.")
self.set(self.partuuid_to_index(partition['device_instance'].part_uuid), 'boot on')
prev_partition = partition
def find_partition(self, mountpoint :str) -> Partition:
for partition in self.blockdevice:
if partition.target_mountpoint == mountpoint or partition.mountpoint == mountpoint:
return partition
def partprobe(self) -> bool:
try:
SysCommand(f'partprobe {self.blockdevice.device}')
except SysCallError as error:
log(f"Could not execute partprobe: {error!r}", level=logging.ERROR, fg="red")
raise DiskError(f"Could not run partprobe on {self.blockdevice.device}: {error!r}")
return True
def raw_parted(self, string: str) -> SysCommand:
try:
cmd_handle = SysCommand(f'/usr/bin/parted -s {string}')
time.sleep(0.5)
return cmd_handle
except SysCallError as error:
log(f"Parted ended with a bad exit code: {error.exit_code} ({error})", level=logging.ERROR, fg="red")
return error
def parted(self, string: str) -> bool:
"""
Performs a parted execution of the given string
:param string: A raw string passed to /usr/bin/parted -s <string>
:type string: str
"""
if (parted_handle := self.raw_parted(string)).exit_code == 0:
return self.partprobe()
else:
raise DiskError(f"Parted failed to add a partition: {parted_handle}")
def use_entire_disk(self, root_filesystem_type :str = 'ext4') -> Partition:
# TODO: Implement this with declarative profiles instead.
raise ValueError("Installation().use_entire_disk() has to be re-worked.")
def add_partition(
class FilesystemHandler:
def __init__(
self,
partition_type :str,
start :str,
end :str,
partition_format :Optional[str] = None,
skip_mklabel :bool = False
) -> Partition:
log(f'Adding partition to {self.blockdevice}, {start}->{end}', level=logging.INFO)
disk_config: DiskLayoutConfiguration,
enc_conf: Optional[DiskEncryption] = None
):
self._disk_config = disk_config
self._enc_config = enc_conf
if len(self.blockdevice.partitions) == 0 and skip_mklabel is False:
# If it's a completely empty drive, and we're about to add partitions to it
# we need to make sure there's a filesystem label.
if self.mode == GPT:
if not self.parted_mklabel(self.blockdevice.device, "gpt"):
raise KeyError(f"Could not create a GPT label on {self}")
elif self.mode == MBR:
if not self.parted_mklabel(self.blockdevice.device, "msdos"):
raise KeyError(f"Could not create a MS-DOS label on {self}")
def perform_filesystem_operations(self, show_countdown: bool = True):
if self._disk_config.config_type == DiskLayoutType.Pre_mount:
log('Disk layout configuration is set to pre-mount, not performing any operations', level=logging.DEBUG)
return
self.blockdevice.flush_cache()
device_mods = list(filter(lambda x: len(x.partitions) > 0, self._disk_config.device_modifications))
previous_partuuids = []
for partition in self.blockdevice.partitions.values():
try:
previous_partuuids.append(partition.part_uuid)
except DiskError:
pass
if not device_mods:
log('No modifications required', level=logging.DEBUG)
return
# TODO this check should probably run in the setup process rather than during the installation
if self.mode == MBR:
if len(self.blockdevice.partitions) > 3:
DiskError("Too many partitions on disk, MBR disks can only have 3 primary partitions")
device_paths = ', '.join([str(mod.device.device_info.path) for mod in device_mods])
if partition_format:
parted_string = f'{self.blockdevice.device} mkpart {partition_type} {partition_format} {start} {end}'
else:
parted_string = f'{self.blockdevice.device} mkpart {partition_type} {start} {end}'
# Issue a final warning before we continue with something un-revertable.
# We mention the drive one last time, and count from 5 to 0.
print(str(_(' ! Formatting {} in ')).format(device_paths))
log(f"Adding partition using the following parted command: {parted_string}", level=logging.DEBUG)
if show_countdown:
self._do_countdown()
if self.parted(parted_string):
for count in range(storage.get('DISK_RETRY_ATTEMPTS', 3)):
self.blockdevice.flush_cache()
# Setup the blockdevice, filesystem (and optionally encryption).
# Once that's done, we'll hand over to perform_installation()
partition_table = PartitionTable.GPT
if has_uefi() is False:
partition_table = PartitionTable.MBR
new_partition_uuids = [partition.part_uuid for partition in self.blockdevice.partitions.values()]
new_partuuid_set = (set(previous_partuuids) ^ set(new_partition_uuids))
for mod in device_mods:
device_handler.partition(mod, partition_table=partition_table)
device_handler.format(mod, enc_conf=self._enc_config)
if len(new_partuuid_set) and (new_partuuid := new_partuuid_set.pop()):
try:
return self.blockdevice.get_partition(partuuid=new_partuuid)
except Exception as err:
log(f'Blockdevice: {self.blockdevice}', level=logging.ERROR, fg="red")
log(f'Partitions: {self.blockdevice.partitions}', level=logging.ERROR, fg="red")
log(f'Partition set: {new_partuuid_set}', level=logging.ERROR, fg="red")
log(f'New PARTUUID: {[new_partuuid]}', level=logging.ERROR, fg="red")
log(f'get_partition(): {self.blockdevice.get_partition}', level=logging.ERROR, fg="red")
raise err
else:
log(f"Could not get UUID for partition. Waiting {storage.get('DISK_TIMEOUTS', 1) * count}s before retrying.",level=logging.DEBUG)
self.partprobe()
time.sleep(max(0.1, storage.get('DISK_TIMEOUTS', 1)))
else:
print("Parted did not return True during partition creation")
for part_mod in mod.partitions:
if part_mod.fs_type == FilesystemType.Btrfs:
device_handler.create_btrfs_volumes(part_mod, enc_conf=self._enc_config)
total_partitions = set([partition.part_uuid for partition in self.blockdevice.partitions.values()])
total_partitions.update(previous_partuuids)
def _do_countdown(self) -> bool:
SIG_TRIGGER = False
# TODO: This should never be able to happen
log(f"Could not find the new PARTUUID after adding the partition.", level=logging.ERROR, fg="red")
log(f"Previous partitions: {previous_partuuids}", level=logging.ERROR, fg="red")
log(f"New partitions: {total_partitions}", level=logging.ERROR, fg="red")
def kill_handler(sig: int, frame: Any) -> None:
print()
exit(0)
raise DiskError(f"Could not add partition using: {parted_string}")
def sig_handler(sig: int, frame: Any) -> None:
signal.signal(signal.SIGINT, kill_handler)
def set_name(self, partition: int, name: str) -> bool:
return self.parted(f'{self.blockdevice.device} name {partition + 1} "{name}"') == 0
original_sigint_handler = signal.getsignal(signal.SIGINT)
signal.signal(signal.SIGINT, sig_handler)
def set(self, partition: int, string: str) -> bool:
log(f"Setting {string} on (parted) partition index {partition+1}", level=logging.INFO)
return self.parted(f'{self.blockdevice.device} set {partition + 1} {string}') == 0
for i in range(5, 0, -1):
print(f"{i}", end='')
def parted_mklabel(self, device: str, disk_label: str) -> bool:
log(f"Creating a new partition label on {device}", level=logging.INFO, fg="yellow")
# Try to unmount devices before attempting to run mklabel
try:
SysCommand(f'bash -c "umount {device}?"')
except:
pass
for x in range(4):
sys.stdout.flush()
time.sleep(0.25)
print(".", end='')
self.partprobe()
worked = self.raw_parted(f'{device} mklabel {disk_label}').exit_code == 0
self.partprobe()
if SIG_TRIGGER:
prompt = _('Do you really want to abort?')
choice = Menu(prompt, Menu.yes_no(), skip=False).run()
if choice.value == Menu.yes():
exit(0)
return worked
if SIG_TRIGGER is False:
sys.stdin.read()
SIG_TRIGGER = False
signal.signal(signal.SIGINT, sig_handler)
print()
signal.signal(signal.SIGINT, original_sigint_handler)
return True

556
archinstall/lib/disk/helpers.py
View File

@ -1,556 +0,0 @@
from __future__ import annotations
import json
import logging
import os # type: ignore
import pathlib
import re
import time
import glob
from typing import Union, List, Iterator, Dict, Optional, Any, TYPE_CHECKING
# https://stackoverflow.com/a/39757388/929999
from .diskinfo import get_lsblk_info
from ..models.subvolume import Subvolume
from .blockdevice import BlockDevice
from .dmcryptdev import DMCryptDev
from .mapperdev import MapperDev
from ..exceptions import SysCallError, DiskError
from ..general import SysCommand
from ..output import log
from ..storage import storage
if TYPE_CHECKING:
from .partition import Partition
ROOT_DIR_PATTERN = re.compile('^.*?/devices')
GIGA = 2 ** 30
def convert_size_to_gb(size :Union[int, float]) -> float:
return round(size / GIGA,1)
def sort_block_devices_based_on_performance(block_devices :List[BlockDevice]) -> Dict[BlockDevice, int]:
result = {device: 0 for device in block_devices}
for device, weight in result.items():
if device.spinning:
weight -= 10
else:
weight += 5
if device.bus_type == 'nvme':
weight += 20
elif device.bus_type == 'sata':
weight += 10
result[device] = weight
return result
def filter_disks_below_size_in_gb(devices :List[BlockDevice], gigabytes :int) -> Iterator[BlockDevice]:
for disk in devices:
if disk.size >= gigabytes:
yield disk
def select_largest_device(devices :List[BlockDevice], gigabytes :int, filter_out :Optional[List[BlockDevice]] = None) -> BlockDevice:
if not filter_out:
filter_out = []
copy_devices = [*devices]
for filter_device in filter_out:
if filter_device in copy_devices:
copy_devices.pop(copy_devices.index(filter_device))
copy_devices = list(filter_disks_below_size_in_gb(copy_devices, gigabytes))
if not len(copy_devices):
return None
return max(copy_devices, key=(lambda device : device.size))
def select_disk_larger_than_or_close_to(devices :List[BlockDevice], gigabytes :int, filter_out :Optional[List[BlockDevice]] = None) -> BlockDevice:
if not filter_out:
filter_out = []
copy_devices = [*devices]
for filter_device in filter_out:
if filter_device in copy_devices:
copy_devices.pop(copy_devices.index(filter_device))
if not len(copy_devices):
return None
return min(copy_devices, key=(lambda device : abs(device.size - gigabytes)))
def convert_to_gigabytes(string :str) -> float:
unit = string.strip()[-1]
size = float(string.strip()[:-1])
if unit == 'M':
size = size / 1024
elif unit == 'T':
size = size * 1024
return size
def device_state(name :str, *args :str, **kwargs :str) -> Optional[bool]:
# Based out of: https://askubuntu.com/questions/528690/how-to-get-list-of-all-non-removable-disk-device-names-ssd-hdd-and-sata-ide-onl/528709#528709
if os.path.isfile('/sys/block/{}/device/block/{}/removable'.format(name, name)):
with open('/sys/block/{}/device/block/{}/removable'.format(name, name)) as f:
if f.read(1) == '1':
return
path = ROOT_DIR_PATTERN.sub('', os.readlink('/sys/block/{}'.format(name)))
hotplug_buses = ("usb", "ieee1394", "mmc", "pcmcia", "firewire")
for bus in hotplug_buses:
if os.path.exists('/sys/bus/{}'.format(bus)):
for device_bus in os.listdir('/sys/bus/{}/devices'.format(bus)):
device_link = ROOT_DIR_PATTERN.sub('', os.readlink('/sys/bus/{}/devices/{}'.format(bus, device_bus)))
if re.search(device_link, path):
return
return True
def cleanup_bash_escapes(data :str) -> str:
return data.replace(r'\ ', ' ')
def blkid(cmd :str) -> Dict[str, Any]:
if '-o' in cmd and '-o export' not in cmd:
raise ValueError(f"blkid() requires '-o export' to be used and can therefore not continue reliably.")
elif '-o' not in cmd:
cmd += ' -o export'
try:
raw_data = SysCommand(cmd).decode()
except SysCallError as error:
log(f"Could not get block device information using blkid() using command {cmd}", level=logging.DEBUG)
raise error
result = {}
# Process the raw result
devname = None
for line in raw_data.split('\r\n'):
if not len(line):
devname = None
continue
key, val = line.split('=', 1)
if key.lower() == 'devname':
devname = val
# Lowercase for backwards compatibility with all_disks() previous use cases
result[devname] = {
"path": devname,
"PATH": devname
}
continue
result[devname][key] = cleanup_bash_escapes(val)
return result
def get_loop_info(path :str) -> Dict[str, Any]:
for drive in json.loads(SysCommand(['losetup', '--json']).decode('UTF_8'))['loopdevices']:
if not drive['name'] == path:
continue
return {
path: {
**drive,
'type' : 'loop',
'TYPE' : 'loop',
'DEVTYPE' : 'loop',
'PATH' : drive['name'],
'path' : drive['name']
}
}
return {}
def enrich_blockdevice_information(information :Dict[str, Any]) -> Dict[str, Any]:
result = {}
for device_path, device_information in information.items():
dev_name = pathlib.Path(device_information['PATH']).name
if not device_information.get('TYPE') or not device_information.get('DEVTYPE'):
with open(f"/sys/class/block/{dev_name}/uevent") as fh:
device_information.update(uevent(fh.read()))
if (dmcrypt_name := pathlib.Path(f"/sys/class/block/{dev_name}/dm/name")).exists():
with dmcrypt_name.open('r') as fh:
device_information['DMCRYPT_NAME'] = fh.read().strip()
result[device_path] = device_information
return result
def uevent(data :str) -> Dict[str, Any]:
information = {}
for line in data.replace('\r\n', '\n').split('\n'):
if len((line := line.strip())):
key, val = line.split('=', 1)
information[key] = val
return information
def get_blockdevice_uevent(dev_name :str) -> Dict[str, Any]:
device_information = {}
with open(f"/sys/class/block/{dev_name}/uevent") as fh:
device_information.update(uevent(fh.read()))
return {
f"/dev/{dev_name}" : {
**device_information,
'path' : f'/dev/{dev_name}',
'PATH' : f'/dev/{dev_name}',
'PTTYPE' : None
}
}
def all_disks() -> List[BlockDevice]:
log(f"[Deprecated] archinstall.all_disks() is deprecated. Use archinstall.all_blockdevices() with the appropriate filters instead.", level=logging.WARNING, fg="yellow")
return all_blockdevices(partitions=False, mappers=False)
def get_blockdevice_info(device_path, exclude_iso_dev :bool = True) -> Dict[str, Any]:
for retry_attempt in range(storage['DISK_RETRY_ATTEMPTS']):
partprobe(device_path)
time.sleep(max(0.1, storage['DISK_TIMEOUTS'] * retry_attempt))
try:
if exclude_iso_dev:
# exclude all devices associated with the iso boot locations
iso_devs = ['/run/archiso/airootfs', '/run/archiso/bootmnt']
try:
lsblk_info = get_lsblk_info(device_path)
except DiskError:
continue
if any([dev in lsblk_info.mountpoints for dev in iso_devs]):
continue
information = blkid(f'blkid -p -o export {device_path}')
return enrich_blockdevice_information(information)
except SysCallError as ex:
if ex.exit_code == 2:
# Assume that it's a loop device, and try to get info on it
try:
resolved_device_name = device_path.readlink().name
except OSError:
resolved_device_name = device_path.name
try:
information = get_loop_info(device_path)
if not information:
raise SysCallError(f"Could not get loop information for {resolved_device_name}", exit_code=1)
return enrich_blockdevice_information(information)
except SysCallError:
information = get_blockdevice_uevent(resolved_device_name)
return enrich_blockdevice_information(information)
else:
# We could not reliably get any information, perhaps the disk is clean of information?
if retry_attempt == storage['DISK_RETRY_ATTEMPTS'] - 1:
raise ex
def all_blockdevices(
mappers: bool = False,
partitions: bool = False,
error: bool = False,
exclude_iso_dev: bool = True
) -> Dict[str, Any]:
"""
Returns BlockDevice() and Partition() objects for all available devices.
"""
from .partition import Partition
instances = {}
# Due to lsblk being highly unreliable for this use case,
# we'll iterate the /sys/class definitions and find the information
# from there.
for block_device in glob.glob("/sys/class/block/*"):
try:
device_path = pathlib.Path(f"/dev/{pathlib.Path(block_device).readlink().name}")
except FileNotFoundError:
log(f"Unknown device found by '/sys/class/block/*', ignoring: {device_path}", level=logging.WARNING, fg="yellow")
if device_path.exists() is False:
log(f"Unknown device found by '/sys/class/block/*', ignoring: {device_path}", level=logging.WARNING, fg="yellow")
continue
information = get_blockdevice_info(device_path)
if not information:
continue
for path, path_info in information.items():
if path_info.get('DMCRYPT_NAME'):
instances[path] = DMCryptDev(dev_path=path)
elif path_info.get('PARTUUID') or path_info.get('PART_ENTRY_NUMBER'):
if partitions:
instances[path] = Partition(path, block_device=BlockDevice(get_parent_of_partition(pathlib.Path(path))))
elif path_info.get('PTTYPE', False) is not False or path_info.get('TYPE') == 'loop':
instances[path] = BlockDevice(path, path_info)
elif path_info.get('TYPE') in ('squashfs', 'erofs'):
# We can ignore squashfs devices (usually /dev/loop0 on Arch ISO)
continue
else:
log(f"Unknown device found by all_blockdevices(), ignoring: {information}", level=logging.WARNING, fg="yellow")
if mappers:
for block_device in glob.glob("/dev/mapper/*"):
if (pathobj := pathlib.Path(block_device)).is_symlink():
instances[f"/dev/mapper/{pathobj.name}"] = MapperDev(mappername=pathobj.name)
return instances
def get_parent_of_partition(path :pathlib.Path) -> pathlib.Path:
partition_name = path.name
pci_device = (pathlib.Path("/sys/class/block") / partition_name).resolve()
return f"/dev/{pci_device.parent.name}"
def harddrive(size :Optional[float] = None, model :Optional[str] = None, fuzzy :bool = False) -> Optional[BlockDevice]:
collection = all_blockdevices(partitions=False)
for drive in collection:
if size and convert_to_gigabytes(collection[drive]['size']) != size:
continue
if model and (collection[drive]['model'] is None or collection[drive]['model'].lower() != model.lower()):
continue
return collection[drive]
def split_bind_name(path :Union[pathlib.Path, str]) -> list:
# log(f"[Deprecated] Partition().subvolumes now contain the split bind name via it's subvolume.name instead.", level=logging.WARNING, fg="yellow")
# we check for the bind notation. if exist we'll only use the "true" device path
if '[' in str(path) : # is a bind path (btrfs subvolume path)
device_path, bind_path = str(path).split('[')
bind_path = bind_path[:-1].strip() # remove the ]
else:
device_path = path
bind_path = None
return device_path,bind_path
def find_mountpoint(device_path :str) -> Dict[str, Any]:
try:
for filesystem in json.loads(SysCommand(f'/usr/bin/findmnt -R --json {device_path}').decode())['filesystems']:
yield filesystem
except SysCallError:
return {}
def findmnt(path :pathlib.Path, traverse :bool = False, ignore :List = [], recurse :bool = True) -> Dict[str, Any]:
for traversal in list(map(str, [str(path)] + list(path.parents))):
if traversal in ignore:
continue
try:
log(f"Getting mount information for device path {traversal}", level=logging.DEBUG)
if (output := SysCommand(f"/usr/bin/findmnt --json {'--submounts' if recurse else ''} {traversal}").decode('UTF-8')):
return json.loads(output)
except SysCallError as error:
log(f"Could not get mount information on {path} but continuing and ignoring: {error}", level=logging.INFO, fg="gray")
pass
if not traverse:
break
raise DiskError(f"Could not get mount information for path {path}")
def get_mount_info(path :Union[pathlib.Path, str], traverse :bool = False, return_real_path :bool = False, ignore :List = []) -> Dict[str, Any]:
import traceback
log(f"Deprecated: archinstall.get_mount_info(). Use archinstall.findmnt() instead, which does not do any automatic parsing. Please change at:\n{''.join(traceback.format_stack())}")
device_path, bind_path = split_bind_name(path)
output = {}
for traversal in list(map(str, [str(device_path)] + list(pathlib.Path(str(device_path)).parents))):
if traversal in ignore:
continue
try:
log(f"Getting mount information for device path {traversal}", level=logging.DEBUG)
if (output := SysCommand(f'/usr/bin/findmnt --json {traversal}').decode('UTF-8')):
break
except SysCallError as error:
print('ERROR:', error)
pass
if not traverse:
break
if not output:
raise DiskError(f"Could not get mount information for device path {device_path}")
output = json.loads(output)
# for btrfs partitions we redice the filesystem list to the one with the source equals to the parameter
# i.e. the subvolume filesystem we're searching for
if 'filesystems' in output and len(output['filesystems']) > 1 and bind_path is not None:
output['filesystems'] = [entry for entry in output['filesystems'] if entry['source'] == str(path)]
if 'filesystems' in output:
if len(output['filesystems']) > 1:
raise DiskError(f"Path '{device_path}' contains multiple mountpoints: {output['filesystems']}")
if return_real_path:
return output['filesystems'][0], traversal
else:
return output['filesystems'][0]
if return_real_path:
return {}, traversal
else:
return {}
def get_all_targets(data :Dict[str, Any], filters :Dict[str, None] = {}) -> Dict[str, None]:
for info in data:
if info.get('target') not in filters:
filters[info.get('target')] = None
filters.update(get_all_targets(info.get('children', [])))
return filters
def get_partitions_in_use(mountpoint :str) -> Dict[str, Any]:
from .partition import Partition
try:
output = SysCommand(f"/usr/bin/findmnt --json -R {mountpoint}").decode('UTF-8')
except SysCallError:
return {}
if not output:
return {}
output = json.loads(output)
mounts = {}
block_devices_available = all_blockdevices(mappers=True, partitions=True, error=True)
block_devices_mountpoints = {}
for blockdev in block_devices_available.values():
if not type(blockdev) in (Partition, MapperDev):
continue
if isinstance(blockdev, Partition):
if blockdev.mountpoints:
for blockdev_mountpoint in blockdev.mountpoints:
block_devices_mountpoints[blockdev_mountpoint] = blockdev
else:
if blockdev.mount_information:
for blockdev_mountpoint in blockdev.mount_information:
block_devices_mountpoints[blockdev_mountpoint['target']] = blockdev
log(f'Filtering available mounts {block_devices_mountpoints} to those under {mountpoint}', level=logging.DEBUG)
for mountpoint in list(get_all_targets(output['filesystems']).keys()):
# Since all_blockdevices() returns PosixPath objects, we need to convert
# findmnt paths to pathlib.Path() first:
mountpoint = pathlib.Path(mountpoint)
if mountpoint in block_devices_mountpoints:
if mountpoint not in mounts:
mounts[mountpoint] = block_devices_mountpoints[mountpoint]
# If the already defined mountpoint is a DMCryptDev, and the newly found
# mountpoint is a MapperDev, it has precedence and replaces the old mountpoint definition.
elif type(mounts[mountpoint]) == DMCryptDev and type(block_devices_mountpoints[mountpoint]) == MapperDev:
mounts[mountpoint] = block_devices_mountpoints[mountpoint]
log(f"Available partitions: {mounts}", level=logging.DEBUG)
return mounts
def get_filesystem_type(path :str) -> Optional[str]:
try:
return SysCommand(f"blkid -o value -s TYPE {path}").decode('UTF-8').strip()
except SysCallError:
return None
def disk_layouts() -> Optional[Dict[str, Any]]:
try:
if (handle := SysCommand("lsblk -f -o+TYPE,SIZE -J")).exit_code == 0:
return {str(key): val for key, val in json.loads(handle.decode('UTF-8')).items()}
else:
log(f"Could not return disk layouts: {handle}", level=logging.WARNING, fg="yellow")
return None
except SysCallError as err:
log(f"Could not return disk layouts: {err}", level=logging.WARNING, fg="yellow")
return None
except json.decoder.JSONDecodeError as err:
log(f"Could not return disk layouts: {err}", level=logging.WARNING, fg="yellow")
return None
def find_partition_by_mountpoint(block_devices :List[BlockDevice], relative_mountpoint :str) -> Partition:
for device in block_devices:
for partition in block_devices[device]['partitions']:
if partition.get('mountpoint', None) == relative_mountpoint:
return partition
def partprobe(path :str = '') -> bool:
try:
if SysCommand(f'bash -c "partprobe {path}"').exit_code == 0:
return True
except SysCallError:
pass
return False
def convert_device_to_uuid(path :str) -> str:
device_name, bind_name = split_bind_name(path)
for i in range(storage['DISK_RETRY_ATTEMPTS']):
partprobe(device_name)
time.sleep(max(0.1, storage['DISK_TIMEOUTS'] * i)) # TODO: Remove, we should be relying on blkid instead of lsblk
# TODO: Convert lsblk to blkid
# (lsblk supports BlockDev and Partition UUID grabbing, blkid requires you to pick PTUUID and PARTUUID)
output = json.loads(SysCommand(f"lsblk --json -o+UUID {device_name}").decode('UTF-8'))
for device in output['blockdevices']:
if (dev_uuid := device.get('uuid', None)):
return dev_uuid
raise DiskError(f"Could not retrieve the UUID of {path} within a timely manner.")
def has_mountpoint(partition: Union[dict,Partition,MapperDev], target: str, strict: bool = True) -> bool:
""" Determine if a certain partition is mounted (or has a mountpoint) as specific target (path)
Coded for clarity rather than performance
Input parms:
:parm partition the partition we check
:type Either a Partition object or a dict with the contents of a partition definition in the disk_layouts schema
:parm target (a string representing a mount path we want to check for.
:type str
:parm strict if the check will be strict, target is exactly the mountpoint, or no, where the target is a leaf (f.i. to check if it is in /mnt/archinstall/). Not available for root check ('/') for obvious reasons
"""
# we create the mountpoint list
if isinstance(partition,dict):
subvolumes: List[Subvolume] = partition.get('btrfs',{}).get('subvolumes', [])
mountpoints = [partition.get('mountpoint')]
mountpoints += [volume.mountpoint for volume in subvolumes]
else:
mountpoints = [partition.mountpoint,] + [subvol.target for subvol in partition.subvolumes]
# we check
if strict or target == '/':
if target in mountpoints:
return True
else:
return False
else:
for mp in mountpoints:
if mp and mp.endswith(target):
return True
return False

92
archinstall/lib/disk/mapperdev.py
View File

@ -1,92 +0,0 @@
import glob
import pathlib
import logging
import json
from dataclasses import dataclass
from typing import Optional, List, Dict, Any, Iterator, TYPE_CHECKING
from ..exceptions import SysCallError
from ..general import SysCommand
from ..output import log
if TYPE_CHECKING:
from .btrfs import BtrfsSubvolumeInfo
@dataclass
class MapperDev:
mappername :str
@property
def name(self):
return self.mappername
@property
def path(self):
return f"/dev/mapper/{self.mappername}"
@property
def part_uuid(self):
return self.partition.part_uuid
@property
def partition(self):
from .helpers import uevent, get_parent_of_partition
from .partition import Partition
from .blockdevice import BlockDevice
for mapper in glob.glob('/dev/mapper/*'):
path_obj = pathlib.Path(mapper)
if path_obj.name == self.mappername and pathlib.Path(mapper).is_symlink():
dm_device = (pathlib.Path("/dev/mapper/") / path_obj.readlink()).resolve()
for slave in glob.glob(f"/sys/class/block/{dm_device.name}/slaves/*"):
partition_belonging_to_dmcrypt_device = pathlib.Path(slave).name
try:
uevent_data = SysCommand(f"blkid -o export /dev/{partition_belonging_to_dmcrypt_device}").decode()
except SysCallError as error:
log(f"Could not get information on device /dev/{partition_belonging_to_dmcrypt_device}: {error}", level=logging.ERROR, fg="red")
information = uevent(uevent_data)
block_device = BlockDevice(get_parent_of_partition('/dev/' / pathlib.Path(information['DEVNAME'])))
return Partition(information['DEVNAME'], block_device=block_device)
raise ValueError(f"Could not convert {self.mappername} to a real dm-crypt device")
@property
def mountpoint(self) -> Optional[pathlib.Path]:
try:
data = json.loads(SysCommand(f"findmnt --json -R {self.path}").decode())
for filesystem in data['filesystems']:
return pathlib.Path(filesystem.get('target'))
except SysCallError as error:
# Not mounted anywhere most likely
log(f"Could not locate mount information for {self.path}: {error}", level=logging.WARNING, fg="yellow")
pass
return None
@property
def mountpoints(self) -> List[Dict[str, Any]]:
return [obj['target'] for obj in self.mount_information]
@property
def mount_information(self) -> List[Dict[str, Any]]:
from .helpers import find_mountpoint
return [{**obj, 'target' : pathlib.Path(obj.get('target', '/dev/null'))} for obj in find_mountpoint(self.path)]
@property
def filesystem(self) -> Optional[str]:
from .helpers import get_filesystem_type
return get_filesystem_type(self.path)
@property
def subvolumes(self) -> Iterator['BtrfsSubvolumeInfo']:
from .btrfs import subvolume_info_from_path
for mountpoint in self.mount_information:
if target := mountpoint.get('target'):
if subvolume := subvolume_info_from_path(pathlib.Path(target)):
yield subvolume

661
archinstall/lib/disk/partition.py
View File

@ -1,661 +0,0 @@
import glob
import time
import logging
import json
import os
import hashlib
import typing
from dataclasses import dataclass, field
from pathlib import Path
from typing import Optional, Dict, Any, List, Union, Iterator
from .blockdevice import BlockDevice
from .helpers import get_filesystem_type, convert_size_to_gb, split_bind_name
from ..storage import storage
from ..exceptions import DiskError, SysCallError, UnknownFilesystemFormat
from ..output import log
from ..general import SysCommand
from .btrfs.btrfs_helpers import subvolume_info_from_path
from .btrfs.btrfssubvolumeinfo import BtrfsSubvolumeInfo
@dataclass
class PartitionInfo:
partition_object: 'Partition'
device_path: str # This would be /dev/sda1 for instance
bootable: bool
size: float
sector_size: int
start: Optional[int]
end: Optional[int]
pttype: Optional[str]
filesystem_type: Optional[str]
partuuid: Optional[str]
uuid: Optional[str]
mountpoints: List[Path] = field(default_factory=list)
def __post_init__(self):
if not all([self.partuuid, self.uuid]):
for i in range(storage['DISK_RETRY_ATTEMPTS']):
lsblk_info = SysCommand(f"lsblk --json -b -o+LOG-SEC,SIZE,PTTYPE,PARTUUID,UUID,FSTYPE {self.device_path}").decode('UTF-8')
try:
lsblk_info = json.loads(lsblk_info)
except json.decoder.JSONDecodeError:
log(f"Could not decode JSON: {lsblk_info}", fg="red", level=logging.ERROR)
raise DiskError(f'Failed to retrieve information for "{self.device_path}" with lsblk')
if not (device := lsblk_info.get('blockdevices', [None])[0]):
raise DiskError(f'Failed to retrieve information for "{self.device_path}" with lsblk')
self.partuuid = device.get('partuuid')
self.uuid = device.get('uuid')
# Lets build a list of requirements that we would like
# to retry and build (stuff that can take time between partprobes)
requirements = []
requirements.append(self.partuuid)
# Unformatted partitions won't have a UUID
if lsblk_info.get('fstype') is not None:
requirements.append(self.uuid)
if all(requirements):
break
self.partition_object.partprobe()
time.sleep(max(0.1, storage['DISK_TIMEOUTS'] * i))
def get_first_mountpoint(self) -> Optional[Path]:
if len(self.mountpoints) > 0:
return self.mountpoints[0]
return None
class Partition:
def __init__(
self,
path: str,
block_device: BlockDevice,
part_id :Optional[str] = None,
filesystem :Optional[str] = None,
mountpoint :Optional[str] = None,
encrypted :bool = False,
autodetect_filesystem :bool = True,
):
if not part_id:
part_id = os.path.basename(path)
if type(block_device) is str:
raise ValueError(f"Partition()'s 'block_device' parameter has to be a archinstall.BlockDevice() instance!")
self.block_device = block_device
self._path = path
self._part_id = part_id
self._target_mountpoint = mountpoint
self._encrypted = encrypted
self._wipe = False
self._type = 'primary'
if mountpoint:
self.mount(mountpoint)
try:
self._partition_info = self._fetch_information()
if not autodetect_filesystem and filesystem:
self._partition_info.filesystem_type = filesystem
if self._partition_info.filesystem_type == 'crypto_LUKS':
self._encrypted = True
except DiskError:
self._partition_info = None
@typing.no_type_check # I hate doint this but I'm currently unsure where this is used.
def __lt__(self, left_comparitor :BlockDevice) -> bool:
if type(left_comparitor) == Partition:
left_comparitor = left_comparitor.path
else:
left_comparitor = str(left_comparitor)
# The goal is to check if /dev/nvme0n1p1 comes before /dev/nvme0n1p5
return self._path < left_comparitor
def __repr__(self, *args :str, **kwargs :str) -> str:
mount_repr = ''
if self._partition_info:
if mountpoint := self._partition_info.get_first_mountpoint():
mount_repr = f", mounted={mountpoint}"
elif self._target_mountpoint:
mount_repr = f", rel_mountpoint={self._target_mountpoint}"
classname = self.__class__.__name__
if not self._partition_info:
return f'{classname}(path={self._path})'
elif self._encrypted:
return f'{classname}(path={self._path}, size={self.size}, PARTUUID={self.part_uuid}, parent={self.real_device}, fs={self._partition_info.filesystem_type}{mount_repr})'
else:
return f'{classname}(path={self._path}, size={self.size}, PARTUUID={self.part_uuid}, fs={self._partition_info.filesystem_type}{mount_repr})'
def as_json(self) -> Dict[str, Any]:
"""
this is used for the table representation of the partition (see FormattedOutput)
"""
partition_info = {
'type': self._type,
'PARTUUID': self.part_uuid,
'wipe': self._wipe,
'boot': self.boot,
'ESP': self.boot,
'mountpoint': self._target_mountpoint,
'encrypted': self._encrypted,
'start': self.start,
'size': self.end,
'filesystem': self._partition_info.filesystem_type if self._partition_info else 'Unknown'
}
return partition_info
def __dump__(self) -> Dict[str, Any]:
# TODO remove this in favour of as_json
return {
'type': self._type,
'PARTUUID': self.part_uuid,
'wipe': self._wipe,
'boot': self.boot,
'ESP': self.boot,
'mountpoint': self._target_mountpoint,
'encrypted': self._encrypted,
'start': self.start,
'size': self.end,
'filesystem': {
'format': self._partition_info.filesystem_type if self._partition_info else 'None'
}
}
def _call_lsblk(self) -> Dict[str, Any]:
for retry_attempt in range(storage['DISK_RETRY_ATTEMPTS']):
self.partprobe()
time.sleep(max(0.1, storage['DISK_TIMEOUTS'] * retry_attempt)) # TODO: Remove, we should be relying on blkid instead of lsblk
# This sleep might be overkill, but lsblk is known to
# work against a chaotic cache that can change during call
# causing no information to be returned (blkid is better)
# time.sleep(1)
# TODO: Maybe incorporate a re-try system here based on time.sleep(max(0.1, storage.get('DISK_TIMEOUTS', 1)))
try:
output = SysCommand(f"lsblk --json -b -o+LOG-SEC,SIZE,PTTYPE,PARTUUID,UUID,FSTYPE {self.device_path}").decode('UTF-8')
except SysCallError as error:
# Get the output minus the message/info from lsblk if it returns a non-zero exit code.
output = error.worker.decode('UTF-8')
if '{' in output:
output = output[output.find('{'):]
if output:
try:
lsblk_info = json.loads(output)
return lsblk_info
except json.decoder.JSONDecodeError:
log(f"Could not decode JSON: {output}", fg="red", level=logging.ERROR)
raise DiskError(f'Failed to get partition information "{self.device_path}" with lsblk')
def _call_sfdisk(self) -> Dict[str, Any]:
output = SysCommand(f"sfdisk --json {self.block_device.path}").decode('UTF-8')
if output:
sfdisk_info = json.loads(output)
partitions = sfdisk_info.get('partitiontable', {}).get('partitions', [])
node = list(filter(lambda x: x['node'] == self._path, partitions))
if len(node) > 0:
return node[0]
return {}
raise DiskError(f'Failed to read disk "{self.block_device.path}" with sfdisk')
def _fetch_information(self) -> PartitionInfo:
lsblk_info = self._call_lsblk()
sfdisk_info = self._call_sfdisk()
if not (device := lsblk_info.get('blockdevices', [])):
raise DiskError(f'Failed to retrieve information for "{self.device_path}" with lsblk')
# Grab the first (and only) block device in the list as we're targeting a specific partition
device = device[0]
mountpoints = [Path(mountpoint) for mountpoint in device['mountpoints'] if mountpoint]
bootable = sfdisk_info.get('bootable', False) or sfdisk_info.get('type', '') == 'C12A7328-F81F-11D2-BA4B-00A0C93EC93B'
return PartitionInfo(
partition_object=self,
device_path=self._path,
pttype=device['pttype'],
partuuid=device['partuuid'],
uuid=device['uuid'],
sector_size=device['log-sec'],
size=convert_size_to_gb(device['size']),
start=sfdisk_info.get('start', None),
end=sfdisk_info.get('size', None),
bootable=bootable,
filesystem_type=device['fstype'],
mountpoints=mountpoints
)
@property
def target_mountpoint(self) -> Optional[str]:
return self._target_mountpoint
@property
def path(self) -> str:
return self._path
@property
def filesystem(self) -> str:
if self._partition_info:
return self._partition_info.filesystem_type
@property
def mountpoint(self) -> Optional[Path]:
if len(self.mountpoints) > 0:
return self.mountpoints[0]
return None
@property
def mountpoints(self) -> List[Path]:
if self._partition_info:
return self._partition_info.mountpoints
@property
def sector_size(self) -> int:
if self._partition_info:
return self._partition_info.sector_size
@property
def start(self) -> Optional[int]:
if self._partition_info:
return self._partition_info.start
@property
def end(self) -> Optional[int]:
if self._partition_info:
return self._partition_info.end
@property
def end_sectors(self) -> Optional[int]:
if self._partition_info:
start = self._partition_info.start
end = self._partition_info.end
if start and end:
return start + end
@property
def size(self) -> Optional[float]:
if self._partition_info:
return self._partition_info.size
@property
def boot(self) -> bool:
if self._partition_info:
return self._partition_info.bootable
@property
def partition_type(self) -> Optional[str]:
if self._partition_info:
return self._partition_info.pttype
@property
def part_uuid(self) -> str:
if self._partition_info:
return self._partition_info.partuuid
@property
def uuid(self) -> Optional[str]:
"""
Returns the UUID as returned by lsblk for the **partition**.
This is more reliable than relying on /dev/disk/by-uuid as
it doesn't seam to be able to detect md raid partitions.
For bind mounts all the subvolumes share the same uuid
"""
for i in range(storage['DISK_RETRY_ATTEMPTS']):
if not self.partprobe():
raise DiskError(f"Could not perform partprobe on {self.device_path}")
time.sleep(storage.get('DISK_TIMEOUTS', 1) * i)
partuuid = self._safe_uuid
if partuuid:
return partuuid
raise DiskError(f"Could not get PARTUUID for {self.path} using 'blkid -s PARTUUID -o value {self.path}'")
@property
def _safe_uuid(self) -> Optional[str]:
"""
A near copy of self.uuid but without any delays.
This function should only be used where uuid is not crucial.
For instance when you want to get a __repr__ of the class.
"""
if not self.partprobe():
if self.block_device.partition_type == 'iso9660':
return None
log(f"Could not reliably refresh PARTUUID of partition {self.device_path} due to partprobe error.", level=logging.DEBUG)
try:
return SysCommand(f'blkid -s UUID -o value {self.device_path}').decode('UTF-8').strip()
except SysCallError as error:
if self.block_device.partition_type == 'iso9660':
# Parent device is a Optical Disk (.iso dd'ed onto a device for instance)
return None
log(f"Could not get PARTUUID of partition using 'blkid -s UUID -o value {self.device_path}': {error}")
@property
def _safe_part_uuid(self) -> Optional[str]:
"""
A near copy of self.uuid but without any delays.
This function should only be used where uuid is not crucial.
For instance when you want to get a __repr__ of the class.
"""
if not self.partprobe():
if self.block_device.partition_type == 'iso9660':
return None
log(f"Could not reliably refresh PARTUUID of partition {self.device_path} due to partprobe error.", level=logging.DEBUG)
try:
return SysCommand(f'blkid -s PARTUUID -o value {self.device_path}').decode('UTF-8').strip()
except SysCallError as error:
if self.block_device.partition_type == 'iso9660':
# Parent device is a Optical Disk (.iso dd'ed onto a device for instance)
return None
log(f"Could not get PARTUUID of partition using 'blkid -s PARTUUID -o value {self.device_path}': {error}")
if self._partition_info:
return self._partition_info.uuid
@property
def encrypted(self) -> Union[bool, None]:
return self._encrypted
@property
def parent(self) -> str:
return self.real_device
@property
def real_device(self) -> str:
output = SysCommand('lsblk -J').decode('UTF-8')
if output:
for blockdevice in json.loads(output)['blockdevices']:
if parent := self.find_parent_of(blockdevice, os.path.basename(self.device_path)):
return f"/dev/{parent}"
return self._path
raise DiskError('Unable to get disk information for command "lsblk -J"')
@property
def device_path(self) -> str:
""" for bind mounts returns the physical path of the partition
"""
device_path, bind_name = split_bind_name(self._path)
return device_path
@property
def bind_name(self) -> str:
""" for bind mounts returns the bind name (subvolume path).
Returns none if this property does not exist
"""
device_path, bind_name = split_bind_name(self._path)
return bind_name
@property
def subvolumes(self) -> Iterator[BtrfsSubvolumeInfo]:
from .helpers import findmnt
def iterate_children_recursively(information):
for child in information.get('children', []):
if target := child.get('target'):
if child.get('fstype') == 'btrfs':
if subvolume := subvolume_info_from_path(Path(target)):
yield subvolume
if child.get('children'):
for subchild in iterate_children_recursively(child):
yield subchild
if self._partition_info.filesystem_type == 'btrfs':
for mountpoint in self._partition_info.mountpoints:
if result := findmnt(mountpoint):
for filesystem in result.get('filesystems', []):
if subvolume := subvolume_info_from_path(mountpoint):
yield subvolume
for child in iterate_children_recursively(filesystem):
yield child
def partprobe(self) -> bool:
try:
if self.block_device:
return 0 == SysCommand(f'partprobe {self.block_device.device}').exit_code
except SysCallError as error:
log(f"Unreliable results might be given for {self._path} due to partprobe error: {error}", level=logging.DEBUG)
return False
def detect_inner_filesystem(self, password :str) -> Optional[str]:
log(f'Trying to detect inner filesystem format on {self} (This might take a while)', level=logging.INFO)
from ..luks import luks2
try:
with luks2(self, storage.get('ENC_IDENTIFIER', 'ai') + 'loop', password, auto_unmount=True) as unlocked_device:
return unlocked_device.filesystem
except SysCallError:
pass
return None
def has_content(self) -> bool:
fs_type = self._partition_info.filesystem_type
if not fs_type or "swap" in fs_type:
return False
temporary_mountpoint = '/tmp/' + hashlib.md5(bytes(f"{time.time()}", 'UTF-8') + os.urandom(12)).hexdigest()
temporary_path = Path(temporary_mountpoint)
temporary_path.mkdir(parents=True, exist_ok=True)
if (handle := SysCommand(f'/usr/bin/mount {self._path} {temporary_mountpoint}')).exit_code != 0:
raise DiskError(f'Could not mount and check for content on {self._path} because: {handle}')
files = len(glob.glob(f"{temporary_mountpoint}/*"))
iterations = 0
while SysCommand(f"/usr/bin/umount -R {temporary_mountpoint}").exit_code != 0 and (iterations := iterations + 1) < 10:
time.sleep(1)
temporary_path.rmdir()
return True if files > 0 else False
def encrypt(self, password: Optional[str] = None) -> str:
"""
A wrapper function for luks2() instances and the .encrypt() method of that instance.
"""
from ..luks import luks2
handle = luks2(self, None, None)
return handle.encrypt(self, password=password)
def format(self, filesystem :Optional[str] = None, path :Optional[str] = None, log_formatting :bool = True, options :List[str] = [], retry :bool = True) -> bool:
"""
Format can be given an overriding path, for instance /dev/null to test
the formatting functionality and in essence the support for the given filesystem.
"""
if filesystem is None:
filesystem = self._partition_info.filesystem_type
if path is None:
path = self._path
# This converts from fat32 -> vfat to unify filesystem names
filesystem = get_mount_fs_type(filesystem)
# To avoid "unable to open /dev/x: No such file or directory"
start_wait = time.time()
while Path(path).exists() is False and time.time() - start_wait < 10:
time.sleep(0.025)
if log_formatting:
log(f'Formatting {path} -> {filesystem}', level=logging.INFO)
try:
if filesystem == 'btrfs':
options = ['-f'] + options
mkfs = SysCommand(f"/usr/bin/mkfs.btrfs {' '.join(options)} {path}").decode('UTF-8')
if mkfs and 'UUID:' not in mkfs:
raise DiskError(f'Could not format {path} with {filesystem} because: {mkfs}')
self._partition_info.filesystem_type = filesystem
elif filesystem == 'vfat':
options = ['-F32'] + options
log(f"/usr/bin/mkfs.vfat {' '.join(options)} {path}")
if (handle := SysCommand(f"/usr/bin/mkfs.vfat {' '.join(options)} {path}")).exit_code != 0:
raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
self._partition_info.filesystem_type = filesystem
elif filesystem == 'ext4':
options = ['-F'] + options
if (handle := SysCommand(f"/usr/bin/mkfs.ext4 {' '.join(options)} {path}")).exit_code != 0:
raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
self._partition_info.filesystem_type = filesystem
elif filesystem == 'ext2':
options = ['-F'] + options
if (handle := SysCommand(f"/usr/bin/mkfs.ext2 {' '.join(options)} {path}")).exit_code != 0:
raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
self._partition_info.filesystem_type = 'ext2'
elif filesystem == 'xfs':
options = ['-f'] + options
if (handle := SysCommand(f"/usr/bin/mkfs.xfs {' '.join(options)} {path}")).exit_code != 0:
raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
self._partition_info.filesystem_type = filesystem
elif filesystem == 'f2fs':
options = ['-f'] + options
if (handle := SysCommand(f"/usr/bin/mkfs.f2fs {' '.join(options)} {path}")).exit_code != 0:
raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
self._partition_info.filesystem_type = filesystem
elif filesystem == 'ntfs3':
options = ['-f'] + options
if (handle := SysCommand(f"/usr/bin/mkfs.ntfs -Q {' '.join(options)} {path}")).exit_code != 0:
raise DiskError(f"Could not format {path} with {filesystem} because: {handle.decode('UTF-8')}")
self._partition_info.filesystem_type = filesystem
elif filesystem == 'crypto_LUKS':
# from ..luks import luks2
# encrypted_partition = luks2(self, None, None)
# encrypted_partition.format(path)
self._partition_info.filesystem_type = filesystem
else:
raise UnknownFilesystemFormat(f"Fileformat '{filesystem}' is not yet implemented.")
except SysCallError as error:
log(f"Formatting ran in to an error: {error}", level=logging.WARNING, fg="orange")
if retry is True:
log(f"Retrying in {storage.get('DISK_TIMEOUTS', 1)} seconds.", level=logging.WARNING, fg="orange")
time.sleep(storage.get('DISK_TIMEOUTS', 1))
return self.format(filesystem, path, log_formatting, options, retry=False)
if get_filesystem_type(path) == 'crypto_LUKS' or get_filesystem_type(self.real_device) == 'crypto_LUKS':
self._encrypted = True
else:
self._encrypted = False
return True
def find_parent_of(self, data :Dict[str, Any], name :str, parent :Optional[str] = None) -> Optional[str]:
if data['name'] == name:
return parent
elif 'children' in data:
for child in data['children']:
if parent := self.find_parent_of(child, name, parent=data['name']):
return parent
return None
def mount(self, target :str, fs :Optional[str] = None, options :str = '') -> bool:
if not self._partition_info.get_first_mountpoint():
log(f'Mounting {self} to {target}', level=logging.INFO)
if not fs:
fs = self._partition_info.filesystem_type
fs_type = get_mount_fs_type(fs)
Path(target).mkdir(parents=True, exist_ok=True)
if self.bind_name:
device_path = self.device_path
# TODO options should be better be a list than a string
if options:
options = f"{options},subvol={self.bind_name}"
else:
options = f"subvol={self.bind_name}"
else:
device_path = self._path
try:
if options:
mnt_handle = SysCommand(f"/usr/bin/mount -t {fs_type} -o {options} {device_path} {target}")
else:
mnt_handle = SysCommand(f"/usr/bin/mount -t {fs_type} {device_path} {target}")
# TODO: Should be redundant to check for exit_code
if mnt_handle.exit_code != 0:
raise DiskError(f"Could not mount {self._path} to {target} using options {options}")
except SysCallError as err:
raise err
# Update the partition info since the mount info has changed after this call.
self._partition_info = self._fetch_information()
return True
return False
def unmount(self) -> bool:
SysCommand(f"/usr/bin/umount {self._path}")
# Update the partition info since the mount info has changed after this call.
self._partition_info = self._fetch_information()
return True
def filesystem_supported(self) -> bool:
"""
The support for a filesystem (this partition) is tested by calling
partition.format() with a path set to '/dev/null' which returns two exceptions:
1. SysCallError saying that /dev/null is not formattable - but the filesystem is supported
2. UnknownFilesystemFormat that indicates that we don't support the given filesystem type
"""
try:
self.format(self._partition_info.filesystem_type, '/dev/null', log_formatting=False)
except (SysCallError, DiskError):
pass # We supported it, but /dev/null is not formattable as expected so the mkfs call exited with an error code
except UnknownFilesystemFormat as err:
raise err
return True
def get_mount_fs_type(fs :str) -> str:
if fs == 'ntfs':
return 'ntfs3' # Needed to use the Paragon R/W NTFS driver
elif fs == 'fat32':
return 'vfat' # This is the actual type used for fat32 mounting
return fs

335
archinstall/lib/disk/partitioning_menu.py Normal file
View File

@ -0,0 +1,335 @@
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Dict, TYPE_CHECKING, List, Optional, Tuple
from .device_model import PartitionModification, FilesystemType, BDevice, Size, Unit, PartitionType, PartitionFlag, \
ModificationStatus
from ..menu import Menu, ListManager, MenuSelection, TextInput
from ..output import FormattedOutput, log
from .subvolume_menu import SubvolumeMenu
if TYPE_CHECKING:
_: Any
class PartitioningList(ListManager):
"""
subclass of ListManager for the managing of user accounts
"""
def __init__(self, prompt: str, device: BDevice, device_partitions: List[PartitionModification]):
self._device = device
self._actions = {
'create_new_partition': str(_('Create a new partition')),
'suggest_partition_layout': str(_('Suggest partition layout')),
'remove_added_partitions': str(_('Remove all newly added partitions')),
'assign_mountpoint': str(_('Assign mountpoint')),
'mark_formatting': str(_('Mark/Unmark to be formatted (wipes data)')),
'mark_bootable': str(_('Mark/Unmark as bootable')),
'set_filesystem': str(_('Change filesystem')),
'btrfs_mark_compressed': str(_('Mark/Unmark as compressed')), # btrfs only
'btrfs_set_subvolumes': str(_('Set subvolumes')), # btrfs only
'delete_partition': str(_('Delete partition'))
}
display_actions = list(self._actions.values())
super().__init__(prompt, device_partitions, display_actions[:2], display_actions[3:])
def reformat(self, data: List[PartitionModification]) -> Dict[str, Optional[PartitionModification]]:
table = FormattedOutput.as_table(data)
rows = table.split('\n')
# these are the header rows of the table and do not map to any User obviously
# we're adding 2 spaces as prefix because the menu selector '> ' will be put before
# the selectable rows so the header has to be aligned
display_data: Dict[str, Optional[PartitionModification]] = {f' {rows[0]}': None, f' {rows[1]}': None}
for row, user in zip(rows[2:], data):
row = row.replace('|', '\\|')
display_data[row] = user
return display_data
def selected_action_display(self, partition: PartitionModification) -> str:
return str(_('Partition'))
def filter_options(self, selection: PartitionModification, options: List[str]) -> List[str]:
not_filter = []
# only display formatting if the partition exists already
if not selection.exists():
not_filter += [self._actions['mark_formatting']]
else:
# only allow these options if the existing partition
# was marked as formatting, otherwise we run into issues where
# 1. select a new fs -> potentially mark as wipe now
# 2. Switch back to old filesystem -> should unmark wipe now, but
# how do we know it was the original one?
not_filter += [
self._actions['set_filesystem'],
self._actions['assign_mountpoint'],
self._actions['mark_bootable'],
self._actions['btrfs_mark_compressed'],
self._actions['btrfs_set_subvolumes']
]
# non btrfs partitions shouldn't get btrfs options
if selection.fs_type != FilesystemType.Btrfs:
not_filter += [self._actions['btrfs_mark_compressed'], self._actions['btrfs_set_subvolumes']]
else:
not_filter += [self._actions['assign_mountpoint']]
return [o for o in options if o not in not_filter]
def handle_action(
self,
action: str,
entry: Optional[PartitionModification],
data: List[PartitionModification]
) -> List[PartitionModification]:
action_key = [k for k, v in self._actions.items() if v == action][0]
match action_key:
case 'create_new_partition':
new_partition = self._create_new_partition()
data += [new_partition]
case 'suggest_partition_layout':
new_partitions = self._suggest_partition_layout(data)
if len(new_partitions) > 0:
data = new_partitions
case 'remove_added_partitions':
choice = self._reset_confirmation()
if choice.value == Menu.yes():
data = [part for part in data if part.is_exists_or_modify()]
case 'assign_mountpoint' if entry:
entry.mountpoint = self._prompt_mountpoint()
if entry.mountpoint == Path('/boot'):
entry.set_flag(PartitionFlag.Boot)
case 'mark_formatting' if entry:
self._prompt_formatting(entry)
case 'mark_bootable' if entry:
entry.invert_flag(PartitionFlag.Boot)
case 'set_filesystem' if entry:
fs_type = self._prompt_partition_fs_type()
if fs_type:
entry.fs_type = fs_type
# btrfs subvolumes will define mountpoints
if fs_type == FilesystemType.Btrfs:
entry.mountpoint = None
case 'btrfs_mark_compressed' if entry:
self._set_compressed(entry)
case 'btrfs_set_subvolumes' if entry:
self._set_btrfs_subvolumes(entry)
case 'delete_partition' if entry:
data = self._delete_partition(entry, data)
return data
def _delete_partition(
self,
entry: PartitionModification,
data: List[PartitionModification]
) -> List[PartitionModification]:
if entry.is_exists_or_modify():
entry.status = ModificationStatus.Delete
return data
else:
return [d for d in data if d != entry]
def _set_compressed(self, partition: PartitionModification):
compression = 'compress=zstd'
if compression in partition.mount_options:
partition.mount_options = [o for o in partition.mount_options if o != compression]
else:
partition.mount_options.append(compression)
def _set_btrfs_subvolumes(self, partition: PartitionModification):
partition.btrfs_subvols = SubvolumeMenu(
_("Manage btrfs subvolumes for current partition"),
partition.btrfs_subvols
).run()
def _prompt_formatting(self, partition: PartitionModification):
# an existing partition can toggle between Exist or Modify
if partition.is_modify():
partition.status = ModificationStatus.Exist
return
elif partition.exists():
partition.status = ModificationStatus.Modify
# If we mark a partition for formatting, but the format is CRYPTO LUKS, there's no point in formatting it really
# without asking the user which inner-filesystem they want to use. Since the flag 'encrypted' = True is already set,
# it's safe to change the filesystem for this partition.
if partition.fs_type == FilesystemType.Crypto_luks:
prompt = str(_('This partition is currently encrypted, to format it a filesystem has to be specified'))
fs_type = self._prompt_partition_fs_type(prompt)
partition.fs_type = fs_type
if fs_type == FilesystemType.Btrfs:
partition.mountpoint = None
def _prompt_mountpoint(self) -> Path:
header = str(_('Partition mount-points are relative to inside the installation, the boot would be /boot as an example.')) + '\n'
header += str(_('If mountpoint /boot is set, then the partition will also be marked as bootable.')) + '\n'
prompt = str(_('Mountpoint: '))
print(header)
while True:
value = TextInput(prompt).run().strip()
if value:
mountpoint = Path(value)
break
return mountpoint
def _prompt_partition_fs_type(self, prompt: str = '') -> FilesystemType:
options = {fs.value: fs for fs in FilesystemType if fs != FilesystemType.Crypto_luks}
prompt = prompt + '\n' + str(_('Enter a desired filesystem type for the partition'))
choice = Menu(prompt, options, sort=False, skip=False).run()
return options[choice.single_value]
def _validate_sector(self, start_sector: str, end_sector: Optional[str] = None) -> bool:
if not start_sector.isdigit():
return False
if end_sector:
if end_sector.endswith('%'):
if not end_sector[:-1].isdigit():
return False
elif not end_sector.isdigit():
return False
elif int(start_sector) > int(end_sector):
return False
return True
def _prompt_sectors(self) -> Tuple[Size, Size]:
device_info = self._device.device_info
text = str(_('Current free sectors on device {}:')).format(device_info.path) + '\n\n'
free_space_table = FormattedOutput.as_table(device_info.free_space_regions)
prompt = text + free_space_table + '\n'
total_sectors = device_info.total_size.format_size(Unit.sectors, device_info.sector_size)
prompt += str(_('Total sectors: {}')).format(total_sectors) + '\n'
print(prompt)
largest_free_area = max(device_info.free_space_regions, key=lambda r: r.get_length())
# prompt until a valid start sector was entered
while True:
start_prompt = str(_('Enter the start sector (default: {}): ')).format(largest_free_area.start)
start_sector = TextInput(start_prompt).run().strip()
if not start_sector or self._validate_sector(start_sector):
break
log(f'Invalid start sector entered: {start_sector}', fg='red', level=logging.INFO)
if not start_sector:
start_sector = str(largest_free_area.start)
end_sector = str(largest_free_area.end)
else:
end_sector = '100%'
# prompt until valid end sector was entered
while True:
end_prompt = str(_('Enter the end sector of the partition (percentage or block number, default: {}): ')).format(end_sector)
end_value = TextInput(end_prompt).run().strip()
if not end_value or self._validate_sector(start_sector, end_value):
break
log(f'Invalid end sector entered: {start_sector}', fg='red', level=logging.INFO)
# override the default value with the user value
if end_value:
end_sector = end_value
start_size = Size(int(start_sector), Unit.sectors, device_info.sector_size)
if end_sector.endswith('%'):
end_size = Size(int(end_sector[:-1]), Unit.Percent, device_info.sector_size, device_info.total_size)
else:
end_size = Size(int(end_sector), Unit.sectors, device_info.sector_size)
return start_size, end_size
def _create_new_partition(self) -> PartitionModification:
fs_type = self._prompt_partition_fs_type()
start_size, end_size = self._prompt_sectors()
length = end_size - start_size
# new line for the next prompt
print()
mountpoint = None
if fs_type != FilesystemType.Btrfs:
mountpoint = self._prompt_mountpoint()
partition = PartitionModification(
status=ModificationStatus.Create,
type=PartitionType.Primary,
start=start_size,
length=length,
fs_type=fs_type,
mountpoint=mountpoint
)
if partition.mountpoint == Path('/boot'):
partition.set_flag(PartitionFlag.Boot)
return partition
def _reset_confirmation(self) -> MenuSelection:
prompt = str(_('This will remove all newly added partitions, continue?'))
choice = Menu(prompt, Menu.yes_no(), default_option=Menu.no(), skip=False).run()
return choice
def _suggest_partition_layout(self, data: List[PartitionModification]) -> List[PartitionModification]:
# if modifications have been done already, inform the user
# that this operation will erase those modifications
if any([not entry.exists() for entry in data]):
choice = self._reset_confirmation()
if choice.value == Menu.no():
return []
from ..user_interaction.disk_conf import suggest_single_disk_layout
device_modification = suggest_single_disk_layout(self._device)
return device_modification.partitions
def manual_partitioning(
device: BDevice,
prompt: str = '',
preset: List[PartitionModification] = []
) -> List[PartitionModification]:
if not prompt:
prompt = str(_('Partition management: {}')).format(device.device_info.path) + '\n'
prompt += str(_('Total length: {}')).format(device.device_info.total_size.format_size(Unit.MiB))
manual_preset = []
if not preset:
# we'll display the existing partitions of the device
for partition in device.partition_infos:
manual_preset.append(
PartitionModification.from_existing_partition(partition)
)
else:
manual_preset = preset
menu_list = PartitioningList(prompt, device, manual_preset)
partitions = menu_list.run()
if menu_list.is_last_choice_cancel():
return preset
return partitions

43
archinstall/lib/user_interaction/subvolume_config.py → archinstall/lib/disk/subvolume_menu.py
View File

@ -1,33 +1,31 @@
from pathlib import Path
from typing import Dict, List, Optional, Any, TYPE_CHECKING
from ..menu.list_manager import ListManager
from ..menu.menu import MenuSelectionType
from ..menu.text_input import TextInput
from ..menu import Menu
from ..models.subvolume import Subvolume
from ... import FormattedOutput
from .device_model import SubvolumeModification
from ..menu import Menu, TextInput, MenuSelectionType, ListManager
from ..output import FormattedOutput
if TYPE_CHECKING:
_: Any
class SubvolumeList(ListManager):
def __init__(self, prompt: str, subvolumes: List[Subvolume]):
class SubvolumeMenu(ListManager):
def __init__(self, prompt: str, btrfs_subvols: List[SubvolumeModification]):
self._actions = [
str(_('Add subvolume')),
str(_('Edit subvolume')),
str(_('Delete subvolume'))
]
super().__init__(prompt, subvolumes, [self._actions[0]], self._actions[1:])
super().__init__(prompt, btrfs_subvols, [self._actions[0]], self._actions[1:])
def reformat(self, data: List[Subvolume]) -> Dict[str, Optional[Subvolume]]:
def reformat(self, data: List[SubvolumeModification]) -> Dict[str, Optional[SubvolumeModification]]:
table = FormattedOutput.as_table(data)
rows = table.split('\n')
# these are the header rows of the table and do not map to any User obviously
# we're adding 2 spaces as prefix because the menu selector '> ' will be put before
# the selectable rows so the header has to be aligned
display_data: Dict[str, Optional[Subvolume]] = {f' {rows[0]}': None, f' {rows[1]}': None}
display_data: Dict[str, Optional[SubvolumeModification]] = {f' {rows[0]}': None, f' {rows[1]}': None}
for row, subvol in zip(rows[2:], data):
row = row.replace('|', '\\|')
@ -35,17 +33,17 @@ class SubvolumeList(ListManager):
return display_data
def selected_action_display(self, subvolume: Subvolume) -> str:
return subvolume.name
def selected_action_display(self, subvolume: SubvolumeModification) -> str:
return str(subvolume.name)
def _prompt_options(self, editing: Optional[Subvolume] = None) -> List[str]:
def _prompt_options(self, editing: Optional[SubvolumeModification] = None) -> List[str]:
preset_options = []
if editing:
preset_options = editing.options
preset_options = editing.mount_options
choice = Menu(
str(_("Select the desired subvolume options ")),
['nodatacow','compress'],
['nodatacow', 'compress'],
skip=True,
preset_values=preset_options,
multi=True
@ -56,26 +54,31 @@ class SubvolumeList(ListManager):
return []
def _add_subvolume(self, editing: Optional[Subvolume] = None) -> Optional[Subvolume]:
def _add_subvolume(self, editing: Optional[SubvolumeModification] = None) -> Optional[SubvolumeModification]:
name = TextInput(f'\n\n{_("Subvolume name")}: ', editing.name if editing else '').run()
if not name:
return None
mountpoint = TextInput(f'\n{_("Subvolume mountpoint")}: ', editing.mountpoint if editing else '').run()
mountpoint = TextInput(f'{_("Subvolume mountpoint")}: ', str(editing.mountpoint) if editing else '').run()
if not mountpoint:
return None
options = self._prompt_options(editing)
subvolume = Subvolume(name, mountpoint)
subvolume = SubvolumeModification(Path(name), Path(mountpoint))
subvolume.compress = 'compress' in options
subvolume.nodatacow = 'nodatacow' in options
return subvolume
def handle_action(self, action: str, entry: Optional[Subvolume], data: List[Subvolume]) -> List[Subvolume]:
def handle_action(
self,
action: str,
entry: Optional[SubvolumeModification],
data: List[SubvolumeModification]
) -> List[SubvolumeModification]:
if action == self._actions[0]: # add
new_subvolume = self._add_subvolume()

240
archinstall/lib/disk/user_guides.py
View File

@ -1,240 +0,0 @@
from __future__ import annotations
import logging
from typing import Optional, Dict, Any, List, TYPE_CHECKING
# https://stackoverflow.com/a/39757388/929999
from ..models.subvolume import Subvolume
if TYPE_CHECKING:
from .blockdevice import BlockDevice
_: Any
from .helpers import sort_block_devices_based_on_performance, select_largest_device, select_disk_larger_than_or_close_to
from ..hardware import has_uefi
from ..output import log
from ..menu import Menu
def suggest_single_disk_layout(block_device :BlockDevice,
default_filesystem :Optional[str] = None,
advanced_options :bool = False) -> Dict[str, Any]:
if not default_filesystem:
from ..user_interaction import ask_for_main_filesystem_format
default_filesystem = ask_for_main_filesystem_format(advanced_options)
MIN_SIZE_TO_ALLOW_HOME_PART = 40 # GiB
using_subvolumes = False
using_home_partition = False
compression = False
if default_filesystem == 'btrfs':
prompt = str(_('Would you like to use BTRFS subvolumes with a default structure?'))
choice = Menu(prompt, Menu.yes_no(), skip=False, default_option=Menu.yes()).run()
using_subvolumes = choice.value == Menu.yes()
prompt = str(_('Would you like to use BTRFS compression?'))
choice = Menu(prompt, Menu.yes_no(), skip=False, default_option=Menu.yes()).run()
compression = choice.value == Menu.yes()
layout = {
block_device.path : {
"wipe" : True,
"partitions" : []
}
}
# Used for reference: https://wiki.archlinux.org/title/partitioning
# 2 MiB is unallocated for GRUB on BIOS. Potentially unneeded for
# other bootloaders?
# TODO: On BIOS, /boot partition is only needed if the drive will
# be encrypted, otherwise it is not recommended. We should probably
# add a check for whether the drive will be encrypted or not.
layout[block_device.path]['partitions'].append({
# Boot
"type" : "primary",
"start" : "3MiB",
"size" : "203MiB",
"boot" : True,
"encrypted" : False,
"wipe" : True,
"mountpoint" : "/boot",
"filesystem" : {
"format" : "fat32"
}
})
# Increase the UEFI partition if UEFI is detected.
# Also re-align the start to 1MiB since we don't need the first sectors
# like we do in MBR layouts where the boot loader is installed traditionally.
if has_uefi():
layout[block_device.path]['partitions'][-1]['start'] = '1MiB'
layout[block_device.path]['partitions'][-1]['size'] = '512MiB'
layout[block_device.path]['partitions'].append({
# Root
"type" : "primary",
"start" : "206MiB",
"encrypted" : False,
"wipe" : True,
"mountpoint" : "/" if not using_subvolumes else None,
"filesystem" : {
"format" : default_filesystem,
"mount_options" : ["compress=zstd"] if compression else []
}
})
if has_uefi():
layout[block_device.path]['partitions'][-1]['start'] = '513MiB'
if not using_subvolumes and block_device.size >= MIN_SIZE_TO_ALLOW_HOME_PART:
prompt = str(_('Would you like to create a separate partition for /home?'))
choice = Menu(prompt, Menu.yes_no(), skip=False, default_option=Menu.yes()).run()
using_home_partition = choice.value == Menu.yes()
# Set a size for / (/root)
if using_subvolumes or block_device.size < MIN_SIZE_TO_ALLOW_HOME_PART or not using_home_partition:
# We'll use subvolumes
# Or the disk size is too small to allow for a separate /home
# Or the user doesn't want to create a separate partition for /home
layout[block_device.path]['partitions'][-1]['size'] = '100%'
else:
layout[block_device.path]['partitions'][-1]['size'] = f"{min(block_device.size, 20)}GiB"
if default_filesystem == 'btrfs' and using_subvolumes:
# if input('Do you want to use a recommended structure? (Y/n): ').strip().lower() in ('', 'y', 'yes'):
# https://btrfs.wiki.kernel.org/index.php/FAQ
# https://unix.stackexchange.com/questions/246976/btrfs-subvolume-uuid-clash
# https://github.com/classy-giraffe/easy-arch/blob/main/easy-arch.sh
layout[block_device.path]['partitions'][1]['btrfs'] = {
'subvolumes': [
Subvolume('@', '/'),
Subvolume('@home', '/home'),
Subvolume('@log', '/var/log'),
Subvolume('@pkg', '/var/cache/pacman/pkg'),
Subvolume('@.snapshots', '/.snapshots')
]
}
elif using_home_partition:
# If we don't want to use subvolumes,
# But we want to be able to re-use data between re-installs..
# A second partition for /home would be nice if we have the space for it
layout[block_device.path]['partitions'].append({
# Home
"type" : "primary",
"start" : f"{min(block_device.size, 20)}GiB",
"size" : "100%",
"encrypted" : False,
"wipe" : True,
"mountpoint" : "/home",
"filesystem" : {
"format" : default_filesystem,
"mount_options" : ["compress=zstd"] if compression else []
}
})
return layout
def suggest_multi_disk_layout(block_devices :List[BlockDevice], default_filesystem :Optional[str] = None, advanced_options :bool = False):
if not default_filesystem:
from ..user_interaction import ask_for_main_filesystem_format
default_filesystem = ask_for_main_filesystem_format(advanced_options)
# Not really a rock solid foundation of information to stand on, but it's a start:
# https://www.reddit.com/r/btrfs/comments/m287gp/partition_strategy_for_two_physical_disks/
# https://www.reddit.com/r/btrfs/comments/9us4hr/what_is_your_btrfs_partitionsubvolumes_scheme/
MIN_SIZE_TO_ALLOW_HOME_PART = 40 # GiB
ARCH_LINUX_INSTALLED_SIZE = 20 # GiB, rough estimate taking in to account user desktops etc. TODO: Catch user packages to detect size?
block_devices = sort_block_devices_based_on_performance(block_devices).keys()
home_device = select_largest_device(block_devices, gigabytes=MIN_SIZE_TO_ALLOW_HOME_PART)
root_device = select_disk_larger_than_or_close_to(block_devices, gigabytes=ARCH_LINUX_INSTALLED_SIZE, filter_out=[home_device])
if home_device is None or root_device is None:
text = _('The selected drives do not have the minimum capacity required for an automatic suggestion\n')
text += _('Minimum capacity for /home partition: {}GB\n').format(MIN_SIZE_TO_ALLOW_HOME_PART)
text += _('Minimum capacity for Arch Linux partition: {}GB').format(ARCH_LINUX_INSTALLED_SIZE)
Menu(str(text), [str(_('Continue'))], skip=False).run()
return None
compression = False
if default_filesystem == 'btrfs':
# prompt = 'Would you like to use BTRFS subvolumes with a default structure?'
# choice = Menu(prompt, ['yes', 'no'], skip=False, default_option='yes').run()
# using_subvolumes = choice == 'yes'
prompt = str(_('Would you like to use BTRFS compression?'))
choice = Menu(prompt, Menu.yes_no(), skip=False, default_option=Menu.yes()).run()
compression = choice.value == Menu.yes()
log(f"Suggesting multi-disk-layout using {len(block_devices)} disks, where {root_device} will be /root and {home_device} will be /home", level=logging.DEBUG)
layout = {
root_device.path : {
"wipe" : True,
"partitions" : []
},
home_device.path : {
"wipe" : True,
"partitions" : []
},
}
# TODO: Same deal as with the single disk layout, we should
# probably check if the drive will be encrypted.
layout[root_device.path]['partitions'].append({
# Boot
"type" : "primary",
"start" : "3MiB",
"size" : "203MiB",
"boot" : True,
"encrypted" : False,
"wipe" : True,
"mountpoint" : "/boot",
"filesystem" : {
"format" : "fat32"
}
})
if has_uefi():
layout[root_device.path]['partitions'][-1]['start'] = '1MiB'
layout[root_device.path]['partitions'][-1]['size'] = '512MiB'
layout[root_device.path]['partitions'].append({
# Root
"type" : "primary",
"start" : "206MiB",
"size" : "100%",
"encrypted" : False,
"wipe" : True,
"mountpoint" : "/",
"filesystem" : {
"format" : default_filesystem,
"mount_options" : ["compress=zstd"] if compression else []
}
})
if has_uefi():
layout[root_device.path]['partitions'][-1]['start'] = '513MiB'
layout[home_device.path]['partitions'].append({
# Home
"type" : "primary",
"start" : "1MiB",
"size" : "100%",
"encrypted" : False,
"wipe" : True,
"mountpoint" : "/home",
"filesystem" : {
"format" : default_filesystem,
"mount_options" : ["compress=zstd"] if compression else []
}
})
return layout

48
archinstall/lib/disk/validators.py
View File

@ -1,48 +0,0 @@
from typing import List
def valid_parted_position(pos :str) -> bool:
if not len(pos):
return False
if pos.isdigit():
return True
pos_lower = pos.lower()
if (pos_lower.endswith('b') or pos_lower.endswith('s')) and pos[:-1].isdigit():
return True
if any(pos_lower.endswith(size) and pos[:-len(size)].replace(".", "", 1).isdigit()
for size in ['%', 'kb', 'mb', 'gb', 'tb', 'kib', 'mib', 'gib', 'tib']):
return True
return False
def fs_types() -> List[str]:
# https://www.gnu.org/software/parted/manual/html_node/mkpart.html
# Above link doesn't agree with `man parted` /mkpart documentation:
"""
fs-type can
be one of "btrfs", "ext2",
"ext3", "ext4", "fat16",
"fat32", "hfs", "hfs+",
"linux-swap", "ntfs", "reis‐
erfs", "udf", or "xfs".
"""
return [
"btrfs",
"ext2",
"ext3", "ext4", # `man parted` allows these
"fat16", "fat32",
"hfs", "hfs+", # "hfsx", not included in `man parted`
"linux-swap",
"ntfs",
"reiserfs",
"udf", # "ufs", not included in `man parted`
"xfs", # `man parted` allows this
]
def valid_fs_type(fstype :str) -> bool:
return fstype.lower() in fs_types()

31
archinstall/lib/general.py
View File

@ -1,4 +1,5 @@
from __future__ import annotations
import hashlib
import json
import logging
@ -17,7 +18,7 @@ import urllib.error
import pathlib
from datetime import datetime, date
from typing import Callable, Optional, Dict, Any, List, Union, Iterator, TYPE_CHECKING
# https://stackoverflow.com/a/39757388/929999
if TYPE_CHECKING:
from .installer import Installer
@ -140,7 +141,7 @@ class JsonEncoder:
return obj.isoformat()
elif isinstance(obj, (list, set, tuple)):
return [json.loads(json.dumps(item, cls=JSON)) for item in obj]
elif isinstance(obj, (pathlib.Path)):
elif isinstance(obj, pathlib.Path):
return str(obj)
else:
return obj
@ -184,22 +185,21 @@ class UNSAFE_JSON(json.JSONEncoder, json.JSONDecoder):
def encode(self, obj :Any) -> Any:
return super(UNSAFE_JSON, self).encode(self._encode(obj))
class SysCommandWorker:
def __init__(self,
def __init__(
self,
cmd :Union[str, List[str]],
callbacks :Optional[Dict[str, Any]] = None,
peek_output :Optional[bool] = False,
peak_output :Optional[bool] = False,
environment_vars :Optional[Dict[str, Any]] = None,
logfile :Optional[None] = None,
working_directory :Optional[str] = './',
remove_vt100_escape_codes_from_lines :bool = True):
if peak_output:
log("SysCommandWorker()'s peak_output is deprecated, use peek_output instead.", level=logging.WARNING, fg='red')
remove_vt100_escape_codes_from_lines :bool = True
):
if not callbacks:
callbacks = {}
if not environment_vars:
environment_vars = {}
@ -216,8 +216,6 @@ class SysCommandWorker:
self.cmd = cmd
self.callbacks = callbacks
self.peek_output = peek_output
if not self.peek_output and peak_output:
self.peek_output = peak_output
# define the standard locale for command outputs. For now the C ascii one. Can be overridden
self.environment_vars = {**storage.get('CMD_LOCALE',{}),**environment_vars}
self.logfile = logfile
@ -396,7 +394,7 @@ class SysCommandWorker:
os.chmod(str(history_logfile), stat.S_IRUSR | stat.S_IWUSR | stat.S_IRGRP)
except PermissionError:
pass
# If history_logfile does not exist, ignore the error
# If history_logfile does not exist, ignore the error
except FileNotFoundError:
pass
except Exception as e:
@ -431,14 +429,10 @@ class SysCommand:
callbacks :Optional[Dict[str, Callable[[Any], Any]]] = None,
start_callback :Optional[Callable[[Any], Any]] = None,
peek_output :Optional[bool] = False,
peak_output :Optional[bool] = False,
environment_vars :Optional[Dict[str, Any]] = None,
working_directory :Optional[str] = './',
remove_vt100_escape_codes_from_lines :bool = True):
if peak_output:
log("SysCommandWorker()'s peak_output is deprecated, use peek_output instead.", level=logging.WARNING, fg='red')
_callbacks = {}
if callbacks:
for hook, func in callbacks.items():
@ -449,8 +443,6 @@ class SysCommand:
self.cmd = cmd
self._callbacks = _callbacks
self.peek_output = peek_output
if not self.peek_output and peak_output:
self.peek_output = peak_output
self.environment_vars = environment_vars
self.working_directory = working_directory
self.remove_vt100_escape_codes_from_lines = remove_vt100_escape_codes_from_lines
@ -575,9 +567,8 @@ def run_custom_user_commands(commands :List[str], installation :Installer) -> No
with open(f"{installation.target}/var/tmp/user-command.{index}.sh", "w") as temp_script:
temp_script.write(command)
execution_output = SysCommand(f"arch-chroot {installation.target} bash /var/tmp/user-command.{index}.sh")
SysCommand(f"arch-chroot {installation.target} bash /var/tmp/user-command.{index}.sh")
log(execution_output)
os.unlink(f"{installation.target}/var/tmp/user-command.{index}.sh")
def json_stream_to_structure(configuration_identifier : str, stream :str, target :dict) -> bool :

364
archinstall/lib/global_menu.py Normal file
View File

@ -0,0 +1,364 @@
from __future__ import annotations
from typing import Any, List, Optional, Union, Dict, TYPE_CHECKING
from . import disk
from .general import SysCommand, secret
from .menu import Selector, AbstractMenu
from .models import NetworkConfiguration
from .models.bootloader import Bootloader
from .models.users import User
from .output import FormattedOutput
from .profile.profile_menu import ProfileConfiguration
from .storage import storage
from .user_interaction import add_number_of_parrallel_downloads
from .user_interaction import ask_additional_packages_to_install
from .user_interaction import ask_for_additional_users
from .user_interaction import ask_for_audio_selection
from .user_interaction import ask_for_bootloader
from .user_interaction import ask_for_swap
from .user_interaction import ask_hostname
from .user_interaction import ask_ntp
from .user_interaction import ask_to_configure_network
from .user_interaction import get_password, ask_for_a_timezone
from .user_interaction import select_additional_repositories
from .user_interaction import select_kernel
from .user_interaction import select_language
from .user_interaction import select_locale_enc
from .user_interaction import select_locale_lang
from .user_interaction import select_mirror_regions
from .user_interaction.disk_conf import select_disk_config
from .user_interaction.save_conf import save_config
if TYPE_CHECKING:
_: Any
class GlobalMenu(AbstractMenu):
def __init__(self, data_store: Dict[str, Any]):
super().__init__(data_store=data_store, auto_cursor=True, preview_size=0.3)
def setup_selection_menu_options(self):
# archinstall.Language will not use preset values
self._menu_options['archinstall-language'] = \
Selector(
_('Archinstall language'),
lambda x: self._select_archinstall_language(x),
display_func=lambda x: x.display_name,
default=self.translation_handler.get_language_by_abbr('en'))
self._menu_options['keyboard-layout'] = \
Selector(
_('Keyboard layout'),
lambda preset: select_language(preset),
default='us')
self._menu_options['mirror-region'] = \
Selector(
_('Mirror region'),
lambda preset: select_mirror_regions(preset),
display_func=lambda x: list(x.keys()) if x else '[]',
default={})
self._menu_options['sys-language'] = \
Selector(
_('Locale language'),
lambda preset: select_locale_lang(preset),
default='en_US')
self._menu_options['sys-encoding'] = \
Selector(
_('Locale encoding'),
lambda preset: select_locale_enc(preset),
default='UTF-8')
self._menu_options['disk_config'] = \
Selector(
_('Disk configuration'),
lambda preset: self._select_disk_config(preset),
preview_func=self._prev_disk_layouts,
display_func=lambda x: self._display_disk_layout(x),
)
self._menu_options['disk_encryption'] = \
Selector(
_('Disk encryption'),
lambda preset: self._disk_encryption(preset),
preview_func=self._prev_disk_encryption,
display_func=lambda x: self._display_disk_encryption(x),
dependencies=['disk_config'])
self._menu_options['swap'] = \
Selector(
_('Swap'),
lambda preset: ask_for_swap(preset),
default=True)
self._menu_options['bootloader'] = \
Selector(
_('Bootloader'),
lambda preset: ask_for_bootloader(preset),
display_func=lambda x: x.value,
default=Bootloader.get_default())
self._menu_options['hostname'] = \
Selector(
_('Hostname'),
lambda preset: ask_hostname(preset),
default='archlinux')
# root password won't have preset value
self._menu_options['!root-password'] = \
Selector(
_('Root password'),
lambda preset:self._set_root_password(),
display_func=lambda x: secret(x) if x else 'None')
self._menu_options['!users'] = \
Selector(
_('User account'),
lambda x: self._create_user_account(x),
default={},
display_func=lambda x: f'{len(x)} {_("User(s)")}' if len(x) > 0 else None,
preview_func=self._prev_users)
self._menu_options['profile_config'] = \
Selector(
_('Profile'),
lambda preset: self._select_profile(preset),
display_func=lambda x: x.profile.name if x else 'None',
preview_func=self._prev_profile
)
self._menu_options['audio'] = \
Selector(
_('Audio'),
lambda preset: self._select_audio(preset),
display_func=lambda x: x if x else 'None',
default=None
)
self._menu_options['parallel downloads'] = \
Selector(
_('Parallel Downloads'),
add_number_of_parrallel_downloads,
display_func=lambda x: x if x else '0',
default=0
)
self._menu_options['kernels'] = \
Selector(
_('Kernels'),
lambda preset: select_kernel(preset),
display_func=lambda x: ', '.join(x) if x else None,
default=['linux'])
self._menu_options['packages'] = \
Selector(
_('Additional packages'),
# lambda x: ask_additional_packages_to_install(storage['arguments'].get('packages', None)),
ask_additional_packages_to_install,
default=[])
self._menu_options['additional-repositories'] = \
Selector(
_('Optional repositories'),
select_additional_repositories,
display_func=lambda x: ', '.join(x) if x else None,
default=[])
self._menu_options['nic'] = \
Selector(
_('Network configuration'),
ask_to_configure_network,
display_func=lambda x: self._display_network_conf(x),
preview_func=self._prev_network_config,
default={})
self._menu_options['timezone'] = \
Selector(
_('Timezone'),
lambda preset: ask_for_a_timezone(preset),
default='UTC')
self._menu_options['ntp'] = \
Selector(
_('Automatic time sync (NTP)'),
lambda preset: self._select_ntp(preset),
default=True)
self._menu_options['__separator__'] = \
Selector('')
self._menu_options['save_config'] = \
Selector(
_('Save configuration'),
lambda preset: save_config(self._data_store),
no_store=True)
self._menu_options['install'] = \
Selector(
self._install_text(),
exec_func=lambda n,v: True if len(self._missing_configs()) == 0 else False,
preview_func=self._prev_install_missing_config,
no_store=True)
self._menu_options['abort'] = Selector(_('Abort'), exec_func=lambda n,v:exit(1))
def _update_install_text(self, name: str, value: str):
text = self._install_text()
self._menu_options['install'].update_description(text)
def post_callback(self, name: str, value: str):
self._update_install_text(name, value)
def _install_text(self):
missing = len(self._missing_configs())
if missing > 0:
return _('Install ({} config(s) missing)').format(missing)
return _('Install')
def _display_network_conf(self, cur_value: Union[NetworkConfiguration, List[NetworkConfiguration]]) -> str:
if not cur_value:
return _('Not configured, unavailable unless setup manually')
else:
if isinstance(cur_value, list):
return str(_('Configured {} interfaces')).format(len(cur_value))
else:
return str(cur_value)
def _disk_encryption(self, preset: Optional[disk.DiskEncryption]) -> Optional[disk.DiskEncryption]:
mods: Optional[List[disk.DeviceModification]] = self._menu_options['disk_config'].current_selection
if not mods:
# this should not happen as the encryption menu has the disk_config as dependency
raise ValueError('No disk layout specified')
data_store: Dict[str, Any] = {}
disk_encryption = disk.DiskEncryptionMenu(mods, data_store, preset=preset).run()
return disk_encryption
def _prev_network_config(self) -> Optional[str]:
selector = self._menu_options['nic']
if selector.has_selection():
ifaces = selector.current_selection
if isinstance(ifaces, list):
return FormattedOutput.as_table(ifaces)
return None
def _prev_disk_layouts(self) -> Optional[str]:
selector = self._menu_options['disk_config']
disk_layout_conf: Optional[disk.DiskLayoutConfiguration] = selector.current_selection
if disk_layout_conf:
device_mods: List[disk.DeviceModification] = \
list(filter(lambda x: len(x.partitions) > 0, disk_layout_conf.device_modifications))
if device_mods:
output_partition = '{}: {}\n'.format(str(_('Configuration')), disk_layout_conf.config_type.display_msg())
output_btrfs = ''
for mod in device_mods:
# create partition table
partition_table = FormattedOutput.as_table(mod.partitions)
output_partition += f'{mod.device_path}: {mod.device.device_info.model}\n'
output_partition += partition_table + '\n'
# create btrfs table
btrfs_partitions = list(
filter(lambda p: len(p.btrfs_subvols) > 0, mod.partitions)
)
for partition in btrfs_partitions:
output_btrfs += FormattedOutput.as_table(partition.btrfs_subvols) + '\n'
output = output_partition + output_btrfs
return output.rstrip()
return None
def _display_disk_layout(self, current_value: Optional[disk.DiskLayoutConfiguration] = None) -> str:
if current_value:
return current_value.config_type.display_msg()
return ''
def _prev_disk_encryption(self) -> Optional[str]:
encryption: Optional[disk.DiskEncryption] = self._menu_options['disk_encryption'].current_selection
if encryption:
enc_type = disk.EncryptionType.type_to_text(encryption.encryption_type)
output = str(_('Encryption type')) + f': {enc_type}\n'
output += str(_('Password')) + f': {secret(encryption.encryption_password)}\n'
if encryption.partitions:
output += 'Partitions: {} selected'.format(len(encryption.partitions)) + '\n'
if encryption.hsm_device:
output += f'HSM: {encryption.hsm_device.manufacturer}'
return output
return None
def _display_disk_encryption(self, current_value: Optional[disk.DiskEncryption]) -> str:
if current_value:
return disk.EncryptionType.type_to_text(current_value.encryption_type)
return ''
def _prev_install_missing_config(self) -> Optional[str]:
if missing := self._missing_configs():
text = str(_('Missing configurations:\n'))
for m in missing:
text += f'- {m}\n'
return text[:-1] # remove last new line
return None
def _prev_users(self) -> Optional[str]:
selector = self._menu_options['!users']
users: Optional[List[User]] = selector.current_selection
if users:
return FormattedOutput.as_table(users)
return None
def _prev_profile(self) -> Optional[str]:
selector = self._menu_options['profile_config']
profile_config: Optional[ProfileConfiguration] = selector.current_selection
if profile_config and profile_config.profile:
output = str(_('Profiles')) + ': '
if profile_names := profile_config.profile.current_selection_names():
output += ', '.join(profile_names) + '\n'
else:
output += profile_config.profile.name + '\n'
if profile_config.gfx_driver:
output += str(_('Graphics driver')) + ': ' + profile_config.gfx_driver + '\n'
if profile_config.greeter:
output += str(_('Greeter')) + ': ' + profile_config.greeter.value + '\n'
return output
return None
def _set_root_password(self) -> Optional[str]:
prompt = str(_('Enter root password (leave blank to disable root): '))
password = get_password(prompt=prompt)
return password
def _select_ntp(self, preset :bool = True) -> bool:
ntp = ask_ntp(preset)
value = str(ntp).lower()
SysCommand(f'timedatectl set-ntp {value}')
return ntp
def _select_disk_config(
self,
preset: Optional[disk.DiskLayoutConfiguration] = None
) -> Optional[disk.DiskLayoutConfiguration]:
disk_config = select_disk_config(
preset,
storage['arguments'].get('advanced', False)
)
if disk_config != preset:
self._menu_options['disk_encryption'].set_current_selection(None)
return disk_config
def _select_profile(self, current_profile: Optional[ProfileConfiguration]):
from .profile.profile_menu import ProfileMenu
store: Dict[str, Any] = {}
profile_config = ProfileMenu(store, preset=current_profile).run()
return profile_config
def _select_audio(self, current: Union[str, None]) -> Optional[str]:
profile_config: Optional[ProfileConfiguration] = self._menu_options['profile_config'].current_selection
if profile_config and profile_config.profile:
is_desktop = profile_config.profile.is_desktop_profile() if profile_config else False
selection = ask_for_audio_selection(is_desktop, current)
return selection
return None
def _create_user_account(self, defined_users: List[User]) -> List[User]:
users = ask_for_additional_users(defined_users=defined_users)
return users

1
archinstall/lib/hsm/__init__.py
View File

@ -1 +0,0 @@
from .fido import Fido2

676
archinstall/lib/installer.py
View File

@ -1,30 +1,29 @@
import time
import glob
import logging
import os
import re
import shutil
import shlex
import pathlib
import shutil
import subprocess
import glob
from types import ModuleType
from typing import Union, Dict, Any, List, Optional, Iterator, Mapping, TYPE_CHECKING
from .disk import get_partitions_in_use, Partition
from .general import SysCommand, generate_password
import time
from pathlib import Path
from typing import Any, Iterator, List, Mapping, Optional, TYPE_CHECKING, Union, Dict
from . import disk
from .exceptions import DiskError, ServiceException, RequirementError, HardwareIncompatibilityError, SysCallError
from .general import SysCommand
from .hardware import has_uefi, is_vm, cpu_vendor
from .locale_helpers import verify_keyboard_layout, verify_x11_keyboard_layout
from .disk.helpers import findmnt
from .luks import Luks2
from .mirrors import use_mirrors
from .models.disk_encryption import DiskEncryption
from .plugins import plugins
from .storage import storage
from .output import log
from .profiles import Profile
from .disk.partition import get_mount_fs_type
from .exceptions import DiskError, ServiceException, RequirementError, HardwareIncompatibilityError, SysCallError
from .models.bootloader import Bootloader
from .models.network_configuration import NetworkConfiguration
from .models.users import User
from .models.subvolume import Subvolume
from .hsm import Fido2
from .output import log
from .pacman import run_pacman
from .plugins import plugins
from .services import service_state
from .storage import storage
if TYPE_CHECKING:
_: Any
@ -36,9 +35,6 @@ __packages__ = ["base", "base-devel", "linux-firmware", "linux", "linux-lts", "l
# Additional packages that are installed if the user is running the Live ISO with accessibility tools enabled
__accessibility_packages__ = ["brltty", "espeakup", "alsa-utils"]
from .pacman import run_pacman
from .models.network_configuration import NetworkConfiguration
class InstallationFile:
def __init__(self, installation :'Installer', filename :str, owner :str, mode :str = "w"):
@ -92,26 +88,35 @@ class Installer:
:param hostname: The given /etc/hostname for the machine.
:type hostname: str, optional
"""
def __init__(self, target :str, *, base_packages :Optional[List[str]] = None, kernels :Optional[List[str]] = None):
if base_packages is None:
def __init__(
self,
target: Path,
disk_config: disk.DiskLayoutConfiguration,
disk_encryption: Optional[disk.DiskEncryption] = None,
base_packages: List[str] = [],
kernels: Optional[List[str]] = None
):
if not base_packages:
base_packages = __packages__[:3]
if kernels is None:
self.kernels = ['linux']
else:
self.kernels = kernels
self._disk_config = disk_config
self._disk_encryption = disk_encryption
if self._disk_encryption is None:
self._disk_encryption = disk.DiskEncryption(disk.EncryptionType.NoEncryption)
self.target = target
self.init_time = time.strftime('%Y-%m-%d_%H-%M-%S')
self.milliseconds = int(str(time.time()).split('.')[1])
self.helper_flags = {'base': False, 'bootloader': False}
self.base_packages = base_packages
self.helper_flags = {
'base': False,
'bootloader': False
}
self.base_packages = base_packages.split(' ') if type(base_packages) is str else base_packages
for kernel in self.kernels:
self.base_packages.append(kernel)
@ -136,19 +141,10 @@ class Installer:
self._zram_enabled = False
self._disk_encryption: DiskEncryption = storage['arguments'].get('disk_encryption')
def log(self, *args :str, level :int = logging.DEBUG, **kwargs :str):
"""
installer.log() wraps output.log() mainly to set a default log-level for this install session.
Any manual override can be done per log() call.
"""
log(*args, level=level, **kwargs)
def __enter__(self, *args :str, **kwargs :str) -> 'Installer':
def __enter__(self, *args: str, **kwargs: str) -> 'Installer':
return self
def __exit__(self, *args :str, **kwargs :str) -> None:
def __exit__(self, *args :str, **kwargs :str) -> bool:
# TODO: https://stackoverflow.com/questions/28157929/how-to-safely-handle-an-exception-inside-a-context-manager
if len(args) >= 2 and args[1]:
@ -165,7 +161,6 @@ class Installer:
if not (missing_steps := self.post_install_check()):
self.log('Installation completed without any errors. You may now reboot.', fg='green', level=logging.INFO)
self.sync_log_to_install_medium()
return True
else:
self.log('Some required steps were not successfully installed/configured before leaving the installer:', fg='red', level=logging.WARNING)
@ -178,9 +173,154 @@ class Installer:
self.sync_log_to_install_medium()
return False
@property
def partitions(self) -> List[Partition]:
return get_partitions_in_use(self.target).values()
def log(self, *args :str, level :int = logging.DEBUG, **kwargs :str):
"""
installer.log() wraps output.log() mainly to set a default log-level for this install session.
Any manual override can be done per log() call.
"""
log(*args, level=level, **kwargs)
def _verify_service_stop(self):
"""
Certain services might be running that affects the system during installation.
Currently, only one such service is "reflector.service" which updates /etc/pacman.d/mirrorlist
We need to wait for it before we continue since we opted in to use a custom mirror/region.
"""
log('Waiting for automatic mirror selection (reflector) to complete...', level=logging.INFO)
while service_state('reflector') not in ('dead', 'failed', 'exited'):
time.sleep(1)
log('Waiting pacman-init.service to complete.', level=logging.INFO)
while service_state('pacman-init') not in ('dead', 'failed', 'exited'):
time.sleep(1)
log('Waiting Arch Linux keyring sync (archlinux-keyring-wkd-sync) to complete.', level=logging.INFO)
while service_state('archlinux-keyring-wkd-sync') not in ('dead', 'failed', 'exited'):
time.sleep(1)
def _verify_boot_part(self):
"""
Check that mounted /boot device has at minimum size for installation
The reason this check is here is to catch pre-mounted device configuration and potentially
configured one that has not gone through any previous checks (e.g. --silence mode)
NOTE: this function should be run AFTER running the mount_ordered_layout function
"""
boot_mount = self.target / 'boot'
lsblk_info = disk.get_lsblk_by_mountpoint(boot_mount)
if len(lsblk_info) > 0:
if lsblk_info[0].size < disk.Size(200, disk.Unit.MiB):
raise DiskError(
f'The boot partition mounted at {boot_mount} is not large enough to install a boot loader. '
f'Please resize it to at least 200MiB and re-run the installation.'
)
def sanity_check(self):
self._verify_boot_part()
self._verify_service_stop()
def mount_ordered_layout(self):
log('Mounting partitions in order', level=logging.INFO)
for mod in self._disk_config.device_modifications:
# partitions have to mounted in the right order on btrfs the mountpoint will
# be empty as the actual subvolumes are getting mounted instead so we'll use
# '/' just for sorting
sorted_part_mods = sorted(mod.partitions, key=lambda x: x.mountpoint if x.mountpoint else Path('/'))
if self._disk_encryption.encryption_type is not disk.EncryptionType.NoEncryption:
enc_partitions = list(filter(lambda x: x in self._disk_encryption.partitions, sorted_part_mods))
else:
enc_partitions = []
# attempt to decrypt all luks partitions
luks_handlers = self._prepare_luks_partitions(enc_partitions)
for part_mod in sorted_part_mods:
if part_mod not in luks_handlers: # partition is not encrypted
self._mount_partition(part_mod)
else: # mount encrypted partition
self._mount_luks_partiton(part_mod, luks_handlers[part_mod])
def _prepare_luks_partitions(self, partitions: List[disk.PartitionModification]) -> Dict[disk.PartitionModification, Luks2]:
luks_handlers = {}
for part_mod in partitions:
luks_handler = disk.device_handler.unlock_luks2_dev(
part_mod.dev_path,
part_mod.mapper_name,
self._disk_encryption.encryption_password
)
luks_handlers[part_mod] = luks_handler
return luks_handlers
def _mount_partition(self, part_mod: disk.PartitionModification):
# it would be none if it's btrfs as the subvolumes will have the mountpoints defined
if part_mod.mountpoint is not None:
target = self.target / part_mod.relative_mountpoint
disk.device_handler.mount(part_mod.dev_path, target, options=part_mod.mount_options)
if part_mod.fs_type == disk.FilesystemType.Btrfs:
self._mount_btrfs_subvol(part_mod.dev_path, part_mod.btrfs_subvols)
def _mount_luks_partiton(self, part_mod: disk.PartitionModification, luks_handler: Luks2):
# it would be none if it's btrfs as the subvolumes will have the mountpoints defined
if part_mod.mountpoint is not None:
target = self.target / part_mod.relative_mountpoint
disk.device_handler.mount(luks_handler.mapper_dev, target, options=part_mod.mount_options)
if part_mod.fs_type == disk.FilesystemType.Btrfs:
self._mount_btrfs_subvol(luks_handler.mapper_dev, part_mod.btrfs_subvols)
def _mount_btrfs_subvol(self, dev_path: Path, subvolumes: List[disk.SubvolumeModification]):
for subvol in subvolumes:
mountpoint = self.target / subvol.relative_mountpoint
mount_options = subvol.mount_options + [f'subvol={subvol.name}']
disk.device_handler.mount(dev_path, mountpoint, options=mount_options)
def generate_key_files(self):
for part_mod in self._disk_encryption.partitions:
gen_enc_file = self._disk_encryption.should_generate_encryption_file(part_mod)
luks_handler = Luks2(
part_mod.dev_path,
mapper_name=part_mod.mapper_name,
password=self._disk_encryption.encryption_password
)
if gen_enc_file and not part_mod.is_root():
log(f'Creating key-file: {part_mod.dev_path}', level=logging.INFO)
luks_handler.create_keyfile(self.target)
if part_mod.is_root() and not gen_enc_file:
if self._disk_encryption.hsm_device:
disk.Fido2.fido2_enroll(
self._disk_encryption.hsm_device,
part_mod,
self._disk_encryption.encryption_password
)
def activate_ntp(self):
"""
If NTP is activated, confirm activiation in the ISO and at least one time-sync finishes
"""
SysCommand('timedatectl set-ntp true')
logged = False
while service_state('dbus-org.freedesktop.timesync1.service') not in ['running']:
if not logged:
log(f"Waiting for dbus-org.freedesktop.timesync1.service to enter running state", level=logging.INFO)
logged = True
time.sleep(1)
logged = False
while 'Server: n/a' in SysCommand('timedatectl timesync-status --no-pager --property=Server --value'):
if not logged:
log(f"Waiting for timedatectl timesync-status to report a timesync against a server", level=logging.INFO)
logged = True
time.sleep(1)
def sync_log_to_install_medium(self) -> bool:
# Copy over the install log (if there is one) to the install medium if
@ -196,129 +336,6 @@ class Installer:
return True
def _create_keyfile(self,luks_handle , partition :dict, password :str):
""" roiutine to create keyfiles, so it can be moved elsewhere
"""
if self._disk_encryption and self._disk_encryption.generate_encryption_file(partition):
if not (cryptkey_dir := pathlib.Path(f"{self.target}/etc/cryptsetup-keys.d")).exists():
cryptkey_dir.mkdir(parents=True)
# Once we store the key as ../xyzloop.key systemd-cryptsetup can automatically load this key
# if we name the device to "xyzloop".
if partition.get('mountpoint',None):
encryption_key_path = f"/etc/cryptsetup-keys.d/{pathlib.Path(partition['mountpoint']).name}loop.key"
else:
encryption_key_path = f"/etc/cryptsetup-keys.d/{pathlib.Path(partition['device_instance'].path).name}.key"
with open(f"{self.target}{encryption_key_path}", "w") as keyfile:
keyfile.write(generate_password(length=512))
os.chmod(f"{self.target}{encryption_key_path}", 0o400)
luks_handle.add_key(pathlib.Path(f"{self.target}{encryption_key_path}"), password=password)
luks_handle.crypttab(self, encryption_key_path, options=["luks", "key-slot=1"])
def _has_root(self, partition :dict) -> bool:
"""
Determine if an encrypted partition contains root in it
"""
if partition.get("mountpoint") is None:
if (sub_list := partition.get("btrfs",{}).get('subvolumes',{})):
for mountpoint in [sub_list[subvolume].get("mountpoint") if isinstance(subvolume, dict) else subvolume.mountpoint for subvolume in sub_list]:
if mountpoint == '/':
return True
return False
else:
return False
elif partition.get("mountpoint") == '/':
return True
else:
return False
def mount_ordered_layout(self, layouts: Dict[str, Any]) -> None:
from .luks import luks2
from .disk.btrfs import setup_subvolumes, mount_subvolume
# set the partitions as a list not part of a tree (which we don't need anymore (i think)
list_part = []
list_luks_handles = []
for blockdevice in layouts:
list_part.extend(layouts[blockdevice]['partitions'])
# TODO: Implement a proper mount-queue system that does not depend on return values.
mount_queue = {}
# we manage the encrypted partititons
if self._disk_encryption:
for partition in self._disk_encryption.all_partitions:
# open the luks device and all associate stuff
loopdev = f"{storage.get('ENC_IDENTIFIER', 'ai')}{pathlib.Path(partition['device_instance'].path).name}"
# note that we DON'T auto_unmount (i.e. close the encrypted device so it can be used
with (luks_handle := luks2(partition['device_instance'], loopdev, self._disk_encryption.encryption_password, auto_unmount=False)) as unlocked_device:
if self._disk_encryption.generate_encryption_file(partition) and not self._has_root(partition):
list_luks_handles.append([luks_handle, partition, self._disk_encryption.encryption_password])
# this way all the requesrs will be to the dm_crypt device and not to the physical partition
partition['device_instance'] = unlocked_device
if self._has_root(partition) and self._disk_encryption.generate_encryption_file(partition) is False:
if self._disk_encryption.hsm_device:
Fido2.fido2_enroll(self._disk_encryption.hsm_device, partition['device_instance'], self._disk_encryption.encryption_password)
btrfs_subvolumes = [entry for entry in list_part if entry.get('btrfs', {}).get('subvolumes', [])]
for partition in btrfs_subvolumes:
device_instance = partition['device_instance']
mount_options = partition.get('filesystem', {}).get('mount_options', [])
self.mount(device_instance, "/", options=','.join(mount_options))
setup_subvolumes(installation=self, partition_dict=partition)
device_instance.unmount()
# We then handle any special cases, such as btrfs
for partition in btrfs_subvolumes:
subvolumes: List[Subvolume] = partition['btrfs']['subvolumes']
for subvolume in sorted(subvolumes, key=lambda item: item.mountpoint):
# We cache the mount call for later
mount_queue[subvolume.mountpoint] = lambda sub_vol=subvolume, device=partition['device_instance']: mount_subvolume(
installation=self,
device=device,
subvolume=sub_vol
)
# We mount ordinary partitions, and we sort them by the mountpoint
for partition in sorted([entry for entry in list_part if entry.get('mountpoint', False)], key=lambda part: part['mountpoint']):
mountpoint = partition['mountpoint']
log(f"Mounting {mountpoint} to {self.target}{mountpoint} using {partition['device_instance']}", level=logging.INFO)
if partition.get('filesystem',{}).get('mount_options',[]):
mount_options = ','.join(partition['filesystem']['mount_options'])
mount_queue[mountpoint] = lambda instance=partition['device_instance'], target=f"{self.target}{mountpoint}", options=mount_options: instance.mount(target, options=options)
else:
mount_queue[mountpoint] = lambda instance=partition['device_instance'], target=f"{self.target}{mountpoint}": instance.mount(target)
log(f"Using mount order: {list(sorted(mount_queue.items(), key=lambda item: item[0]))}", level=logging.DEBUG, fg="white")
# We mount everything by sorting on the mountpoint itself.
for mountpoint, frozen_func in sorted(mount_queue.items(), key=lambda item: item[0]):
frozen_func()
time.sleep(1)
try:
findmnt(pathlib.Path(f"{self.target}{mountpoint}"), traverse=False)
except DiskError:
raise DiskError(f"Target {self.target}{mountpoint} never got mounted properly (unable to get mount information using findmnt).")
# once everything is mounted, we generate the key files in the correct place
for handle in list_luks_handles:
ppath = handle[1]['device_instance'].path
log(f"creating key-file for {ppath}",level=logging.INFO)
self._create_keyfile(handle[0],handle[1],handle[2])
def mount(self, partition :Partition, mountpoint :str, create_mountpoint :bool = True, options='') -> None:
if create_mountpoint and not os.path.isdir(f'{self.target}{mountpoint}'):
os.makedirs(f'{self.target}{mountpoint}')
partition.mount(f'{self.target}{mountpoint}', options=options)
def add_swapfile(self, size='4G', enable_resume=True, file='/swapfile'):
if file[:1] != '/':
file = f"/{file}"
@ -394,7 +411,7 @@ class Installer:
else:
pacman_conf.write(line)
def pacstrap(self, *packages :str, **kwargs :str) -> bool:
def pacstrap(self, *packages: Union[str, List[str]], **kwargs :str) -> bool:
if type(packages[0]) in (list, tuple):
packages = packages[0]
@ -437,7 +454,7 @@ class Installer:
return use_mirrors(mirrors, destination=f'{self.target}/etc/pacman.d/mirrorlist')
def genfstab(self, flags :str = '-pU') -> bool:
def genfstab(self, flags :str = '-pU'):
self.log(f"Updating {self.target}/etc/fstab", level=logging.INFO)
try:
@ -460,7 +477,37 @@ class Installer:
for entry in self.FSTAB_ENTRIES:
fstab_fh.write(f'{entry}\n')
return True
for mod in self._disk_config.device_modifications:
for part_mod in mod.partitions:
if part_mod.fs_type != disk.FilesystemType.Btrfs:
continue
fstab_file = Path(f'{self.target}/etc/fstab')
with fstab_file.open('r') as fp:
fstab = fp.readlines()
# Replace the {installation}/etc/fstab with entries
# using the compress=zstd where the mountpoint has compression set.
for index, line in enumerate(fstab):
# So first we grab the mount options by using subvol=.*? as a locator.
# And we also grab the mountpoint for the entry, for instance /var/log
subvoldef = re.findall(',.*?subvol=.*?[\t ]', line)
mountpoint = re.findall('[\t ]/.*?[\t ]', line)
if not subvoldef or not mountpoint:
continue
for sub_vol in part_mod.btrfs_subvols:
# We then locate the correct subvolume and check if it's compressed,
# and skip entries where compression is already defined
# We then sneak in the compress=zstd option if it doesn't already exist:
if sub_vol.compress and str(sub_vol.mountpoint) == Path(mountpoint[0].strip()) and ',compress=zstd,' not in line:
fstab[index] = line.replace(subvoldef[0], f',compress=zstd{subvoldef[0]}')
break
with fstab_file.open('w') as fp:
fp.writelines(fstab)
def set_hostname(self, hostname: str, *args :str, **kwargs :str) -> None:
with open(f'{self.target}/etc/hostname', 'w') as fh:
@ -509,8 +556,8 @@ class Installer:
if result := plugin.on_timezone(zone):
zone = result
if (pathlib.Path("/usr") / "share" / "zoneinfo" / zone).exists():
(pathlib.Path(self.target) / "etc" / "localtime").unlink(missing_ok=True)
if (Path("/usr") / "share" / "zoneinfo" / zone).exists():
(Path(self.target) / "etc" / "localtime").unlink(missing_ok=True)
SysCommand(f'/usr/bin/arch-chroot {self.target} ln -s /usr/share/zoneinfo/{zone} /etc/localtime')
return True
@ -523,10 +570,6 @@ class Installer:
return False
def activate_ntp(self) -> None:
log(f"activate_ntp() is deprecated, use activate_time_syncronization()", fg="yellow", level=logging.INFO)
self.activate_time_syncronization()
def activate_time_syncronization(self) -> None:
self.log('Activating systemd-timesyncd for time synchronization using Arch Linux and ntp.org NTP servers.', level=logging.INFO)
self.enable_service('systemd-timesyncd')
@ -540,7 +583,10 @@ class Installer:
# fstrim is owned by util-linux, a dependency of both base and systemd.
self.enable_service("fstrim.timer")
def enable_service(self, *services :str) -> None:
def enable_service(self, *services: Union[str, List[str]]) -> None:
if type(services[0]) in (list, tuple):
services = services[0]
for service in services:
self.log(f'Enabling service {service}', level=logging.INFO)
try:
@ -552,10 +598,10 @@ class Installer:
if hasattr(plugin, 'on_service'):
plugin.on_service(service)
def run_command(self, cmd :str, *args :str, **kwargs :str) -> None:
def run_command(self, cmd :str, *args :str, **kwargs :str) -> SysCommand:
return SysCommand(f'/usr/bin/arch-chroot {self.target} {cmd}')
def arch_chroot(self, cmd :str, run_as :Optional[str] = None):
def arch_chroot(self, cmd :str, run_as :Optional[str] = None) -> SysCommand:
if run_as:
cmd = f"su - {run_as} -c {shlex.quote(cmd)}"
@ -645,21 +691,6 @@ class Installer:
return True
def detect_encryption(self, partition :Partition) -> bool:
from .disk.mapperdev import MapperDev
from .disk.dmcryptdev import DMCryptDev
from .disk.helpers import get_filesystem_type
if type(partition) is MapperDev:
# Returns MapperDev.partition
return partition.partition
elif type(partition) is DMCryptDev:
return partition.MapperDev.partition
elif get_filesystem_type(partition.path) == 'crypto_LUKS':
return partition
return False
def mkinitcpio(self, *flags :str) -> bool:
for plugin in plugins.values():
if hasattr(plugin, 'on_mkinitcpio'):
@ -668,7 +699,7 @@ class Installer:
return True
# mkinitcpio will error out if there's no vconsole.
if (vconsole := pathlib.Path(f"{self.target}/etc/vconsole.conf")).exists() is False:
if (vconsole := Path(f"{self.target}/etc/vconsole.conf")).exists() is False:
with vconsole.open('w') as fh:
fh.write(f"KEYMAP={storage['arguments']['keyboard-layout']}\n")
@ -677,7 +708,7 @@ class Installer:
mkinit.write(f"BINARIES=({' '.join(self.BINARIES)})\n")
mkinit.write(f"FILES=({' '.join(self.FILES)})\n")
if self._disk_encryption and not self._disk_encryption.hsm_device:
if not self._disk_encryption.hsm_device:
# For now, if we don't use HSM we revert to the old
# way of setting up encryption hooks for mkinitcpio.
# This is purely for stability reasons, we're going away from this.
@ -694,46 +725,36 @@ class Installer:
return False
def minimal_installation(
self, testing: bool = False, multilib: bool = False,
hostname: str = 'archinstall', locales: List[str] = ['en_US.UTF-8 UTF-8']) -> bool:
# Add necessary packages if encrypting the drive
# (encrypted partitions default to btrfs for now, so we need btrfs-progs)
# TODO: Perhaps this should be living in the function which dictates
# the partitioning. Leaving here for now.
self,
testing: bool = False,
multilib: bool = False,
hostname: str = 'archinstall',
locales: List[str] = ['en_US.UTF-8 UTF-8']
):
for mod in self._disk_config.device_modifications:
for part in mod.partitions:
if (pkg := part.fs_type.installation_pkg) is not None:
self.base_packages.append(pkg)
if (module := part.fs_type.installation_module) is not None:
self.MODULES.append(module)
if (binary := part.fs_type.installation_binary) is not None:
self.BINARIES.append(binary)
for partition in self.partitions:
if partition.filesystem == 'btrfs':
# if partition.encrypted:
if 'btrfs-progs' not in self.base_packages:
self.base_packages.append('btrfs-progs')
if partition.filesystem == 'xfs':
if 'xfs' not in self.base_packages:
self.base_packages.append('xfsprogs')
if partition.filesystem == 'f2fs':
if 'f2fs' not in self.base_packages:
self.base_packages.append('f2fs-tools')
# There is not yet an fsck tool for NTFS. If it's being used for the root filesystem, the hook should be removed.
if part.fs_type.fs_type_mount == 'ntfs3' and part.mountpoint == self.target:
if 'fsck' in self.HOOKS:
self.HOOKS.remove('fsck')
# Configure mkinitcpio to handle some specific use cases.
if partition.filesystem == 'btrfs':
if 'btrfs' not in self.MODULES:
self.MODULES.append('btrfs')
if '/usr/bin/btrfs' not in self.BINARIES:
self.BINARIES.append('/usr/bin/btrfs')
# There is not yet an fsck tool for NTFS. If it's being used for the root filesystem, the hook should be removed.
if partition.filesystem == 'ntfs3' and partition.mountpoint == self.target:
if 'fsck' in self.HOOKS:
self.HOOKS.remove('fsck')
if part in self._disk_encryption.partitions:
if self._disk_encryption.hsm_device:
# Required bby mkinitcpio to add support for fido2-device options
self.pacstrap('libfido2')
if self.detect_encryption(partition):
if self._disk_encryption and self._disk_encryption.hsm_device:
# Required bby mkinitcpio to add support for fido2-device options
self.pacstrap('libfido2')
if 'sd-encrypt' not in self.HOOKS:
self.HOOKS.insert(self.HOOKS.index('filesystems'), 'sd-encrypt')
else:
if 'encrypt' not in self.HOOKS:
self.HOOKS.insert(self.HOOKS.index('filesystems'), 'encrypt')
if 'sd-encrypt' not in self.HOOKS:
self.HOOKS.insert(self.HOOKS.index('filesystems'), 'sd-encrypt')
else:
if 'encrypt' not in self.HOOKS:
self.HOOKS.insert(self.HOOKS.index('filesystems'), 'encrypt')
if not has_uefi():
self.base_packages.append('grub')
@ -742,11 +763,11 @@ class Installer:
vendor = cpu_vendor()
if vendor == "AuthenticAMD":
self.base_packages.append("amd-ucode")
if (ucode := pathlib.Path(f"{self.target}/boot/amd-ucode.img")).exists():
if (ucode := Path(f"{self.target}/boot/amd-ucode.img")).exists():
ucode.unlink()
elif vendor == "GenuineIntel":
self.base_packages.append("intel-ucode")
if (ucode := pathlib.Path(f"{self.target}/boot/intel-ucode.img")).exists():
if (ucode := Path(f"{self.target}/boot/intel-ucode.img")).exists():
ucode.unlink()
else:
self.log(f"Unknown CPU vendor '{vendor}' detected. Archinstall won't install any ucode.", level=logging.DEBUG)
@ -802,9 +823,7 @@ class Installer:
if hasattr(plugin, 'on_install'):
plugin.on_install(self)
return True
def setup_swap(self, kind :str = 'zram') -> bool:
def setup_swap(self, kind :str = 'zram'):
if kind == 'zram':
self.log(f"Setting up swap on zram")
self.pacstrap('zram-generator')
@ -818,16 +837,27 @@ class Installer:
self.enable_service('systemd-zram-setup@zram0.service')
self._zram_enabled = True
return True
else:
raise ValueError(f"Archinstall currently only supports setting up swap on zram")
def add_systemd_bootloader(self, boot_partition :Partition, root_partition :Partition) -> bool:
def _get_boot_partition(self) -> Optional[disk.PartitionModification]:
for layout in self._disk_config.device_modifications:
if boot := layout.get_boot_partition():
return boot
return None
def _get_root_partition(self) -> Optional[disk.PartitionModification]:
for mod in self._disk_config.device_modifications:
if root := mod.get_root_partition(self._disk_config.relative_mountpoint):
return root
return None
def _add_systemd_bootloader(self, root_partition: disk.PartitionModification):
self.pacstrap('efibootmgr')
if not has_uefi():
raise HardwareIncompatibilityError
# TODO: Ideally we would want to check if another config
# points towards the same disk and/or partition.
# And in which case we should do some clean up.
@ -882,74 +912,73 @@ class Installer:
elif vendor == "GenuineIntel":
entry.write("initrd /intel-ucode.img\n")
else:
self.log(f"Unknown CPU vendor '{vendor}' detected. Archinstall won't add any ucode to systemd-boot config.", level=logging.DEBUG)
self.log(
f"Unknown CPU vendor '{vendor}' detected. Archinstall won't add any ucode to systemd-boot config.",
level=logging.DEBUG)
entry.write(f"initrd /initramfs-{kernel}{variant}.img\n")
# blkid doesn't trigger on loopback devices really well,
# so we'll use the old manual method until we get that sorted out.
root_fs_type = get_mount_fs_type(root_partition.filesystem)
if root_fs_type is not None:
options_entry = f'rw rootfstype={root_fs_type} {" ".join(self.KERNEL_PARAMS)}\n'
else:
options_entry = f'rw {" ".join(self.KERNEL_PARAMS)}\n'
options_entry = f'rw rootfstype={root_partition.fs_type.fs_type_mount} {" ".join(self.KERNEL_PARAMS)}\n'
for subvolume in root_partition.subvolumes:
if subvolume.root is True and subvolume.name != '<FS_TREE>':
options_entry = f"rootflags=subvol={subvolume.name} " + options_entry
for sub_vol in root_partition.btrfs_subvols:
if sub_vol.is_root():
options_entry = f"rootflags=subvol={sub_vol.name} " + options_entry
# Zswap should be disabled when using zram.
#
# https://github.com/archlinux/archinstall/issues/881
if self._zram_enabled:
options_entry = "zswap.enabled=0 " + options_entry
if real_device := self.detect_encryption(root_partition):
if root_partition.fs_type.is_crypto():
# TODO: We need to detect if the encrypted device is a whole disk encryption,
# or simply a partition encryption. Right now we assume it's a partition (and we always have)
log(f"Identifying root partition by PART-UUID on {real_device}: '{real_device.uuid}/{real_device.part_uuid}'.", level=logging.DEBUG)
log('Root partition is an encrypted device, identifying by PARTUUID: {root_partition.partuuid}', level=logging.DEBUG)
kernel_options = f"options"
if self._disk_encryption and self._disk_encryption.hsm_device:
# Note: lsblk UUID must be used, not PARTUUID for sd-encrypt to work
kernel_options += f" rd.luks.name={real_device.uuid}=luksdev"
kernel_options += f' rd.luks.name={root_partition.uuid}=luksdev'
# Note: tpm2-device and fido2-device don't play along very well:
# https://github.com/archlinux/archinstall/pull/1196#issuecomment-1129715645
kernel_options += f" rd.luks.options=fido2-device=auto,password-echo=no"
kernel_options += f' rd.luks.options=fido2-device=auto,password-echo=no'
else:
kernel_options += f" cryptdevice=PARTUUID={real_device.part_uuid}:luksdev"
kernel_options += f' cryptdevice=PARTUUID={root_partition.partuuid}:luksdev'
entry.write(f'{kernel_options} root=/dev/mapper/luksdev {options_entry}')
else:
log(f"Identifying root partition by PARTUUID on {root_partition}, looking for '{root_partition.part_uuid}'.", level=logging.DEBUG)
entry.write(f'options root=PARTUUID={root_partition.part_uuid} {options_entry}')
log(f'Identifying root partition by PARTUUID: {root_partition.partuuid}', level=logging.DEBUG)
entry.write(f'options root=PARTUUID={root_partition.partuuid} {options_entry}')
self.helper_flags['bootloader'] = "systemd"
self.helper_flags['bootloader'] = 'systemd'
return True
def add_grub_bootloader(self, boot_partition :Partition, root_partition :Partition) -> bool:
def _add_grub_bootloader(
self,
boot_partition: disk.PartitionModification,
root_partition: disk.PartitionModification
):
self.pacstrap('grub') # no need?
root_fs_type = get_mount_fs_type(root_partition.filesystem)
_file = "/etc/default/grub"
if real_device := self.detect_encryption(root_partition):
root_uuid = SysCommand(f"blkid -s UUID -o value {real_device.path}").decode().rstrip()
_file = "/etc/default/grub"
add_to_CMDLINE_LINUX = f"sed -i 's/GRUB_CMDLINE_LINUX=\"\"/GRUB_CMDLINE_LINUX=\"cryptdevice=UUID={root_uuid}:cryptlvm rootfstype={root_fs_type}\"/'"
enable_CRYPTODISK = "sed -i 's/#GRUB_ENABLE_CRYPTODISK=y/GRUB_ENABLE_CRYPTODISK=y/'"
if root_partition.fs_type.is_crypto():
log(f"Using UUID {root_partition.uuid} as encrypted root identifier", level=logging.DEBUG)
log(f"Using UUID {root_uuid} of {real_device} as encrypted root identifier.", level=logging.INFO)
SysCommand(f"/usr/bin/arch-chroot {self.target} {add_to_CMDLINE_LINUX} {_file}")
SysCommand(f"/usr/bin/arch-chroot {self.target} {enable_CRYPTODISK} {_file}")
cmd_line_linux = f"sed -i 's/GRUB_CMDLINE_LINUX=\"\"/GRUB_CMDLINE_LINUX=\"cryptdevice=UUID={root_partition.uuid}:cryptlvm rootfstype={root_partition.fs_type.value}\"/'"
enable_cryptdisk = "sed -i 's/#GRUB_ENABLE_CRYPTODISK=y/GRUB_ENABLE_CRYPTODISK=y/'"
SysCommand(f"/usr/bin/arch-chroot {self.target} {enable_cryptdisk} {_file}")
else:
_file = "/etc/default/grub"
add_to_CMDLINE_LINUX = f"sed -i 's/GRUB_CMDLINE_LINUX=\"\"/GRUB_CMDLINE_LINUX=\"rootfstype={root_fs_type}\"/'"
SysCommand(f"/usr/bin/arch-chroot {self.target} {add_to_CMDLINE_LINUX} {_file}")
cmd_line_linux = f"sed -i 's/GRUB_CMDLINE_LINUX=\"\"/GRUB_CMDLINE_LINUX=\"rootfstype={root_partition.fs_type.value}\"/'"
SysCommand(f"/usr/bin/arch-chroot {self.target} {cmd_line_linux} {_file}")
log(f"GRUB boot partition: {boot_partition.dev_path}", level=logging.INFO)
log(f"GRUB uses {boot_partition.path} as the boot partition.", level=logging.INFO)
if has_uefi():
self.pacstrap('efibootmgr') # TODO: Do we need? Yes, but remove from minimal_installation() instead?
try:
SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --debug --target=x86_64-efi --efi-directory=/boot --bootloader-id=GRUB --removable', peek_output=True)
except SysCallError:
@ -961,7 +990,7 @@ class Installer:
try:
SysCommand(f'/usr/bin/arch-chroot {self.target} grub-install --debug --target=i386-pc --recheck {boot_partition.parent}', peek_output=True)
except SysCallError as error:
raise DiskError(f"Could not install GRUB to {boot_partition.path}: {error}")
raise DiskError(f"Failed to install GRUB boot on {boot_partition.dev_path}: {error}")
try:
SysCommand(f'/usr/bin/arch-chroot {self.target} grub-mkconfig -o /boot/grub/grub.cfg')
@ -970,22 +999,22 @@ class Installer:
self.helper_flags['bootloader'] = "grub"
return True
def add_efistub_bootloader(self, boot_partition :Partition, root_partition :Partition) -> bool:
def _add_efistub_bootloader(
self,
boot_partition: disk.PartitionModification,
root_partition: disk.PartitionModification
):
self.pacstrap('efibootmgr')
if not has_uefi():
raise HardwareIncompatibilityError
# TODO: Ideally we would want to check if another config
# points towards the same disk and/or partition.
# And in which case we should do some clean up.
root_fs_type = get_mount_fs_type(root_partition.filesystem)
for kernel in self.kernels:
# Setup the firmware entry
label = f'Arch Linux ({kernel})'
loader = f"/vmlinuz-{kernel}"
@ -1004,22 +1033,22 @@ class Installer:
# blkid doesn't trigger on loopback devices really well,
# so we'll use the old manual method until we get that sorted out.
if real_device := self.detect_encryption(root_partition):
if root_partition.fs_type.is_crypto():
# TODO: We need to detect if the encrypted device is a whole disk encryption,
# or simply a partition encryption. Right now we assume it's a partition (and we always have)
log(f"Identifying root partition by PART-UUID on {real_device}: '{real_device.part_uuid}'.", level=logging.DEBUG)
kernel_parameters.append(f'cryptdevice=PARTUUID={real_device.part_uuid}:luksdev root=/dev/mapper/luksdev rw rootfstype={root_fs_type} {" ".join(self.KERNEL_PARAMS)}')
log(f'Identifying root partition by PARTUUID: {root_partition.partuuid}', level=logging.DEBUG)
kernel_parameters.append(f'cryptdevice=PARTUUID={root_partition.partuuid}:luksdev root=/dev/mapper/luksdev rw rootfstype={root_partition.fs_type.value} {" ".join(self.KERNEL_PARAMS)}')
else:
log(f"Identifying root partition by PART-UUID on {root_partition}, looking for '{root_partition.part_uuid}'.", level=logging.DEBUG)
kernel_parameters.append(f'root=PARTUUID={root_partition.part_uuid} rw rootfstype={root_fs_type} {" ".join(self.KERNEL_PARAMS)}')
log(f'Root partition is an encrypted device identifying by PARTUUID: {root_partition.partuuid}', level=logging.DEBUG)
kernel_parameters.append(f'root=PARTUUID={root_partition.partuuid} rw rootfstype={root_partition.fs_type.value} {" ".join(self.KERNEL_PARAMS)}')
SysCommand(f'efibootmgr --disk {boot_partition.path[:-1]} --part {boot_partition.path[-1]} --create --label "{label}" --loader {loader} --unicode \'{" ".join(kernel_parameters)}\' --verbose')
device = disk.device_handler.get_device_by_partition_path(boot_partition.dev_path)
SysCommand(f'efibootmgr --disk {device.path} --part {device.path} --create --label "{label}" --loader {loader} --unicode \'{" ".join(kernel_parameters)}\' --verbose')
self.helper_flags['bootloader'] = "efistub"
return True
def add_bootloader(self, bootloader :str = 'systemd-bootctl') -> bool:
def add_bootloader(self, bootloader: Bootloader) -> bool:
"""
Adds a bootloader to the installation instance.
Archinstall supports one of three types:
@ -1039,52 +1068,33 @@ class Installer:
return True
if type(self.target) == str:
self.target = pathlib.Path(self.target)
self.target = Path(self.target)
boot_partition = None
root_partition = None
for partition in self.partitions:
if self.target / 'boot' in partition.mountpoints:
boot_partition = partition
elif self.target in partition.mountpoints:
root_partition = partition
boot_partition = self._get_boot_partition()
root_partition = self._get_root_partition()
if boot_partition is None or root_partition is None:
raise ValueError(f"Could not detect root ({root_partition}) or boot ({boot_partition}) in {self.target} based on: {self.partitions}")
if boot_partition is None:
raise ValueError(f'Could not detect boot at mountpoint {self.target}')
self.log(f'Adding bootloader {bootloader} to {boot_partition if boot_partition else root_partition}', level=logging.INFO)
if root_partition is None:
raise ValueError(f'Could not detect root at mountpoint {self.target}')
if bootloader == 'systemd-bootctl':
self.add_systemd_bootloader(boot_partition, root_partition)
elif bootloader == "grub-install":
self.add_grub_bootloader(boot_partition, root_partition)
elif bootloader == 'efistub':
self.add_efistub_bootloader(boot_partition, root_partition)
else:
raise RequirementError(f"Unknown (or not yet implemented) bootloader requested: {bootloader}")
self.log(f'Adding bootloader {bootloader.value} to {boot_partition.dev_path}', level=logging.INFO)
return True
match bootloader:
case Bootloader.Systemd:
self._add_systemd_bootloader(root_partition)
case Bootloader.Grub:
self._add_grub_bootloader(boot_partition, root_partition)
case Bootloader.Efistub:
self._add_efistub_bootloader(boot_partition, root_partition)
def add_additional_packages(self, *packages :str) -> bool:
def add_additional_packages(self, *packages: Union[str, List[str]]) -> bool:
return self.pacstrap(*packages)
def install_profile(self, profile :str) -> ModuleType:
"""
Installs a archinstall profile script (.py file).
This profile can be either local, remote or part of the library.
:param profile: Can be a local path or a remote path (URL)
:return: Returns the imported script as a module, this way
you can access any remaining functions exposed by the profile.
:rtype: module
"""
storage['installation_session'] = self
if type(profile) == str:
profile = Profile(self, profile)
self.log(f'Installing archinstall profile {profile}', level=logging.INFO)
return profile.install()
def _enable_users(self, service: str, users: List[User]):
for user in users:
self.arch_chroot(f'systemctl enable --user {service}', run_as=user.username)
def enable_sudo(self, entity: str, group :bool = False):
self.log(f'Enabling sudo permissions for {entity}.', level=logging.INFO)
@ -1092,7 +1102,7 @@ class Installer:
sudoers_dir = f"{self.target}/etc/sudoers.d"
# Creates directory if not exists
if not (sudoers_path := pathlib.Path(sudoers_dir)).exists():
if not (sudoers_path := Path(sudoers_dir)).exists():
sudoers_path.mkdir(parents=True)
# Guarantees sudoer confs directory recommended perms
os.chmod(sudoers_dir, 0o440)
@ -1114,7 +1124,7 @@ class Installer:
sudoers.write(f'{"%" if group else ""}{entity} ALL=(ALL) ALL\n')
# Guarantees sudoer conf file recommended perms
os.chmod(pathlib.Path(rule_file_name), 0o440)
os.chmod(Path(rule_file_name), 0o440)
def create_users(self, users: Union[User, List[User]]):
if not isinstance(users, list):

20
archinstall/lib/locale_helpers.py
View File

@ -1,11 +1,12 @@
import logging
from typing import Iterator, List, Callable
from typing import Iterator, List, Callable, Optional
from .exceptions import ServiceException
from .general import SysCommand
from .output import log
from .storage import storage
def list_keyboard_languages() -> Iterator[str]:
for line in SysCommand("localectl --no-pager list-keymaps", environment_vars={'SYSTEMD_COLORS': '0'}):
yield line.decode('UTF-8').strip()
@ -45,20 +46,25 @@ def get_locale_mode_text(mode):
mode_text = "Unassigned"
return mode_text
def reset_cmd_locale():
""" sets the cmd_locale to its saved default """
storage['CMD_LOCALE'] = storage.get('CMD_LOCALE_DEFAULT',{})
def unset_cmd_locale():
""" archinstall will use the execution environment default """
storage['CMD_LOCALE'] = {}
def set_cmd_locale(general :str = None,
charset :str = 'C',
numbers :str = 'C',
time :str = 'C',
collate :str = 'C',
messages :str = 'C'):
def set_cmd_locale(
general: Optional[str] = None,
charset :str = 'C',
numbers :str = 'C',
time :str = 'C',
collate :str = 'C',
messages :str = 'C'
):
"""
Set the cmd locale.
If the parameter general is specified, it takes precedence over the rest (might as well not exist)

303
archinstall/lib/luks.py
View File

@ -1,92 +1,78 @@
from __future__ import annotations
import json
import logging
import os
import pathlib
import shlex
import time
from typing import Optional, List,TYPE_CHECKING
# https://stackoverflow.com/a/39757388/929999
if TYPE_CHECKING:
from .installer import Installer
from dataclasses import dataclass
from pathlib import Path
from typing import Optional, List
from .disk import Partition, convert_device_to_uuid
from .general import SysCommand, SysCommandWorker
from . import disk
from .general import SysCommand, generate_password, SysCommandWorker
from .output import log
from .exceptions import SysCallError, DiskError
from .storage import storage
from .disk.helpers import get_filesystem_type
from .disk.mapperdev import MapperDev
from .disk.btrfs import BTRFSPartition
class luks2:
def __init__(self,
partition: Partition,
mountpoint: Optional[str],
password: Optional[str],
key_file :Optional[str] = None,
auto_unmount :bool = False,
*args :str,
**kwargs :str):
@dataclass
class Luks2:
luks_dev_path: Path
mapper_name: Optional[str] = None
password: Optional[str] = None
key_file: Optional[Path] = None
auto_unmount: bool = False
self.password = password
self.partition = partition
self.mountpoint = mountpoint
self.args = args
self.kwargs = kwargs
self.key_file = key_file
self.auto_unmount = auto_unmount
self.filesystem = 'crypto_LUKS'
self.mapdev = None
# will be set internally after unlocking the device
_mapper_dev: Optional[Path] = None
def __enter__(self) -> Partition:
if not self.key_file:
self.key_file = f"/tmp/{os.path.basename(self.partition.path)}.disk_pw" # TODO: Make disk-pw-file randomly unique?
@property
def mapper_dev(self) -> Optional[Path]:
if self.mapper_name:
return Path(f'/dev/mapper/{self.mapper_name}')
return None
if type(self.password) != bytes:
self.password = bytes(self.password, 'UTF-8')
def __post_init__(self):
if self.luks_dev_path is None:
raise ValueError('Partition must have a path set')
with open(self.key_file, 'wb') as fh:
fh.write(self.password)
def __enter__(self):
self.unlock(self.key_file)
return self.unlock(self.partition, self.mountpoint, self.key_file)
def __exit__(self, *args :str, **kwargs :str) -> bool:
# TODO: https://stackoverflow.com/questions/28157929/how-to-safely-handle-an-exception-inside-a-context-manager
def __exit__(self, *args: str, **kwargs: str):
if self.auto_unmount:
self.close()
self.lock()
if len(args) >= 2 and args[1]:
raise args[1]
def _default_key_file(self) -> Path:
return Path(f'/tmp/{self.luks_dev_path.name}.disk_pw')
return True
def _password_bytes(self) -> bytes:
if not self.password:
raise ValueError('Password for luks2 device was not specified')
def encrypt(self, partition :Partition,
password :Optional[str] = None,
key_size :int = 512,
hash_type :str = 'sha512',
iter_time :int = 10000,
key_file :Optional[str] = None) -> str:
if isinstance(self.password, bytes):
return self.password
else:
return bytes(self.password, 'UTF-8')
log(f'Encrypting {partition} (This might take a while)', level=logging.INFO)
def encrypt(
self,
key_size: int = 512,
hash_type: str = 'sha512',
iter_time: int = 10000,
key_file: Optional[Path] = None
) -> Path:
log(f'Luks2 encrypting: {self.luks_dev_path}', level=logging.INFO)
byte_password = self._password_bytes()
if not key_file:
if self.key_file:
key_file = self.key_file
else:
key_file = f"/tmp/{os.path.basename(self.partition.path)}.disk_pw" # TODO: Make disk-pw-file randomly unique?
key_file = self._default_key_file()
if not password:
password = self.password
if type(password) != bytes:
password = bytes(password, 'UTF-8')
with open(key_file, 'wb') as fh:
fh.write(password)
partition.partprobe()
with open(key_file, 'wb') as fh:
fh.write(byte_password)
cryptsetup_args = shlex.join([
'/usr/bin/cryptsetup',
@ -97,120 +83,163 @@ class luks2:
'--hash', hash_type,
'--key-size', str(key_size),
'--iter-time', str(iter_time),
'--key-file', os.path.abspath(key_file),
'--key-file', str(key_file),
'--use-urandom',
'luksFormat', partition.path,
'luksFormat', str(self.luks_dev_path),
])
try:
# Retry formatting the volume because archinstall can some times be too quick
# which generates a "Device /dev/sdX does not exist or access denied." between
# setting up partitions and us trying to encrypt it.
cmd_handle = None
for i in range(storage['DISK_RETRY_ATTEMPTS']):
if (cmd_handle := SysCommand(cryptsetup_args)).exit_code != 0:
time.sleep(storage['DISK_TIMEOUTS'])
else:
break
if cmd_handle.exit_code != 0:
raise DiskError(f'Could not encrypt volume "{partition.path}": {b"".join(cmd_handle)}')
if cmd_handle is not None and cmd_handle.exit_code != 0:
output = str(b''.join(cmd_handle))
raise DiskError(f'Could not encrypt volume "{self.luks_dev_path}": {output}')
except SysCallError as err:
if err.exit_code == 1:
log(f'{partition} is being used, trying to unmount and crypt-close the device and running one more attempt at encrypting the device.', level=logging.DEBUG)
# Partition was in use, unmount it and try again
partition.unmount()
# Get crypt-information about the device by doing a reverse lookup starting with the partition path
# For instance: /dev/sda
SysCommand(f'bash -c "partprobe"')
devinfo = json.loads(b''.join(SysCommand(f"lsblk --fs -J {partition.path}")).decode('UTF-8'))['blockdevices'][0]
# For each child (sub-partition/sub-device)
if len(children := devinfo.get('children', [])):
for child in children:
# Unmount the child location
if child_mountpoint := child.get('mountpoint', None):
log(f'Unmounting {child_mountpoint}', level=logging.DEBUG)
SysCommand(f"umount -R {child_mountpoint}")
# And close it if possible.
log(f"Closing crypt device {child['name']}", level=logging.DEBUG)
SysCommand(f"cryptsetup close {child['name']}")
log(f'luks2 partition currently in use: {self.luks_dev_path}')
log('Attempting to unmount, crypt-close and trying encryption again')
self.lock()
# Then try again to set up the crypt-device
cmd_handle = SysCommand(cryptsetup_args)
SysCommand(cryptsetup_args)
else:
raise err
return key_file
def unlock(self, partition :Partition, mountpoint :str, key_file :str) -> Partition:
"""
Mounts a luks2 compatible partition to a certain mountpoint.
Keyfile must be specified as there's no way to interact with the pw-prompt atm.
def _get_luks_uuid(self) -> str:
command = f'/usr/bin/cryptsetup luksUUID {self.luks_dev_path}'
:param mountpoint: The name without absolute path, for instance "luksdev" will point to /dev/mapper/luksdev
:type mountpoint: str
"""
try:
result = SysCommand(command)
if result.exit_code != 0:
raise DiskError(f'Unable to get UUID for Luks device: {result.decode()}')
if '/' in mountpoint:
os.path.basename(mountpoint) # TODO: Raise exception instead?
return result.decode() # type: ignore
except SysCallError as err:
log(f'Unable to get UUID for Luks device: {self.luks_dev_path}', level=logging.INFO)
raise err
def is_unlocked(self) -> bool:
return self.mapper_name is not None and Path(f'/dev/mapper/{self.mapper_name}').exists()
def unlock(self, key_file: Optional[Path] = None):
"""
Unlocks the luks device, an optional key file location for unlocking can be specified,
otherwise a default location for the key file will be used.
:param key_file: An alternative key file
:type key_file: Path
"""
log(f'Unlocking luks2 device: {self.luks_dev_path}', level=logging.DEBUG)
if not self.mapper_name:
raise ValueError('mapper name missing')
byte_password = self._password_bytes()
if not key_file:
if self.key_file:
key_file = self.key_file
else:
key_file = self._default_key_file()
with open(key_file, 'wb') as fh:
fh.write(byte_password)
wait_timer = time.time()
while pathlib.Path(partition.path).exists() is False and time.time() - wait_timer < 10:
while Path(self.luks_dev_path).exists() is False and time.time() - wait_timer < 10:
time.sleep(0.025)
SysCommand(f'/usr/bin/cryptsetup open {partition.path} {mountpoint} --key-file {os.path.abspath(key_file)} --type luks2')
if os.path.islink(f'/dev/mapper/{mountpoint}'):
self.mapdev = f'/dev/mapper/{mountpoint}'
SysCommand(f'/usr/bin/cryptsetup open {self.luks_dev_path} {self.mapper_name} --key-file {key_file} --type luks2')
if (filesystem_type := get_filesystem_type(pathlib.Path(self.mapdev))) == 'btrfs':
return BTRFSPartition(
self.mapdev,
block_device=MapperDev(mountpoint).partition.block_device,
encrypted=True,
filesystem=filesystem_type,
autodetect_filesystem=False
)
if not self.mapper_dev or not self.mapper_dev.is_symlink():
raise DiskError(f'Failed to open luks2 device: {self.luks_dev_path}')
return Partition(
self.mapdev,
block_device=MapperDev(mountpoint).partition.block_device,
encrypted=True,
filesystem=get_filesystem_type(self.mapdev),
autodetect_filesystem=False
)
def lock(self):
disk.device_handler.umount(self.luks_dev_path)
def close(self, mountpoint :Optional[str] = None) -> bool:
if not mountpoint:
mountpoint = self.mapdev
# Get crypt-information about the device by doing a reverse lookup starting with the partition path
# For instance: /dev/sda
disk.device_handler.partprobe(self.luks_dev_path)
lsblk_info = disk.get_lsblk_info(self.luks_dev_path)
SysCommand(f'/usr/bin/cryptsetup close {self.mapdev}')
return os.path.islink(self.mapdev) is False
# For each child (sub-partition/sub-device)
for child in lsblk_info.children:
# Unmount the child location
for mountpoint in child.mountpoints:
log(f'Unmounting {mountpoint}', level=logging.DEBUG)
disk.device_handler.umount(mountpoint, recursive=True)
def format(self, path :str) -> None:
if (handle := SysCommand(f"/usr/bin/cryptsetup -q -v luksErase {path}")).exit_code != 0:
raise DiskError(f'Could not format {path} with {self.filesystem} because: {b"".join(handle)}')
# And close it if possible.
log(f"Closing crypt device {child.name}", level=logging.DEBUG)
SysCommand(f"cryptsetup close {child.name}")
def add_key(self, path :pathlib.Path, password :str) -> bool:
if not path.exists():
raise OSError(2, f"Could not import {path} as a disk encryption key, file is missing.", str(path))
self._mapper_dev = None
log(f'Adding additional key-file {path} for {self.partition}', level=logging.INFO)
worker = SysCommandWorker(f"/usr/bin/cryptsetup -q -v luksAddKey {self.partition.path} {path}",
environment_vars={'LC_ALL':'C'})
def create_keyfile(self, target_path: Path, override: bool = False):
"""
Routine to create keyfiles, so it can be moved elsewhere
"""
if self.mapper_name is None:
raise ValueError('Mapper name must be provided')
# Once we store the key as ../xyzloop.key systemd-cryptsetup can
# automatically load this key if we name the device to "xyzloop"
key_file_path = target_path / 'etc/cryptsetup-keys.d/' / self.mapper_name
key_file = key_file_path / '.key'
crypttab_path = target_path / 'etc/crypttab'
if key_file.exists():
if not override:
log(f'Key file {key_file} already exists, keeping existing')
return
else:
log(f'Key file {key_file} already exists, overriding')
key_file_path.mkdir(parents=True, exist_ok=True)
with open(key_file, "w") as keyfile:
keyfile.write(generate_password(length=512))
key_file_path.chmod(0o400)
self._add_key(key_file)
self._crypttab(crypttab_path, key_file, options=["luks", "key-slot=1"])
def _add_key(self, key_file: Path):
log(f'Adding additional key-file {key_file}', level=logging.INFO)
command = f'/usr/bin/cryptsetup -q -v luksAddKey {self.luks_dev_path} {key_file}'
worker = SysCommandWorker(command, environment_vars={'LC_ALL': 'C'})
pw_injected = False
while worker.is_alive():
if b'Enter any existing passphrase' in worker and pw_injected is False:
worker.write(bytes(password, 'UTF-8'))
worker.write(self._password_bytes())
pw_injected = True
if worker.exit_code != 0:
raise DiskError(f'Could not add encryption key {path} to {self.partition} because: {worker}')
raise DiskError(f'Could not add encryption key {key_file} to {self.luks_dev_path}: {worker.decode()}')
return True
def _crypttab(
self,
crypttab_path: Path,
key_file: Path,
options: List[str]
) -> None:
log(f'Adding crypttab entry for key {key_file}', level=logging.INFO)
def crypttab(self, installation :Installer, key_path :str, options :List[str] = ["luks", "key-slot=1"]) -> None:
log(f'Adding a crypttab entry for key {key_path} in {installation}', level=logging.INFO)
with open(f"{installation.target}/etc/crypttab", "a") as crypttab:
crypttab.write(f"{self.mountpoint} UUID={convert_device_to_uuid(self.partition.path)} {key_path} {','.join(options)}\n")
with open(crypttab_path, 'a') as crypttab:
opt = ','.join(options)
uuid = self._get_luks_uuid()
row = f"{self.mapper_name} UUID={uuid} {key_file} {opt}\n"
crypttab.write(row)

11
archinstall/lib/menu/__init__.py
View File

@ -1,2 +1,9 @@
from .menu import Menu as Menu
from .global_menu import GlobalMenu as GlobalMenu
from .abstract_menu import Selector, AbstractMenu, AbstractSubMenu
from .list_manager import ListManager
from .menu import (
MenuSelectionType,
MenuSelection,
Menu,
)
from .table_selection_menu import TableMenu
from .text_input import TextInput

200
archinstall/lib/menu/abstract_menu.py
View File

@ -7,7 +7,6 @@ from .menu import Menu, MenuSelectionType
from ..locale_helpers import set_keyboard_language
from ..output import log
from ..translationhandler import TranslationHandler, Language
from ..user_interaction.general_conf import select_archinstall_language
if TYPE_CHECKING:
_: Any
@ -16,17 +15,17 @@ if TYPE_CHECKING:
class Selector:
def __init__(
self,
description :str,
func :Optional[Callable] = None,
display_func :Optional[Callable] = None,
default :Any = None,
enabled :bool = False,
dependencies :List = [],
dependencies_not :List = [],
exec_func :Optional[Callable] = None,
preview_func :Optional[Callable] = None,
mandatory :bool = False,
no_store :bool = False
description: str,
func: Optional[Callable[[str], Any]] = None,
display_func: Optional[Callable] = None,
default: Optional[Any] = None,
enabled: bool = False,
dependencies: List = [],
dependencies_not: List = [],
exec_func: Optional[Callable] = None,
preview_func: Optional[Callable] = None,
mandatory: bool = False,
no_store: bool = False
):
"""
Create a new menu selection entry
@ -82,6 +81,11 @@ class Selector:
self._preview_func = preview_func
self.mandatory = mandatory
self._no_store = no_store
self._default = default
@property
def default(self) -> Any:
return self._default
@property
def description(self) -> str:
@ -96,7 +100,7 @@ class Selector:
return self._dependencies_not
@property
def current_selection(self):
def current_selection(self) -> Optional[Any]:
return self._current_selection
@property
@ -106,14 +110,14 @@ class Selector:
def do_store(self) -> bool:
return self._no_store is False
def set_enabled(self, status :bool = True):
def set_enabled(self, status: bool = True):
self.enabled = status
def update_description(self, description :str):
def update_description(self, description: str):
self._description = description
def menu_text(self, padding: int = 0) -> str:
if self._description == '': # special menu option for __separator__
if self._description == '': # special menu option for __separator__
return ''
current = ''
@ -134,7 +138,7 @@ class Selector:
return f'{description} {current}'
def set_current_selection(self, current :Optional[str]):
def set_current_selection(self, current: Optional[Any]):
self._current_selection = current
def has_selection(self) -> bool:
@ -158,14 +162,17 @@ class Selector:
def is_mandatory(self) -> bool:
return self.mandatory
def set_mandatory(self, status :bool = True):
self.mandatory = status
if status and not self.is_enabled():
self.set_enabled(True)
def set_mandatory(self, value: bool):
self.mandatory = value
class AbstractMenu:
def __init__(self, data_store: Optional[Dict[str, Any]] = None, auto_cursor=False, preview_size :float = 0.2):
def __init__(
self,
data_store: Dict[str, Any] = {},
auto_cursor: bool = False,
preview_size: float = 0.2
):
"""
Create a new selection menu.
@ -179,25 +186,28 @@ class AbstractMenu:
;type preview_size: float (range 0..1)
"""
self._enabled_order :List[str] = []
self._enabled_order: List[str] = []
self._translation_handler = TranslationHandler()
self.is_context_mgr = False
self._data_store = data_store if data_store is not None else {}
self._data_store = data_store
self.auto_cursor = auto_cursor
self._menu_options: Dict[str, Selector] = {}
self._setup_selection_menu_options()
self.preview_size = preview_size
self._last_choice = None
self.setup_selection_menu_options()
self._sync_all()
self._populate_default_values()
@property
def last_choice(self):
return self._last_choice
def __enter__(self, *args :Any, **kwargs :Any) -> AbstractMenu:
def __enter__(self, *args: Any, **kwargs: Any) -> AbstractMenu:
self.is_context_mgr = True
return self
def __exit__(self, *args :Any, **kwargs :Any) -> None:
def __exit__(self, *args: Any, **kwargs: Any) -> None:
# TODO: https://stackoverflow.com/questions/28157929/how-to-safely-handle-an-exception-inside-a-context-manager
# TODO: skip processing when it comes from a planified exit
if len(args) >= 2 and args[1]:
@ -216,7 +226,50 @@ class AbstractMenu:
def translation_handler(self) -> TranslationHandler:
return self._translation_handler
def _setup_selection_menu_options(self):
def _populate_default_values(self):
for config_key, selector in self._menu_options.items():
if selector.default is not None and config_key not in self._data_store:
self._data_store[config_key] = selector.default
def _sync_all(self):
for key in self._menu_options.keys():
self._sync(key)
def _sync(self, selector_name: str):
value = self._data_store.get(selector_name, None)
selector = self._menu_options.get(selector_name, None)
if value is not None:
self._menu_options[selector_name].set_current_selection(value)
elif selector is not None and selector.has_selection():
self._data_store[selector_name] = selector.current_selection
def _missing_configs(self) -> List[str]:
def check(s):
return self._menu_options.get(s).has_selection()
def has_superuser() -> bool:
sel = self._menu_options['!users']
if sel.current_selection:
return any([u.sudo for u in sel.current_selection])
return False
mandatory_fields = dict(filter(lambda x: x[1].is_mandatory(), self._menu_options.items()))
missing = set()
for key, selector in mandatory_fields.items():
if key in ['!root-password', '!users']:
if not check('!root-password') and not has_superuser():
missing.add(
str(_('Either root-password or at least 1 user with sudo privileges must be specified'))
)
elif key == 'disk_config':
if not check('disk_config'):
missing.add(self._menu_options['disk_config'].description)
return list(missing)
def setup_selection_menu_options(self):
""" Define the menu options.
Menu options can be defined here in a subclass or done per program calling self.set_option()
"""
@ -226,7 +279,7 @@ class AbstractMenu:
""" will be called before each action in the menu """
return
def post_callback(self, selection_name: Optional[str] = None, value: Any = None):
def post_callback(self, selection_name: str, value: Any):
""" will be called after each action in the menu """
return True
@ -234,31 +287,16 @@ class AbstractMenu:
""" will be called at the end of the processing of the menu """
return
def synch(self, selector_name :str, omit_if_set :bool = False,omit_if_disabled :bool = False):
""" loads menu options with data_store value """
arg = self._data_store.get(selector_name, None)
# don't display the menu option if it was defined already
if arg is not None and omit_if_set:
return
if not self.option(selector_name).is_enabled() and omit_if_disabled:
return
if arg is not None:
self._menu_options[selector_name].set_current_selection(arg)
def _update_enabled_order(self, selector_name: str):
self._enabled_order.append(selector_name)
def enable(self, selector_name :str, omit_if_set :bool = False , mandatory :bool = False):
def enable(self, selector_name: str, mandatory: bool = False):
""" activates menu options """
if self._menu_options.get(selector_name, None):
self._menu_options[selector_name].set_enabled(True)
self._update_enabled_order(selector_name)
if mandatory:
self._menu_options[selector_name].set_mandatory(True)
self.synch(selector_name,omit_if_set)
self._menu_options[selector_name].set_mandatory(mandatory)
self._sync(selector_name)
else:
raise ValueError(f'No selector found: {selector_name}')
@ -274,7 +312,11 @@ class AbstractMenu:
def _find_selection(self, selection_name: str) -> Tuple[str, Selector]:
enabled_menus = self._menus_to_enable()
padding = self._get_menu_text_padding(list(enabled_menus.values()))
option = [(k, v) for k, v in self._menu_options.items() if v.menu_text(padding).strip() == selection_name.strip()]
option = []
for k, v in self._menu_options.items():
if v.menu_text(padding).strip() == selection_name.strip():
option.append((k, v))
if len(option) != 1:
raise ValueError(f'Selection not found: {selection_name}')
@ -283,12 +325,7 @@ class AbstractMenu:
return config_name, selector
def run(self, allow_reset: bool = False):
""" Calls the Menu framework"""
# we synch all the options just in case
for item in self.list_options():
self.synch(item)
self.post_callback() # as all the values can vary i have to exec this callback
self._sync_all()
cursor_pos = None
while True:
@ -341,13 +378,13 @@ class AbstractMenu:
break
# we get the last action key
actions = {str(v.description):k for k,v in self._menu_options.items()}
actions = {str(v.description): k for k, v in self._menu_options.items()}
self._last_choice = actions[selection.value.strip()] # type: ignore
if not self.is_context_mgr:
self.__exit__()
def _process_selection(self, selection_name :str) -> bool:
def _process_selection(self, selection_name: str) -> bool:
""" determines and executes the selection y
Can / Should be extended to handle specific selection issues
Returns true if the menu shall continue, False if it has ended
@ -356,7 +393,7 @@ class AbstractMenu:
config_name, selector = self._find_selection(selection_name)
return self.exec_option(config_name, selector)
def exec_option(self, config_name :str, p_selector :Optional[Selector] = None) -> bool:
def exec_option(self, config_name: str, p_selector: Optional[Selector] = None) -> bool:
""" processes the execution of a given menu entry
- pre process callback
- selection function
@ -372,17 +409,21 @@ class AbstractMenu:
self.pre_callback(config_name)
result = None
if selector.func is not None:
presel_val = self.option(config_name).get_selection()
result = selector.func(presel_val)
self._menu_options[config_name].set_current_selection(result)
if selector.do_store():
self._data_store[config_name] = result
exec_ret_val = selector.exec_func(config_name,result) if selector.exec_func is not None else False
self.post_callback(config_name,result)
if exec_ret_val and self._check_mandatory_status():
exec_ret_val = selector.exec_func(config_name, result) if selector.exec_func else False
self.post_callback(config_name, result)
if exec_ret_val:
return False
return True
def _set_kb_language(self):
@ -392,7 +433,7 @@ class AbstractMenu:
if self._data_store.get('keyboard-layout', None) and len(self._data_store['keyboard-layout']):
set_keyboard_language(self._data_store['keyboard-layout'])
def _verify_selection_enabled(self, selection_name :str) -> bool:
def _verify_selection_enabled(self, selection_name: str) -> bool:
""" general """
if selection := self._menu_options.get(selection_name, None):
if not selection.enabled:
@ -429,16 +470,10 @@ class AbstractMenu:
return ordered_menus
def option(self,name :str) -> Selector:
def option(self, name: str) -> Selector:
# TODO check inexistent name
return self._menu_options[name]
def list_options(self) -> Iterator:
""" Iterator to retrieve the enabled menu option names
"""
for item in self._menu_options:
yield item
def list_enabled_options(self) -> Iterator:
""" Iterator to retrieve the enabled menu options at a given time.
The results are dynamic (if between calls to the iterator some elements -still not retrieved- are (de)activated
@ -447,44 +482,21 @@ class AbstractMenu:
if item in self._menus_to_enable():
yield item
def set_option(self, name :str, selector :Selector):
self._menu_options[name] = selector
self.synch(name)
def _check_mandatory_status(self) -> bool:
for field in self._menu_options:
option = self._menu_options[field]
if option.is_mandatory() and not option.has_selection():
return False
return True
def set_mandatory(self, field :str, status :bool):
self.option(field).set_mandatory(status)
def mandatory_overview(self) -> Tuple[int, int]:
mandatory_fields = 0
mandatory_waiting = 0
for field, option in self._menu_options.items():
if option.is_mandatory():
mandatory_fields += 1
if not option.has_selection():
mandatory_waiting += 1
return mandatory_fields, mandatory_waiting
def _select_archinstall_language(self, preset_value: Language) -> Language:
from ..user_interaction.general_conf import select_archinstall_language
language = select_archinstall_language(self.translation_handler.translated_languages, preset_value)
self._translation_handler.activate(language)
return language
class AbstractSubMenu(AbstractMenu):
def __init__(self, data_store: Optional[Dict[str, Any]] = None):
def __init__(self, data_store: Dict[str, Any] = {}):
super().__init__(data_store=data_store)
self._menu_options['__separator__'] = Selector('')
self._menu_options['back'] = \
Selector(
_('Back'),
Menu.back(),
no_store=True,
enabled=True,
exec_func=lambda n, v: True,

429
archinstall/lib/menu/global_menu.py
View File

@ -1,429 +0,0 @@
from __future__ import annotations
from typing import Any, List, Optional, Union, Dict, TYPE_CHECKING
import archinstall
from ..disk.encryption import DiskEncryptionMenu
from ..general import SysCommand, secret
from ..hardware import has_uefi
from ..menu import Menu
from ..menu.abstract_menu import Selector, AbstractMenu
from ..models import NetworkConfiguration
from ..models.disk_encryption import DiskEncryption, EncryptionType
from ..models.users import User
from ..output import FormattedOutput
from ..profiles import is_desktop_profile, Profile
from ..storage import storage
from ..user_interaction import add_number_of_parrallel_downloads
from ..user_interaction import ask_additional_packages_to_install
from ..user_interaction import ask_for_additional_users
from ..user_interaction import ask_for_audio_selection
from ..user_interaction import ask_for_bootloader
from ..user_interaction import ask_for_swap
from ..user_interaction import ask_hostname
from ..user_interaction import ask_ntp
from ..user_interaction import ask_to_configure_network
from ..user_interaction import get_password, ask_for_a_timezone, save_config
from ..user_interaction import select_additional_repositories
from ..user_interaction import select_disk_layout
from ..user_interaction import select_harddrives
from ..user_interaction import select_kernel
from ..user_interaction import select_language
from ..user_interaction import select_locale_enc
from ..user_interaction import select_locale_lang
from ..user_interaction import select_mirror_regions
from ..user_interaction import select_profile
from ..user_interaction.partitioning_conf import current_partition_layout
if TYPE_CHECKING:
_: Any
class GlobalMenu(AbstractMenu):
def __init__(self,data_store):
self._disk_check = True
super().__init__(data_store=data_store, auto_cursor=True, preview_size=0.3)
def _setup_selection_menu_options(self):
# archinstall.Language will not use preset values
self._menu_options['archinstall-language'] = \
Selector(
_('Archinstall language'),
lambda x: self._select_archinstall_language(x),
display_func=lambda x: x.display_name,
default=self.translation_handler.get_language_by_abbr('en'))
self._menu_options['keyboard-layout'] = \
Selector(
_('Keyboard layout'),
lambda preset: select_language(preset),
default='us')
self._menu_options['mirror-region'] = \
Selector(
_('Mirror region'),
lambda preset: select_mirror_regions(preset),
display_func=lambda x: list(x.keys()) if x else '[]',
default={})
self._menu_options['sys-language'] = \
Selector(
_('Locale language'),
lambda preset: select_locale_lang(preset),
default='en_US')
self._menu_options['sys-encoding'] = \
Selector(
_('Locale encoding'),
lambda preset: select_locale_enc(preset),
default='UTF-8')
self._menu_options['harddrives'] = \
Selector(
_('Drive(s)'),
lambda preset: self._select_harddrives(preset),
display_func=lambda x: f'{len(x)} ' + str(_('Drive(s)')) if x is not None and len(x) > 0 else '',
preview_func=self._prev_harddrives,
)
self._menu_options['disk_layouts'] = \
Selector(
_('Disk layout'),
lambda preset: select_disk_layout(
preset,
storage['arguments'].get('harddrives', []),
storage['arguments'].get('advanced', False)
),
preview_func=self._prev_disk_layouts,
display_func=lambda x: self._display_disk_layout(x),
dependencies=['harddrives'])
self._menu_options['disk_encryption'] = \
Selector(
_('Disk encryption'),
lambda preset: self._disk_encryption(preset),
preview_func=self._prev_disk_encryption,
display_func=lambda x: self._display_disk_encryption(x),
dependencies=['disk_layouts'])
self._menu_options['swap'] = \
Selector(
_('Swap'),
lambda preset: ask_for_swap(preset),
default=True)
self._menu_options['bootloader'] = \
Selector(
_('Bootloader'),
lambda preset: ask_for_bootloader(storage['arguments'].get('advanced', False),preset),
default="systemd-bootctl" if has_uefi() else "grub-install")
self._menu_options['hostname'] = \
Selector(
_('Hostname'),
ask_hostname,
default='archlinux')
# root password won't have preset value
self._menu_options['!root-password'] = \
Selector(
_('Root password'),
lambda preset:self._set_root_password(),
display_func=lambda x: secret(x) if x else 'None')
self._menu_options['!users'] = \
Selector(
_('User account'),
lambda x: self._create_user_account(x),
default={},
display_func=lambda x: f'{len(x)} {_("User(s)")}' if len(x) > 0 else None,
preview_func=self._prev_users)
self._menu_options['profile'] = \
Selector(
_('Profile'),
lambda preset: self._select_profile(preset),
display_func=lambda x: x if x else 'None'
)
self._menu_options['audio'] = \
Selector(
_('Audio'),
lambda preset: ask_for_audio_selection(is_desktop_profile(storage['arguments'].get('profile', None)),preset),
display_func=lambda x: x if x else 'None',
default=None
)
self._menu_options['parallel downloads'] = \
Selector(
_('Parallel Downloads'),
add_number_of_parrallel_downloads,
display_func=lambda x: x if x else '0',
default=0
)
self._menu_options['kernels'] = \
Selector(
_('Kernels'),
lambda preset: select_kernel(preset),
default=['linux'])
self._menu_options['packages'] = \
Selector(
_('Additional packages'),
# lambda x: ask_additional_packages_to_install(storage['arguments'].get('packages', None)),
ask_additional_packages_to_install,
default=[])
self._menu_options['additional-repositories'] = \
Selector(
_('Optional repositories'),
select_additional_repositories,
default=[])
self._menu_options['nic'] = \
Selector(
_('Network configuration'),
ask_to_configure_network,
display_func=lambda x: self._display_network_conf(x),
preview_func=self._prev_network_config,
default={})
self._menu_options['timezone'] = \
Selector(
_('Timezone'),
lambda preset: ask_for_a_timezone(preset),
default='UTC')
self._menu_options['ntp'] = \
Selector(
_('Automatic time sync (NTP)'),
lambda preset: self._select_ntp(preset),
default=True)
self._menu_options['__separator__'] = \
Selector('')
self._menu_options['save_config'] = \
Selector(
_('Save configuration'),
lambda preset: save_config(self._data_store),
no_store=True)
self._menu_options['install'] = \
Selector(
self._install_text(),
exec_func=lambda n,v: True if len(self._missing_configs()) == 0 else False,
preview_func=self._prev_install_missing_config,
no_store=True)
self._menu_options['abort'] = Selector(_('Abort'), exec_func=lambda n,v:exit(1))
def _update_install_text(self, name :Optional[str] = None, result :Any = None):
text = self._install_text()
self._menu_options['install'].update_description(text)
def post_callback(self,name :Optional[str] = None ,result :Any = None):
self._update_install_text(name, result)
def _install_text(self):
missing = len(self._missing_configs())
if missing > 0:
return _('Install ({} config(s) missing)').format(missing)
return _('Install')
def _display_network_conf(self, cur_value: Union[NetworkConfiguration, List[NetworkConfiguration]]) -> str:
if not cur_value:
return _('Not configured, unavailable unless setup manually')
else:
if isinstance(cur_value, list):
return str(_('Configured {} interfaces')).format(len(cur_value))
else:
return str(cur_value)
def _disk_encryption(self, preset: Optional[DiskEncryption]) -> Optional[DiskEncryption]:
data_store: Dict[str, Any] = {}
selector = self._menu_options['disk_layouts']
if selector.has_selection():
layouts: Dict[str, Dict[str, Any]] = selector.current_selection
else:
# this should not happen as the encryption menu has the disk layout as dependency
raise ValueError('No disk layout specified')
disk_encryption = DiskEncryptionMenu(data_store, preset, layouts).run()
return disk_encryption
def _prev_network_config(self) -> Optional[str]:
selector = self._menu_options['nic']
if selector.has_selection():
ifaces = selector.current_selection
if isinstance(ifaces, list):
return FormattedOutput.as_table(ifaces)
return None
def _prev_harddrives(self) -> Optional[str]:
selector = self._menu_options['harddrives']
if selector.has_selection():
drives = selector.current_selection
return FormattedOutput.as_table(drives)
return None
def _prev_disk_layouts(self) -> Optional[str]:
selector = self._menu_options['disk_layouts']
if selector.has_selection():
layouts: Dict[str, Dict[str, Any]] = selector.current_selection
output = ''
for device, layout in layouts.items():
output += f'{_("Device")}: {device}\n\n'
output += current_partition_layout(layout['partitions'], with_title=False)
output += '\n\n'
return output.rstrip()
return None
def _display_disk_layout(self, current_value: Optional[Dict[str, Any]]) -> str:
if current_value:
total_partitions = [entry['partitions'] for entry in current_value.values()]
total_nr = sum([len(p) for p in total_partitions])
return f'{total_nr} {_("Partitions")}'
return ''
def _prev_disk_encryption(self) -> Optional[str]:
selector = self._menu_options['disk_encryption']
if selector.has_selection():
encryption: DiskEncryption = selector.current_selection
enc_type = EncryptionType.type_to_text(encryption.encryption_type)
output = str(_('Encryption type')) + f': {enc_type}\n'
output += str(_('Password')) + f': {secret(encryption.encryption_password)}\n'
if encryption.all_partitions:
output += 'Partitions: {} selected'.format(len(encryption.all_partitions)) + '\n'
if encryption.hsm_device:
output += f'HSM: {encryption.hsm_device.manufacturer}'
return output
return None
def _display_disk_encryption(self, current_value: Optional[DiskEncryption]) -> str:
if current_value:
return EncryptionType.type_to_text(current_value.encryption_type)
return ''
def _prev_install_missing_config(self) -> Optional[str]:
if missing := self._missing_configs():
text = str(_('Missing configurations:\n'))
for m in missing:
text += f'- {m}\n'
return text[:-1] # remove last new line
return None
def _prev_users(self) -> Optional[str]:
selector = self._menu_options['!users']
if selector.has_selection():
users: List[User] = selector.current_selection
return FormattedOutput.as_table(users)
return None
def _missing_configs(self) -> List[str]:
def check(s):
return self._menu_options.get(s).has_selection()
def has_superuser() -> bool:
users = self._menu_options['!users'].current_selection
return any([u.sudo for u in users])
missing = []
if not check('bootloader'):
missing += ['Bootloader']
if not check('hostname'):
missing += ['Hostname']
if not check('!root-password') and not has_superuser():
missing += [str(_('Either root-password or at least 1 user with sudo privileges must be specified'))]
if self._disk_check:
if not check('harddrives'):
missing += [str(_('Drive(s)'))]
if check('harddrives'):
if not self._menu_options['harddrives'].is_empty() and not check('disk_layouts'):
missing += [str(_('Disk layout'))]
return missing
def _set_root_password(self) -> Optional[str]:
prompt = str(_('Enter root password (leave blank to disable root): '))
password = get_password(prompt=prompt)
return password
# def _select_encrypted_password(self) -> Optional[str]:
# if passwd := get_password(prompt=str(_('Enter disk encryption password (leave blank for no encryption): '))):
# return passwd
# return None
def _select_ntp(self, preset :bool = True) -> bool:
ntp = ask_ntp(preset)
value = str(ntp).lower()
SysCommand(f'timedatectl set-ntp {value}')
return ntp
def _select_harddrives(self, old_harddrives: List[str] = []) -> List:
harddrives = select_harddrives(old_harddrives)
if harddrives is not None:
if len(harddrives) == 0:
prompt = _(
"You decided to skip harddrive selection\nand will use whatever drive-setup is mounted at {} (experimental)\n"
"WARNING: Archinstall won't check the suitability of this setup\n"
"Do you wish to continue?"
).format(storage['MOUNT_POINT'])
choice = Menu(prompt, Menu.yes_no(), default_option=Menu.yes(), skip=False).run()
if choice.value == Menu.no():
self._disk_check = True
return self._select_harddrives(old_harddrives)
else:
self._disk_check = False
# in case the harddrives got changed we have to reset the disk layout as well
if old_harddrives != harddrives:
self._menu_options['disk_layouts'].set_current_selection(None)
storage['arguments']['disk_layouts'] = {}
return harddrives
def _select_profile(self, preset) -> Optional[Profile]:
ret: Optional[Profile] = None
profile = select_profile(preset)
if profile is None:
if any([
archinstall.storage.get('profile_minimal', False),
archinstall.storage.get('_selected_servers', None),
archinstall.storage.get('_desktop_profile', None),
archinstall.arguments.get('desktop-environment', None),
archinstall.arguments.get('gfx_driver_packages', None)
]):
return preset
else: # ctrl+c was actioned and all profile settings have been reset
return None
servers = archinstall.storage.get('_selected_servers', [])
desktop = archinstall.storage.get('_desktop_profile', None)
desktop_env = archinstall.arguments.get('desktop-environment', None)
gfx_driver = archinstall.arguments.get('gfx_driver_packages', None)
# Check the potentially selected profiles preparations to get early checks if some additional questions are needed.
if profile and profile.has_prep_function():
namespace = f'{profile.namespace}.py'
with profile.load_instructions(namespace=namespace) as imported:
if imported._prep_function(servers=servers, desktop=desktop, desktop_env=desktop_env, gfx_driver=gfx_driver):
ret = profile
match ret.name:
case 'minimal':
reset = ['_selected_servers', '_desktop_profile', 'desktop-environment', 'gfx_driver_packages']
case 'server':
reset = ['_desktop_profile', 'desktop-environment']
case 'desktop':
reset = ['_selected_servers']
case 'xorg':
reset = ['_selected_servers', '_desktop_profile', 'desktop-environment']
for r in reset:
archinstall.storage[r] = None
else:
return self._select_profile(preset)
elif profile:
ret = profile
return ret
def _create_user_account(self, defined_users: List[User]) -> List[User]:
users = ask_for_additional_users(defined_users=defined_users)
return users

46
archinstall/lib/menu/list_manager.py
View File

@ -34,7 +34,7 @@ class ListManager:
self._data = copy.deepcopy(entries)
explainer = str(_('\n Choose an object from the list, and select one of the available actions for it to execute'))
self._prompt = prompt + explainer if prompt else explainer
self._prompt = prompt if prompt else explainer
self._separator = ''
self._confirm_action = str(_('Confirm and exit'))
@ -44,13 +44,18 @@ class ListManager:
self._base_actions = base_actions
self._sub_menu_actions = sub_menu_actions
self._last_choice = None
self._last_choice: Optional[str] = None
@property
def last_choice(self):
def last_choice(self) -> Optional[str]:
return self._last_choice
def run(self):
def is_last_choice_cancel(self) -> bool:
if self._last_choice is not None:
return self._last_choice == self._cancel_action
return False
def run(self) -> List[Any]:
while True:
# this will return a dictionary with the key as the menu entry to be displayed
# and the value is the original value from the self._data container
@ -76,10 +81,11 @@ class ListManager:
elif choice.value in self._terminate_actions:
break
else: # an entry of the existing selection was choosen
selected_entry = data_formatted[choice.value]
selected_entry = data_formatted[choice.value] # type: ignore
self._run_actions_on_entry(selected_entry)
self._last_choice = choice
self._last_choice = choice.value # type: ignore
if choice.value == self._cancel_action:
return self._original_data # return the original list
else:
@ -122,21 +128,29 @@ class ListManager:
self._data = self.handle_action(choice.value, entry, self._data)
def selected_action_display(self, selection: Any) -> str:
# this will return the value to be displayed in the
# "Select an action for '{}'" string
"""
this will return the value to be displayed in the
"Select an action for '{}'" string
"""
raise NotImplementedError('Please implement me in the child class')
def reformat(self, data: List[Any]) -> Dict[str, Any]:
# this should return a dictionary of display string to actual data entry
# mapping; if the value for a given display string is None it will be used
# in the header value (useful when displaying tables)
def reformat(self, data: List[Any]) -> Dict[str, Optional[Any]]:
"""
this should return a dictionary of display string to actual data entry
mapping; if the value for a given display string is None it will be used
in the header value (useful when displaying tables)
"""
raise NotImplementedError('Please implement me in the child class')
def handle_action(self, action: Any, entry: Optional[Any], data: List[Any]) -> List[Any]:
# this function is called when a base action or
# a specific action for an entry is triggered
"""
this function is called when a base action or
a specific action for an entry is triggered
"""
raise NotImplementedError('Please implement me in the child class')
def filter_options(self, selection :Any, options :List[str]) -> List[str]:
# filter which actions to show for an specific selection
def filter_options(self, selection: Any, options: List[str]) -> List[str]:
"""
filter which actions to show for an specific selection
"""
return options

114
archinstall/lib/menu/menu.py
View File

@ -3,7 +3,7 @@ from enum import Enum, auto
from os import system
from typing import Dict, List, Union, Any, TYPE_CHECKING, Optional, Callable
from .simple_menu import TerminalMenu
from simple_term_menu import TerminalMenu
from ..exceptions import RequirementError
from ..output import log
@ -27,42 +27,56 @@ class MenuSelection:
type_: MenuSelectionType
value: Optional[Union[str, List[str]]] = None
@property
def single_value(self) -> Any:
return self.value
@property
def multi_value(self) -> List[Any]:
return self.value
class Menu(TerminalMenu):
@classmethod
def yes(cls):
def back(cls) -> str:
return str(_('← Back'))
@classmethod
def yes(cls) -> str:
return str(_('yes'))
@classmethod
def no(cls):
def no(cls) -> str:
return str(_('no'))
@classmethod
def yes_no(cls):
def yes_no(cls) -> List[str]:
return [cls.yes(), cls.no()]
def __init__(
self,
title :str,
p_options :Union[List[str], Dict[str, Any]],
skip :bool = True,
multi :bool = False,
default_option : Optional[str] = None,
sort :bool = True,
preset_values :Union[str, List[str]] = None,
cursor_index : Optional[int] = None,
title: str,
p_options: Union[List[str], Dict[str, Any]],
skip: bool = True,
multi: bool = False,
default_option: Optional[str] = None,
sort: bool = True,
preset_values: Optional[Union[str, List[str]]] = None,
cursor_index: Optional[int] = None,
preview_command: Optional[Callable] = None,
preview_size: float = 0.0,
preview_title: str = 'Info',
header :Union[List[str],str] = None,
allow_reset :bool = False,
allow_reset_warning_msg :str = '',
header: Union[List[str],str] = None,
allow_reset: bool = False,
allow_reset_warning_msg: Optional[str] = None,
clear_screen: bool = True,
show_search_hint: bool = True,
cycle_cursor: bool = True,
clear_menu_on_exit: bool = True,
skip_empty_entries: bool = False
skip_empty_entries: bool = False,
display_back_option: bool = False,
extra_bottom_space: bool = False
):
"""
Creates a new menu
@ -72,7 +86,7 @@ class Menu(TerminalMenu):
:param p_options: Options to be displayed in the menu to chose from;
if dict is specified then the keys of such will be used as options
:type options: list, dict
:type p_options: list, dict
:param skip: Indicate if the selection is not mandatory and can be skipped
:type skip: bool
@ -101,16 +115,17 @@ class Menu(TerminalMenu):
:param preview_title: Title of the preview window
:type preview_title: str
param header: one or more header lines for the menu
type param: string or list
:param header: one or more header lines for the menu
:type header: string or list
param raise_error_on_interrupt: This will explicitly handle a ctrl+c instead and return that specific state
type param: bool
:param allow_reset: This will explicitly handle a ctrl+c instead and return that specific state
:type allow_reset: bool
param raise_error_warning_msg: If raise_error_on_interrupt is True and this is non-empty, there will be a warning with a user confirmation displayed
type param: str
param allow_reset_warning_msg: If raise_error_on_interrupt is True the warnign is set, a user confirmation is displayed
type allow_reset_warning_msg: str
:param kwargs : any SimpleTerminal parameter
:param extra_bottom_space: Add an extra empty line at the end of the menu
:type extra_bottom_space: bool
"""
# we guarantee the inmutability of the options outside the class.
# an unknown number of iterables (.keys(),.values(),generator,...) can't be directly copied, in this case
@ -152,7 +167,6 @@ class Menu(TerminalMenu):
self._multi = multi
self._raise_error_on_interrupt = allow_reset
self._raise_error_warning_msg = allow_reset_warning_msg
self._preview_command = preview_command
action_info = ''
if skip:
@ -182,6 +196,14 @@ class Menu(TerminalMenu):
default = f'{default_option} {self._default_str}'
self._menu_options = [default] + [o for o in self._menu_options if default_option != o]
if display_back_option and not multi and skip:
skip_empty_entries = True
self._menu_options += ['', self.back()]
if extra_bottom_space:
skip_empty_entries = True
self._menu_options += ['']
self._preselection(preset_values,cursor_index)
cursor = "> "
@ -194,13 +216,10 @@ class Menu(TerminalMenu):
menu_cursor=cursor,
menu_cursor_style=main_menu_cursor_style,
menu_highlight_style=main_menu_style,
# cycle_cursor=True,
# clear_screen=True,
multi_select=multi,
# show_search_hint=True,
preselected_entries=self.preset_values,
cursor_index=self.cursor_index,
preview_command=lambda x: self._preview_wrapper(preview_command, x),
preview_command=lambda x: self._show_preview(preview_command, x),
preview_size=preview_size,
preview_title=preview_title,
raise_error_on_interrupt=self._raise_error_on_interrupt,
@ -212,6 +231,17 @@ class Menu(TerminalMenu):
skip_empty_entries=skip_empty_entries
)
def _show_preview(self, preview_command: Optional[Callable], selection: str) -> Optional[str]:
if selection == self.back():
return None
if preview_command:
if self._default_option is not None and f'{self._default_option} {self._default_str}' == selection:
selection = self._default_option
return preview_command(selection)
return None
def _show(self) -> MenuSelection:
try:
idx = self.show()
@ -225,39 +255,37 @@ class Menu(TerminalMenu):
return elem
if idx is not None:
if isinstance(idx, (list, tuple)):
if isinstance(idx, (list, tuple)): # on multi selection
results = []
for i in idx:
option = check_default(self._menu_options[i])
results.append(option)
return MenuSelection(type_=MenuSelectionType.Selection, value=results)
else:
else: # on single selection
result = check_default(self._menu_options[idx])
return MenuSelection(type_=MenuSelectionType.Selection, value=result)
else:
return MenuSelection(type_=MenuSelectionType.Skip)
def _preview_wrapper(self, preview_command: Optional[Callable], current_selection: str) -> Optional[str]:
if preview_command:
if self._default_option is not None and f'{self._default_option} {self._default_str}' == current_selection:
current_selection = self._default_option
return preview_command(current_selection)
return None
def run(self) -> MenuSelection:
ret = self._show()
selection = self._show()
if ret.type_ == MenuSelectionType.Reset:
if self._raise_error_on_interrupt and len(self._raise_error_warning_msg) > 0:
if selection.type_ == MenuSelectionType.Reset:
if self._raise_error_on_interrupt and self._raise_error_warning_msg is not None:
response = Menu(self._raise_error_warning_msg, Menu.yes_no(), skip=False).run()
if response.value == Menu.no():
return self.run()
elif ret.type_ is MenuSelectionType.Skip:
elif selection.type_ is MenuSelectionType.Skip:
if not self._skip:
system('clear')
return self.run()
return ret
if selection.type_ == MenuSelectionType.Selection:
if selection.value == self.back():
selection.type_ = MenuSelectionType.Skip
selection.value = None
return selection
def set_cursor_pos(self,pos :int):
if pos and 0 < pos < len(self._menu_entries):

2002
archinstall/lib/menu/simple_menu.py
View File

File diff suppressed because it is too large Load Diff

68
archinstall/lib/menu/table_selection_menu.py
View File

@ -1,19 +1,24 @@
from typing import Any, Tuple, List, Dict, Optional
from typing import Any, Tuple, List, Dict, Optional, Callable
from .menu import MenuSelectionType, MenuSelection
from .menu import MenuSelectionType, MenuSelection, Menu
from ..output import FormattedOutput
from ..menu import Menu
class TableMenu(Menu):
def __init__(
self,
title: str,
data: List[Any] = [],
data: Optional[List[Any]] = None,
table_data: Optional[Tuple[List[Any], str]] = None,
preset: List[Any] = [],
custom_menu_options: List[str] = [],
default: Any = None,
multi: bool = False
multi: bool = False,
preview_command: Optional[Callable] = None,
preview_title: str = 'Info',
preview_size: float = 0.0,
allow_reset: bool = True,
allow_reset_warning_msg: Optional[str] = None,
):
"""
param title: Text that will be displayed above the menu
@ -29,10 +34,10 @@ class TableMenu(Menu):
param custom_options: List of custom options that will be displayed under the table
:type custom_menu_options: List
"""
if not data and not table_data:
raise ValueError('Either "data" or "table_data" must be provided')
:param preview_command: A function that should return a string that will be displayed in a preview window when a menu selection item is in focus
:type preview_command: Callable
"""
self._custom_options = custom_menu_options
self._multi = multi
@ -41,7 +46,7 @@ class TableMenu(Menu):
else:
header_padding = 2
if len(data):
if data is not None:
table_text = FormattedOutput.as_table(data)
rows = table_text.split('\n')
table = self._create_table(data, rows, header_padding=header_padding)
@ -53,20 +58,53 @@ class TableMenu(Menu):
data = table_data[0]
rows = table_data[1].split('\n')
table = self._create_table(data, rows, header_padding=header_padding)
else:
raise ValueError('Either "data" or "table_data" must be provided')
self._options, header = self._prepare_selection(table)
preset_values = self._preset_values(preset)
extra_bottom_space = True if preview_command else False
super().__init__(
title,
self._options,
preset_values=preset_values,
header=header,
skip_empty_entries=True,
show_search_hint=False,
allow_reset=True,
multi=multi,
default_option=default
default_option=default,
preview_command=lambda x: self._table_show_preview(preview_command, x),
preview_size=preview_size,
preview_title=preview_title,
extra_bottom_space=extra_bottom_space,
allow_reset=allow_reset,
allow_reset_warning_msg=allow_reset_warning_msg
)
def _preset_values(self, preset: List[Any]) -> List[str]:
# when we create the table of just the preset values it will
# be formatted a bit different due to spacing, so to determine
# correct rows lets remove all the spaces and compare apples with apples
preset_table = FormattedOutput.as_table(preset).strip()
data_rows = preset_table.split('\n')[2:] # get all data rows
pure_data_rows = [self._escape_row(row.replace(' ', '')) for row in data_rows]
# the actual preset value has to be in non-escaped form
pure_option_rows = {o.replace(' ', ''): self._unescape_row(o) for o in self._options.keys()}
preset_rows = [row for pure, row in pure_option_rows.items() if pure in pure_data_rows]
return preset_rows
def _table_show_preview(self, preview_command: Optional[Callable], selection: Any) -> Optional[str]:
if preview_command:
row = self._escape_row(selection)
obj = self._options[row]
return preview_command(obj)
return None
def run(self) -> MenuSelection:
choice = super().run()
@ -79,6 +117,12 @@ class TableMenu(Menu):
return choice
def _escape_row(self, row: str) -> str:
return row.replace('|', '\\|')
def _unescape_row(self, row: str) -> str:
return row.replace('\\|', '|')
def _create_table(self, data: List[Any], rows: List[str], header_padding: int = 2) -> Dict[str, Any]:
# these are the header rows of the table and do not map to any data obviously
# we're adding 2 spaces as prefix because the menu selector '> ' will be put before
@ -87,7 +131,7 @@ class TableMenu(Menu):
display_data = {f'{padding}{rows[0]}': None, f'{padding}{rows[1]}': None}
for row, entry in zip(rows[2:], data):
row = row.replace('|', '\\|')
row = self._escape_row(row)
display_data[row] = entry
return display_data

4
archinstall/lib/mirrors.py
View File

@ -2,7 +2,7 @@ import logging
import pathlib
import urllib.error
import urllib.request
from typing import Union, Mapping, Iterable, Dict, Any, List
from typing import Union, Iterable, Dict, Any, List
from .general import SysCommand
from .output import log
@ -121,7 +121,7 @@ def insert_mirrors(mirrors :Dict[str, Any], *args :str, **kwargs :str) -> bool:
def use_mirrors(
regions: Mapping[str, Iterable[str]],
regions: Dict[str, Iterable[str]],
destination: str = '/etc/pacman.d/mirrorlist'
) -> None:
log(f'A new package mirror-list has been created: {destination}', level=logging.INFO)

5
archinstall/lib/models/__init__.py
View File

@ -1 +1,4 @@
from .network_configuration import NetworkConfiguration as NetworkConfiguration
from .network_configuration import NetworkConfiguration, NicType, NetworkConfigurationHandler
from .bootloader import Bootloader
from .gen import VersionDef, PackageSearchResult, PackageSearch, LocalPackage
from .users import PasswordStrength, User

40
archinstall/lib/models/bootloader.py Normal file
View File

@ -0,0 +1,40 @@
from __future__ import annotations
import logging
import sys
from enum import Enum
from typing import List
from ..hardware import has_uefi
from ..output import log
class Bootloader(Enum):
Systemd = 'Systemd-boot'
Grub = 'Grub'
Efistub = 'Efistub'
def json(self):
return self.value
@classmethod
def values(cls) -> List[str]:
return [e.value for e in cls]
@classmethod
def get_default(cls) -> Bootloader:
if has_uefi():
return Bootloader.Systemd
else:
return Bootloader.Grub
@classmethod
def from_arg(cls, bootloader: str) -> Bootloader:
# to support old configuration files
bootloader = bootloader.capitalize()
if bootloader not in cls.values():
values = ', '.join(cls.values())
log(f'Invalid bootloader value "{bootloader}". Allowed values: {values}', level=logging.WARN)
sys.exit(1)
return Bootloader(bootloader)

90
archinstall/lib/models/disk_encryption.py
View File

@ -1,90 +0,0 @@
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import Optional, List, Dict, TYPE_CHECKING, Any
from ..hsm.fido import Fido2Device
if TYPE_CHECKING:
_: Any
class EncryptionType(Enum):
Partition = 'partition'
@classmethod
def _encryption_type_mapper(cls) -> Dict[str, 'EncryptionType']:
return {
# str(_('Full disk encryption')): EncryptionType.FullDiskEncryption,
str(_('Partition encryption')): EncryptionType.Partition
}
@classmethod
def text_to_type(cls, text: str) -> 'EncryptionType':
mapping = cls._encryption_type_mapper()
return mapping[text]
@classmethod
def type_to_text(cls, type_: 'EncryptionType') -> str:
mapping = cls._encryption_type_mapper()
type_to_text = {type_: text for text, type_ in mapping.items()}
return type_to_text[type_]
@dataclass
class DiskEncryption:
encryption_type: EncryptionType = EncryptionType.Partition
encryption_password: str = ''
partitions: Dict[str, List[Dict[str, Any]]] = field(default_factory=dict)
hsm_device: Optional[Fido2Device] = None
@property
def all_partitions(self) -> List[Dict[str, Any]]:
_all: List[Dict[str, Any]] = []
for parts in self.partitions.values():
_all += parts
return _all
def generate_encryption_file(self, partition) -> bool:
return partition in self.all_partitions and partition['mountpoint'] != '/'
def json(self) -> Dict[str, Any]:
obj = {
'encryption_type': self.encryption_type.value,
'partitions': self.partitions
}
if self.hsm_device:
obj['hsm_device'] = self.hsm_device.json()
return obj
@classmethod
def parse_arg(
cls,
disk_layout: Dict[str, Any],
arg: Dict[str, Any],
password: str = ''
) -> 'DiskEncryption':
# we have to map the enc partition config to the disk layout objects
# they both need to point to the same object as it will get modified
# during the installation process
enc_partitions: Dict[str, List[Dict[str, Any]]] = {}
for path, partitions in disk_layout.items():
conf_partitions = arg['partitions'].get(path, [])
for part in partitions['partitions']:
if part in conf_partitions:
enc_partitions.setdefault(path, []).append(part)
enc = DiskEncryption(
EncryptionType(arg['encryption_type']),
password,
enc_partitions
)
if hsm := arg.get('hsm_device', None):
enc.hsm_device = Fido2Device.parse_arg(hsm)
return enc

48
archinstall/lib/models/dataclasses.py → archinstall/lib/models/gen.py
View File

@ -1,16 +1,17 @@
from dataclasses import dataclass
from typing import Optional, List
@dataclass
class VersionDef:
version_string: str
@classmethod
def parse_version(self) -> List[str]:
if '.' in self.version_string:
versions = self.version_string.split('.')
def parse_version(cls) -> List[str]:
if '.' in cls.version_string:
versions = cls.version_string.split('.')
else:
versions = [self.version_string]
versions = [cls.version_string]
return versions
@ -19,37 +20,44 @@ class VersionDef:
return self.parse_version()[0]
@classmethod
def minor(self) -> str:
versions = self.parse_version()
def minor(cls) -> Optional[str]:
versions = cls.parse_version()
if len(versions) >= 2:
return versions[1]
return None
@classmethod
def patch(self) -> str:
versions = self.parse_version()
def patch(cls) -> Optional[str]:
versions = cls.parse_version()
if '-' in versions[-1]:
_, patch_version = versions[-1].split('-', 1)
return patch_version
def __eq__(self, other :'VersionDef') -> bool:
return None
def __eq__(self, other) -> bool:
if other.major == self.major and \
other.minor == self.minor and \
other.patch == self.patch:
return True
return False
def __lt__(self, other :'VersionDef') -> bool:
if self.major > other.major:
def __lt__(self, other) -> bool:
if self.major() > other.major():
return False
elif self.minor and other.minor and self.minor > other.minor:
elif self.minor() and other.minor() and self.minor() > other.minor():
return False
elif self.patch and other.patch and self.patch > other.patch:
elif self.patch() and other.patch() and self.patch() > other.patch():
return False
return True
def __str__(self) -> str:
return self.version_string
@dataclass
class PackageSearchResult:
pkgname: str
@ -83,12 +91,13 @@ class PackageSearchResult:
def pkg_version(self) -> str:
return self.pkgver
def __eq__(self, other :'VersionDef') -> bool:
def __eq__(self, other) -> bool:
return self.pkg_version == other.pkg_version
def __lt__(self, other :'VersionDef') -> bool:
def __lt__(self, other) -> bool:
return self.pkg_version < other.pkg_version
@dataclass
class PackageSearch:
version: int
@ -101,6 +110,7 @@ class PackageSearch:
def __post_init__(self):
self.results = [PackageSearchResult(**x) for x in self.results]
@dataclass
class LocalPackage:
name: str
@ -129,8 +139,8 @@ class LocalPackage:
def pkg_version(self) -> str:
return self.version
def __eq__(self, other :'VersionDef') -> bool:
def __eq__(self, other) -> bool:
return self.pkg_version == other.pkg_version
def __lt__(self, other :'VersionDef') -> bool:
return self.pkg_version < other.pkg_version
def __lt__(self, other) -> bool:
return self.pkg_version < other.pkg_version

2
archinstall/lib/models/network_configuration.py
View File

@ -93,7 +93,7 @@ class NetworkConfigurationHandler:
enable_services=True) # Sources the ISO network configuration to the install medium.
elif self._configuration.is_network_manager():
installation.add_additional_packages(["networkmanager"])
if (profile := storage['arguments'].get('profile')) and profile.is_desktop_profile:
if (profile := storage['arguments'].get('profile_config')) and profile.is_desktop_type_profile:
installation.add_additional_packages(["network-manager-applet"])
installation.enable_service('NetworkManager.service')

85
archinstall/lib/models/password_strength.py
View File

@ -1,85 +0,0 @@
from enum import Enum
class PasswordStrength(Enum):
VERY_WEAK = 'very weak'
WEAK = 'weak'
MODERATE = 'moderate'
STRONG = 'strong'
@property
def value(self):
match self:
case PasswordStrength.VERY_WEAK: return str(_('very weak'))
case PasswordStrength.WEAK: return str(_('weak'))
case PasswordStrength.MODERATE: return str(_('moderate'))
case PasswordStrength.STRONG: return str(_('strong'))
def color(self):
match self:
case PasswordStrength.VERY_WEAK: return 'red'
case PasswordStrength.WEAK: return 'red'
case PasswordStrength.MODERATE: return 'yellow'
case PasswordStrength.STRONG: return 'green'
@classmethod
def strength(cls, password: str) -> 'PasswordStrength':
digit = any(character.isdigit() for character in password)
upper = any(character.isupper() for character in password)
lower = any(character.islower() for character in password)
symbol = any(not character.isalnum() for character in password)
return cls._check_password_strength(digit, upper, lower, symbol, len(password))
@classmethod
def _check_password_strength(
cls,
digit: bool,
upper: bool,
lower: bool,
symbol: bool,
length: int
) -> 'PasswordStrength':
# suggested evaluation
# https://github.com/archlinux/archinstall/issues/1304#issuecomment-1146768163
if digit and upper and lower and symbol:
match length:
case num if 13 <= num:
return PasswordStrength.STRONG
case num if 11 <= num <= 12:
return PasswordStrength.MODERATE
case num if 7 <= num <= 10:
return PasswordStrength.WEAK
case num if num <= 6:
return PasswordStrength.VERY_WEAK
elif digit and upper and lower:
match length:
case num if 14 <= num:
return PasswordStrength.STRONG
case num if 11 <= num <= 13:
return PasswordStrength.MODERATE
case num if 7 <= num <= 10:
return PasswordStrength.WEAK
case num if num <= 6:
return PasswordStrength.VERY_WEAK
elif upper and lower:
match length:
case num if 15 <= num:
return PasswordStrength.STRONG
case num if 12 <= num <= 14:
return PasswordStrength.MODERATE
case num if 7 <= num <= 11:
return PasswordStrength.WEAK
case num if num <= 6:
return PasswordStrength.VERY_WEAK
elif lower or upper:
match length:
case num if 18 <= num:
return PasswordStrength.STRONG
case num if 14 <= num <= 17:
return PasswordStrength.MODERATE
case num if 9 <= num <= 13:
return PasswordStrength.WEAK
case num if num <= 8:
return PasswordStrength.VERY_WEAK
return PasswordStrength.VERY_WEAK

134
archinstall/lib/models/pydantic.py
View File

@ -1,134 +0,0 @@
from typing import Optional, List
from pydantic import BaseModel
"""
This python file is not in use.
Pydantic is not a builtin, and we use the dataclasses.py instead!
"""
class VersionDef(BaseModel):
version_string: str
@classmethod
def parse_version(self) -> List[str]:
if '.' in self.version_string:
versions = self.version_string.split('.')
else:
versions = [self.version_string]
return versions
@classmethod
def major(self) -> str:
return self.parse_version()[0]
@classmethod
def minor(self) -> str:
versions = self.parse_version()
if len(versions) >= 2:
return versions[1]
@classmethod
def patch(self) -> str:
versions = self.parse_version()
if '-' in versions[-1]:
_, patch_version = versions[-1].split('-', 1)
return patch_version
def __eq__(self, other :'VersionDef') -> bool:
if other.major == self.major and \
other.minor == self.minor and \
other.patch == self.patch:
return True
return False
def __lt__(self, other :'VersionDef') -> bool:
if self.major > other.major:
return False
elif self.minor and other.minor and self.minor > other.minor:
return False
elif self.patch and other.patch and self.patch > other.patch:
return False
def __str__(self) -> str:
return self.version_string
class PackageSearchResult(BaseModel):
pkgname: str
pkgbase: str
repo: str
arch: str
pkgver: str
pkgrel: str
epoch: int
pkgdesc: str
url: str
filename: str
compressed_size: int
installed_size: int
build_date: str
last_update: str
flag_date: Optional[str]
maintainers: List[str]
packager: str
groups: List[str]
licenses: List[str]
conflicts: List[str]
provides: List[str]
replaces: List[str]
depends: List[str]
optdepends: List[str]
makedepends: List[str]
checkdepends: List[str]
@property
def pkg_version(self) -> str:
return self.pkgver
def __eq__(self, other :'VersionDef') -> bool:
return self.pkg_version == other.pkg_version
def __lt__(self, other :'VersionDef') -> bool:
return self.pkg_version < other.pkg_version
class PackageSearch(BaseModel):
version: int
limit: int
valid: bool
results: List[PackageSearchResult]
class LocalPackage(BaseModel):
name: str
version: str
description:str
architecture: str
url: str
licenses: str
groups: str
depends_on: str
optional_deps: str
required_by: str
optional_for: str
conflicts_with: str
replaces: str
installed_size: str
packager: str
build_date: str
install_date: str
install_reason: str
install_script: str
validated_by: str
@property
def pkg_version(self) -> str:
return self.version
def __eq__(self, other :'VersionDef') -> bool:
return self.pkg_version == other.pkg_version
def __lt__(self, other :'VersionDef') -> bool:
return self.pkg_version < other.pkg_version

68
archinstall/lib/models/subvolume.py
View File

@ -1,68 +0,0 @@
from dataclasses import dataclass
from typing import List, Any, Dict
@dataclass
class Subvolume:
name: str
mountpoint: str
compress: bool = False
nodatacow: bool = False
def display(self) -> str:
options_str = ','.join(self.options)
return f'{_("Subvolume")}: {self.name:15} {_("Mountpoint")}: {self.mountpoint:20} {_("Options")}: {options_str}'
@property
def options(self) -> List[str]:
options = [
'compress' if self.compress else '',
'nodatacow' if self.nodatacow else ''
]
return [o for o in options if len(o)]
def json(self) -> Dict[str, Any]:
return {
'name': self.name,
'mountpoint': self.mountpoint,
'compress': self.compress,
'nodatacow': self.nodatacow
}
@classmethod
def _parse(cls, config_subvolumes: List[Dict[str, Any]]) -> List['Subvolume']:
subvolumes = []
for entry in config_subvolumes:
if not entry.get('name', None) or not entry.get('mountpoint', None):
continue
subvolumes.append(
Subvolume(
entry['name'],
entry['mountpoint'],
entry.get('compress', False),
entry.get('nodatacow', False)
)
)
return subvolumes
@classmethod
def _parse_backwards_compatible(cls, config_subvolumes) -> List['Subvolume']:
subvolumes = []
for name, mountpoint in config_subvolumes.items():
if not name or not mountpoint:
continue
subvolumes.append(Subvolume(name, mountpoint))
return subvolumes
@classmethod
def parse_arguments(cls, config_subvolumes: Any) -> List['Subvolume']:
if isinstance(config_subvolumes, list):
return cls._parse(config_subvolumes)
elif isinstance(config_subvolumes, dict):
return cls._parse_backwards_compatible(config_subvolumes)
raise ValueError('Unknown disk layout btrfs subvolume format')

94
archinstall/lib/models/users.py
View File

@ -1,12 +1,95 @@
from dataclasses import dataclass
from typing import Dict, List, Union, Any, TYPE_CHECKING
from .password_strength import PasswordStrength
from enum import Enum
if TYPE_CHECKING:
_: Any
class PasswordStrength(Enum):
VERY_WEAK = 'very weak'
WEAK = 'weak'
MODERATE = 'moderate'
STRONG = 'strong'
@property
def value(self):
match self:
case PasswordStrength.VERY_WEAK: return str(_('very weak'))
case PasswordStrength.WEAK: return str(_('weak'))
case PasswordStrength.MODERATE: return str(_('moderate'))
case PasswordStrength.STRONG: return str(_('strong'))
def color(self):
match self:
case PasswordStrength.VERY_WEAK: return 'red'
case PasswordStrength.WEAK: return 'red'
case PasswordStrength.MODERATE: return 'yellow'
case PasswordStrength.STRONG: return 'green'
@classmethod
def strength(cls, password: str) -> 'PasswordStrength':
digit = any(character.isdigit() for character in password)
upper = any(character.isupper() for character in password)
lower = any(character.islower() for character in password)
symbol = any(not character.isalnum() for character in password)
return cls._check_password_strength(digit, upper, lower, symbol, len(password))
@classmethod
def _check_password_strength(
cls,
digit: bool,
upper: bool,
lower: bool,
symbol: bool,
length: int
) -> 'PasswordStrength':
# suggested evaluation
# https://github.com/archlinux/archinstall/issues/1304#issuecomment-1146768163
if digit and upper and lower and symbol:
match length:
case num if 13 <= num:
return PasswordStrength.STRONG
case num if 11 <= num <= 12:
return PasswordStrength.MODERATE
case num if 7 <= num <= 10:
return PasswordStrength.WEAK
case num if num <= 6:
return PasswordStrength.VERY_WEAK
elif digit and upper and lower:
match length:
case num if 14 <= num:
return PasswordStrength.STRONG
case num if 11 <= num <= 13:
return PasswordStrength.MODERATE
case num if 7 <= num <= 10:
return PasswordStrength.WEAK
case num if num <= 6:
return PasswordStrength.VERY_WEAK
elif upper and lower:
match length:
case num if 15 <= num:
return PasswordStrength.STRONG
case num if 12 <= num <= 14:
return PasswordStrength.MODERATE
case num if 7 <= num <= 11:
return PasswordStrength.WEAK
case num if num <= 6:
return PasswordStrength.VERY_WEAK
elif lower or upper:
match length:
case num if 18 <= num:
return PasswordStrength.STRONG
case num if 14 <= num <= 17:
return PasswordStrength.MODERATE
case num if 9 <= num <= 13:
return PasswordStrength.WEAK
case num if num <= 8:
return PasswordStrength.VERY_WEAK
return PasswordStrength.VERY_WEAK
@dataclass
class User:
username: str
@ -26,13 +109,6 @@ class User:
'sudo': self.sudo
}
def display(self) -> str:
password = '*' * (len(self.password) if self.password else 0)
if password:
strength = PasswordStrength.strength(self.password)
password += f' ({strength.value})'
return f'{_("Username")}: {self.username:16} {_("Password")}: {password:20} sudo: {str(self.sudo)}'
@classmethod
def _parse(cls, config_users: List[Dict[str, Any]]) -> List['User']:
users = []

33
archinstall/lib/networking.py
View File

@ -1,8 +1,12 @@
import logging
import os
import socket
import ssl
import struct
from typing import Union, Dict, Any, List
from typing import Union, Dict, Any, List, Optional
from urllib.error import URLError
from urllib.parse import urlencode
from urllib.request import urlopen
from .exceptions import HardwareIncompatibilityError, SysCallError
from .general import SysCommand
@ -39,7 +43,7 @@ def check_mirror_reachable() -> bool:
elif os.geteuid() != 0:
log("check_mirror_reachable() uses 'pacman -Sy' which requires root.", level=logging.ERROR, fg="red")
except SysCallError as err:
log(err, level=logging.DEBUG)
log(f'exit_code: {err.exit_code}, Error: {err.message}', level=logging.DEBUG)
return False
@ -75,12 +79,8 @@ def enrich_iface_types(interfaces: Union[Dict[str, Any], List[str]]) -> Dict[str
return result
def get_interface_from_mac(mac :str) -> str:
return list_interfaces().get(mac.lower(), None)
def wireless_scan(interface :str) -> None:
interfaces = enrich_iface_types(list_interfaces().values())
interfaces = enrich_iface_types(list(list_interfaces().values()))
if interfaces[interface] != 'WIRELESS':
raise HardwareIncompatibilityError(f"Interface {interface} is not a wireless interface: {interfaces}")
@ -107,3 +107,22 @@ def get_wireless_networks(interface :str) -> None:
for line in SysCommand(f"iwctl station {interface} get-networks"):
print(line)
def fetch_data_from_url(url: str, params: Optional[Dict] = None) -> str:
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
if params is not None:
encoded = urlencode(params)
full_url = f'{url}?{encoded}'
else:
full_url = url
try:
response = urlopen(full_url, context=ssl_context)
data = response.read().decode('UTF-8')
return data
except URLError:
raise ValueError(f'Unable to fetch data from url: {url}')

47
archinstall/lib/output.py
View File

@ -2,7 +2,7 @@ import logging
import os
import sys
from pathlib import Path
from typing import Dict, Union, List, Any, Callable
from typing import Dict, Union, List, Any, Callable, Optional
from .storage import storage
from dataclasses import asdict, is_dataclass
@ -11,7 +11,12 @@ from dataclasses import asdict, is_dataclass
class FormattedOutput:
@classmethod
def values(cls, o: Any, class_formatter: str = None, filter_list: List[str] = None) -> Dict[str, Any]:
def values(
cls,
o: Any,
class_formatter: Optional[Union[str, Callable]] = None,
filter_list: List[str] = []
) -> Dict[str, Any]:
""" the original values returned a dataclass as dict thru the call to some specific methods
this version allows thru the parameter class_formatter to call a dynamicly selected formatting method.
Can transmit a filter list to the class_formatter,
@ -25,7 +30,8 @@ class FormattedOutput:
elif hasattr(o, class_formatter) and callable(getattr(o, class_formatter)):
func = getattr(o, class_formatter)
return func(filter_list)
# kept as to make it backward compatible
raise ValueError('Unsupported formatting call')
elif hasattr(o, 'as_json'):
return o.as_json()
elif hasattr(o, 'json'):
@ -36,7 +42,13 @@ class FormattedOutput:
return o.__dict__
@classmethod
def as_table(cls, obj: List[Any], class_formatter: Union[str, Callable] = None, filter_list: List[str] = None) -> str:
def as_table(
cls,
obj: List[Any],
class_formatter: Optional[Union[str, Callable]] = None,
filter_list: List[str] = [],
capitalize: bool = False
) -> str:
""" variant of as_table (subtly different code) which has two additional parameters
filter which is a list of fields which will be shon
class_formatter a special method to format the outgoing data
@ -46,6 +58,7 @@ class FormattedOutput:
As_table_filter can be a drop in replacement for as_table
"""
raw_data = [cls.values(o, class_formatter, filter_list) for o in obj]
# determine the maximum column size
column_width: Dict[str, int] = {}
for o in raw_data:
@ -55,14 +68,20 @@ class FormattedOutput:
column_width[k] = max([column_width[k], len(str(v)), len(k)])
if not filter_list:
filter_list = (column_width.keys())
filter_list = list(column_width.keys())
# create the header lines
output = ''
key_list = []
for key in filter_list:
width = column_width[key]
key = key.replace('!', '')
key = key.replace('!', '').replace('_', ' ')
if capitalize:
key = key.capitalize()
key_list.append(key.ljust(width))
output += ' | '.join(key_list) + '\n'
output += '-' * len(output) + '\n'
@ -82,6 +101,20 @@ class FormattedOutput:
return output
@classmethod
def as_columns(cls, entries: List[str], cols: int) -> str:
chunks = []
output = ''
for i in range(0, len(entries), cols):
chunks.append(entries[i:i + cols])
for row in chunks:
out_fmt = '{: <30} ' * len(row)
output += out_fmt.format(*row) + '\n'
return output
class Journald:
@staticmethod
@ -204,6 +237,6 @@ def log(*args :str, **kwargs :Union[str, int, Dict[str, Union[str, int]]]) -> No
# Finally, print the log unless we skipped it based on level.
# We use sys.stdout.write()+flush() instead of print() to try and
# fix issue #94
if kwargs.get('level', logging.INFO) != logging.DEBUG or storage['arguments'].get('verbose', False):
if kwargs.get('level', logging.INFO) != logging.DEBUG or storage.get('arguments', {}).get('verbose', False):
sys.stdout.write(f"{string}\n")
sys.stdout.flush()

4
archinstall/lib/packages/__init__.py
View File

@ -0,0 +1,4 @@
from .packages import (
group_search, package_search, find_package,
find_packages, validate_package_list, installed_package
)

4
archinstall/lib/packages/packages.py
View File

@ -7,7 +7,7 @@ from urllib.parse import urlencode
from urllib.request import urlopen
from ..exceptions import PackageError, SysCallError
from ..models.dataclasses import PackageSearch, PackageSearchResult, LocalPackage
from ..models.gen import PackageSearch, PackageSearchResult, LocalPackage
from ..pacman import run_pacman
BASE_URL_PKG_SEARCH = 'https://archlinux.org/packages/search/json/'
@ -113,4 +113,4 @@ def installed_package(package :str) -> LocalPackage:
except SysCallError:
pass
return LocalPackage({field.name: package_info.get(field.name) for field in dataclasses.fields(LocalPackage)})
return LocalPackage({field.name: package_info.get(field.name) for field in dataclasses.fields(LocalPackage)}) # type: ignore

4
archinstall/lib/pacman.py
View File

@ -1,10 +1,14 @@
import logging
import pathlib
import time
from typing import TYPE_CHECKING, Any
from .general import SysCommand
from .output import log
if TYPE_CHECKING:
_: Any
def run_pacman(args :str, default_cmd :str = 'pacman') -> SysCommand:
"""

0
archinstall/lib/profile/__init__.py Normal file
View File

203
archinstall/lib/profile/profile_menu.py Normal file
View File

@ -0,0 +1,203 @@
from __future__ import annotations
from typing import TYPE_CHECKING, Any, Optional, Dict
from archinstall.default_profiles.profile import Profile, GreeterType
from .profile_model import ProfileConfiguration
from ..hardware import AVAILABLE_GFX_DRIVERS
from ..menu import Menu, MenuSelectionType, AbstractSubMenu, Selector
from ..user_interaction.system_conf import select_driver
if TYPE_CHECKING:
_: Any
class ProfileMenu(AbstractSubMenu):
def __init__(
self,
data_store: Dict[str, Any],
preset: Optional[ProfileConfiguration] = None
):
if preset:
self._preset = preset
else:
self._preset = ProfileConfiguration()
super().__init__(data_store=data_store)
def setup_selection_menu_options(self):
self._menu_options['profile'] = Selector(
_('Profile'),
lambda x: self._select_profile(x),
display_func=lambda x: x.name if x else None,
preview_func=self._preview_profile,
default=self._preset.profile,
enabled=True
)
self._menu_options['gfx_driver'] = Selector(
_('Graphics driver'),
lambda preset: self._select_gfx_driver(preset),
display_func=lambda x: x if x else None,
dependencies=['profile'],
default=self._preset.gfx_driver if self._preset.profile and self._preset.profile.is_graphic_driver_supported() else None,
enabled=self._preset.profile.is_graphic_driver_supported() if self._preset.profile else False
)
self._menu_options['greeter'] = Selector(
_('Greeter'),
lambda preset: select_greeter(self._menu_options['profile'].current_selection, preset),
display_func=lambda x: x.value if x else None,
dependencies=['profile'],
default=self._preset.greeter if self._preset.profile and self._preset.profile.is_greeter_supported() else None,
enabled=self._preset.profile.is_greeter_supported() if self._preset.profile else False
)
def run(self, allow_reset: bool = True) -> Optional[ProfileConfiguration]:
super().run(allow_reset=allow_reset)
if self._data_store.get('profile', None):
return ProfileConfiguration(
self._menu_options['profile'].current_selection,
self._menu_options['gfx_driver'].current_selection,
self._menu_options['greeter'].current_selection
)
return None
def _select_profile(self, preset: Optional[Profile]) -> Optional[Profile]:
profile = select_profile(preset)
if profile is not None:
if not profile.is_graphic_driver_supported():
self._menu_options['gfx_driver'].set_enabled(False)
self._menu_options['gfx_driver'].set_current_selection(None)
else:
self._menu_options['gfx_driver'].set_enabled(True)
self._menu_options['gfx_driver'].set_current_selection('All open-source (default)')
if not profile.is_greeter_supported():
self._menu_options['greeter'].set_enabled(False)
self._menu_options['greeter'].set_current_selection(None)
else:
self._menu_options['greeter'].set_enabled(True)
self._menu_options['greeter'].set_current_selection(profile.default_greeter_type)
else:
self._menu_options['gfx_driver'].set_current_selection(None)
self._menu_options['greeter'].set_current_selection(None)
return profile
def _select_gfx_driver(self, preset: Optional[str] = None) -> Optional[str]:
driver = preset
profile: Optional[Profile] = self._menu_options['profile'].current_selection
if profile:
if profile.is_graphic_driver_supported():
driver = select_driver(current_value=preset)
if driver and 'Sway' in profile.current_selection_names():
packages = AVAILABLE_GFX_DRIVERS[driver]
if packages and "nvidia" in packages:
prompt = str(
_('The proprietary Nvidia driver is not supported by Sway. It is likely that you will run into issues, are you okay with that?'))
choice = Menu(prompt, Menu.yes_no(), default_option=Menu.no(), skip=False).run()
if choice.value == Menu.no():
return None
return driver
def _preview_profile(self) -> Optional[str]:
profile: Optional[Profile] = self._menu_options['profile'].current_selection
if profile:
names = profile.current_selection_names()
return '\n'.join(names)
return None
def select_greeter(
profile: Optional[Profile] = None,
preset: Optional[GreeterType] = None
) -> Optional[GreeterType]:
if not profile or profile.is_greeter_supported():
title = str(_('Please chose which greeter to install'))
greeter_options = [greeter.value for greeter in GreeterType]
default: Optional[GreeterType] = None
if preset is not None:
default = preset
elif profile is not None:
default_greeter = profile.default_greeter_type
default = default_greeter if default_greeter else None
choice = Menu(
title,
greeter_options,
skip=True,
default_option=default.value if default else None
).run()
match choice.type_:
case MenuSelectionType.Skip:
return default
return GreeterType(choice.single_value)
return None
def select_profile(
current_profile: Optional[Profile] = None,
title: Optional[str] = None,
allow_reset: bool = True,
multi: bool = False
) -> Optional[Profile]:
from archinstall.lib.profile.profiles_handler import profile_handler
top_level_profiles = profile_handler.get_top_level_profiles()
display_title = title
if not display_title:
display_title = str(_('This is a list of pre-programmed default_profiles'))
choice = profile_handler.select_profile(
top_level_profiles,
current_profile=current_profile,
title=display_title,
allow_reset=allow_reset,
multi=multi
)
match choice.type_:
case MenuSelectionType.Selection:
profile_selection: Profile = choice.single_value
select_result = profile_selection.do_on_select()
if not select_result:
return select_profile(
current_profile=current_profile,
title=title,
allow_reset=allow_reset,
multi=multi
)
# we're going to reset the currently selected profile(s) to avoid
# any stale data laying around
match select_result:
case select_result.NewSelection:
profile_handler.reset_top_level_profiles(exclude=[profile_selection])
current_profile = profile_selection
case select_result.ResetCurrent:
profile_handler.reset_top_level_profiles()
current_profile = None
case select_result.SameSelection:
pass
return current_profile
case MenuSelectionType.Reset:
return None
case MenuSelectionType.Skip:
return current_profile

35
archinstall/lib/profile/profile_model.py Normal file
View File

@ -0,0 +1,35 @@
from __future__ import annotations
from dataclasses import dataclass
from typing import TYPE_CHECKING, Any, Optional, Dict
from archinstall.default_profiles.profile import Profile, GreeterType
if TYPE_CHECKING:
_: Any
@dataclass
class ProfileConfiguration:
profile: Optional[Profile] = None
gfx_driver: Optional[str] = None
greeter: Optional[GreeterType] = None
def json(self) -> Dict[str, Any]:
from .profiles_handler import profile_handler
return {
'profile': profile_handler.to_json(self.profile),
'gfx_driver': self.gfx_driver,
'greeter': self.greeter.value if self.greeter else None
}
@classmethod
def parse_arg(cls, arg: Dict[str, Any]) -> 'ProfileConfiguration':
from .profiles_handler import profile_handler
greeter = arg.get('greeter', None)
return ProfileConfiguration(
profile_handler.parse_profile_config(arg['profile']),
arg.get('gfx_driver', None),
GreeterType(greeter) if greeter else None
)

391
archinstall/lib/profile/profiles_handler.py Normal file
View File

@ -0,0 +1,391 @@
from __future__ import annotations
import importlib.util
import logging
import sys
from collections import Counter
from functools import cached_property
from pathlib import Path
from tempfile import NamedTemporaryFile
from types import ModuleType
from typing import List, TYPE_CHECKING, Any, Optional, Dict, Union
from archinstall.default_profiles.profile import Profile, TProfile, GreeterType
from .profile_model import ProfileConfiguration
from ..hardware import AVAILABLE_GFX_DRIVERS
from ..menu import MenuSelectionType, Menu, MenuSelection
from ..networking import list_interfaces, fetch_data_from_url
from ..output import log
from ..storage import storage
if TYPE_CHECKING:
from ..installer import Installer
_: Any
class ProfileHandler:
def __init__(self):
self._profiles_path: Path = storage['PROFILE']
self._profiles = None
# special variable to keep track of a profile url configuration
# it is merely used to be able to export the path again when a user
# wants to save the configuration
self._url_path = None
def to_json(self, profile: Optional[Profile]) -> Dict[str, Any]:
"""
Serialize the selected profile setting to JSON
"""
data: Dict[str, Any] = {}
if profile is not None:
data = {
'main': profile.name,
'details': [profile.name for profile in profile.current_selection],
}
if self._url_path is not None:
data['path'] = self._url_path
return data
def parse_profile_config(self, profile_config: Dict[str, Any]) -> Optional[Profile]:
"""
Deserialize JSON configuration
"""
profile = None
# the order of these is important, we want to
# load all the default_profiles from url and custom
# so that we can then apply whatever was specified
# in the main/detail sections
if url_path := profile_config.get('path', None):
self._url_path = url_path
local_path = Path(url_path)
if local_path.is_file():
profiles = self._process_profile_file(local_path)
self.remove_custom_profiles(profiles)
self.add_custom_profiles(profiles)
else:
self._import_profile_from_url(url_path)
if custom := profile_config.get('custom', None):
from archinstall.default_profiles.custom import CustomTypeProfile
custom_types = []
for entry in custom:
custom_types.append(
CustomTypeProfile(
entry['name'],
entry['enabled'],
entry.get('packages', []),
entry.get('services', [])
)
)
self.remove_custom_profiles(custom_types)
self.add_custom_profiles(custom_types)
# this doesn't mean it's actual going to be set as a selection
# but we are simply populating the custom profile with all
# possible custom definitions
if custom_profile := self.get_profile_by_name('Custom'):
custom_profile.set_current_selection(custom_types)
if main := profile_config.get('main', None):
profile = self.get_profile_by_name(main) if main else None
valid: List[Profile] = []
if details := profile_config.get('details', []):
resolved = {detail: self.get_profile_by_name(detail) for detail in details if detail}
valid = [p for p in resolved.values() if p is not None]
invalid = ', '.join([k for k, v in resolved.items() if v is None])
if invalid:
log(f'No profile definition found: {invalid}')
if profile is not None:
profile.set_current_selection(valid)
return profile
@property
def profiles(self) -> List[Profile]:
"""
List of all available default_profiles
"""
if self._profiles is None:
self._profiles = self._find_available_profiles()
return self._profiles
@cached_property
def _local_mac_addresses(self) -> List[str]:
ifaces = list_interfaces()
return list(ifaces.keys())
def add_custom_profiles(self, profiles: Union[TProfile, List[TProfile]]):
if not isinstance(profiles, list):
profiles = [profiles]
for profile in profiles:
self._profiles.append(profile)
self._verify_unique_profile_names(self._profiles)
def remove_custom_profiles(self, profiles: Union[TProfile, List[TProfile]]):
if not isinstance(profiles, list):
profiles = [profiles]
remove_names = [p.name for p in profiles]
self._profiles = [p for p in self._profiles if p.name not in remove_names]
def get_profile_by_name(self, name: str) -> Optional[Profile]:
return next(filter(lambda x: x.name == name, self.profiles), None) # type: ignore
def get_top_level_profiles(self) -> List[Profile]:
return list(filter(lambda x: x.is_top_level_profile(), self.profiles))
def get_server_profiles(self) -> List[Profile]:
return list(filter(lambda x: x.is_server_type_profile(), self.profiles))
def get_desktop_profiles(self) -> List[Profile]:
return list(filter(lambda x: x.is_desktop_type_profile(), self.profiles))
def get_custom_profiles(self) -> List[Profile]:
return list(filter(lambda x: x.is_custom_type_profile(), self.profiles))
def get_mac_addr_profiles(self) -> List[Profile]:
tailored = list(filter(lambda x: x.is_tailored(), self.profiles))
match_mac_addr_profiles = list(filter(lambda x: x.name in self._local_mac_addresses, tailored))
return match_mac_addr_profiles
def install_greeter(self, install_session: 'Installer', greeter: GreeterType):
packages = []
service = None
match greeter:
case GreeterType.Lightdm:
packages = ['lightdm', 'lightdm-gtk-greeter']
service = ['lightdm']
case GreeterType.Sddm:
packages = ['sddm']
service = ['sddm']
case GreeterType.Gdm:
packages = ['gdm']
service = ['gdm']
if packages:
install_session.add_additional_packages(packages)
if service:
install_session.enable_service(service)
def install_gfx_driver(self, install_session: 'Installer', driver: str):
try:
driver_pkgs = AVAILABLE_GFX_DRIVERS[driver] if driver else []
additional_pkg = ' '.join(['xorg-server', 'xorg-xinit'] + driver_pkgs)
if driver is not None:
if 'nvidia' in driver:
if "linux-zen" in install_session.base_packages or "linux-lts" in install_session.base_packages:
for kernel in install_session.kernels:
# Fixes https://github.com/archlinux/archinstall/issues/585
install_session.add_additional_packages(f"{kernel}-headers")
# I've had kernel regen fail if it wasn't installed before nvidia-dkms
install_session.add_additional_packages("dkms xorg-server xorg-xinit nvidia-dkms")
return
elif 'amdgpu' in driver_pkgs:
# The order of these two are important if amdgpu is installed #808
if 'amdgpu' in install_session.MODULES:
install_session.MODULES.remove('amdgpu')
install_session.MODULES.append('amdgpu')
if 'radeon' in install_session.MODULES:
install_session.MODULES.remove('radeon')
install_session.MODULES.append('radeon')
install_session.add_additional_packages(additional_pkg)
except Exception as err:
log(f"Could not handle nvidia and linuz-zen specific situations during xorg installation: {err}", level=logging.WARNING, fg="yellow")
# Prep didn't run, so there's no driver to install
install_session.add_additional_packages("xorg-server xorg-xinit")
def install_profile_config(self, install_session: 'Installer', profile_config: ProfileConfiguration):
profile = profile_config.profile
if profile:
profile.install(install_session)
if profile and profile_config.gfx_driver:
if profile.is_xorg_type_profile() or profile.is_desktop_type_profile():
self.install_gfx_driver(install_session, profile_config.gfx_driver)
if profile and profile_config.greeter:
self.install_greeter(install_session, profile_config.greeter)
def _import_profile_from_url(self, url: str):
"""
Import default_profiles from a url path
"""
try:
data = fetch_data_from_url(url)
b_data = bytes(data, 'utf-8')
with NamedTemporaryFile(delete=False, suffix='.py') as fp:
fp.write(b_data)
filepath = Path(fp.name)
profiles = self._process_profile_file(filepath)
self.remove_custom_profiles(profiles)
self.add_custom_profiles(profiles)
except ValueError:
err = str(_('Unable to fetch profile from specified url: {}')).format(url)
log(err, level=logging.ERROR, fg="red")
def _load_profile_class(self, module: ModuleType) -> List[Profile]:
"""
Load all default_profiles defined in a module
"""
profiles = []
for k, v in module.__dict__.items():
if isinstance(v, type) and v.__module__ == module.__name__:
try:
cls_ = v()
if isinstance(cls_, Profile):
profiles.append(cls_)
except Exception:
log(f'Cannot import {module}, it does not appear to be a Profile class', level=logging.DEBUG)
return profiles
def _verify_unique_profile_names(self, profiles: List[Profile]):
"""
All profile names have to be unique, this function will verify
that the provided list contains only default_profiles with unique names
"""
counter = Counter([p.name for p in profiles])
duplicates = list(filter(lambda x: x[1] != 1, counter.items()))
if len(duplicates) > 0:
err = str(_('Profiles must have unique name, but profile definitions with duplicate name found: {}')).format(duplicates[0][0])
log(err, level=logging.ERROR, fg="red")
sys.exit(1)
def _is_legacy(self, file: Path) -> bool:
"""
Check if the provided profile file contains a
legacy profile definition
"""
with open(file, 'r') as fp:
for line in fp.readlines():
if '__packages__' in line:
return True
return False
def _process_profile_file(self, file: Path) -> List[Profile]:
"""
Process a file for profile definitions
"""
if self._is_legacy(file):
log(f'Cannot import {file} because it is no longer supported, please use the new profile format')
return []
if not file.is_file():
log(f'Cannot find profile file {file}')
return []
name = file.name.removesuffix(file.suffix)
log(f'Importing profile: {file}', level=logging.DEBUG)
try:
spec = importlib.util.spec_from_file_location(name, file)
if spec is not None:
imported = importlib.util.module_from_spec(spec)
if spec.loader is not None:
spec.loader.exec_module(imported)
return self._load_profile_class(imported)
except Exception as e:
log(f'Unable to parse file {file}: {e}', level=logging.ERROR)
return []
def _find_available_profiles(self) -> List[Profile]:
"""
Search the profile path for profile definitions
"""
profiles = []
for file in self._profiles_path.glob('**/*.py'):
# ignore the abstract default_profiles class
if 'profile.py' in file.name:
continue
profiles += self._process_profile_file(file)
self._verify_unique_profile_names(profiles)
return profiles
def reset_top_level_profiles(self, exclude: List[Profile] = []):
"""
Reset all top level profile configurations, this is usually necessary
when a new top level profile is selected
"""
excluded_profiles = [p.name for p in exclude]
for profile in self.get_top_level_profiles():
if profile.name not in excluded_profiles:
profile.reset()
def select_profile(
self,
selectable_profiles: List[Profile],
current_profile: Optional[Union[TProfile, List[TProfile]]] = None,
title: str = '',
allow_reset: bool = True,
multi: bool = False,
) -> MenuSelection:
"""
Helper function to perform a profile selection
"""
options = {p.name: p for p in selectable_profiles}
warning = str(_('Are you sure you want to reset this setting?'))
preset_value: Optional[Union[str, List[str]]] = None
if current_profile is not None:
if isinstance(current_profile, list):
preset_value = [p.name for p in current_profile]
else:
preset_value = current_profile.name
choice = Menu(
title=title,
preset_values=preset_value,
p_options=options,
allow_reset=allow_reset,
allow_reset_warning_msg=warning,
multi=multi,
sort=True,
preview_command=self.preview_text,
preview_size=0.5
).run()
if choice.type_ == MenuSelectionType.Selection:
value = choice.value
if multi:
# this is quite dirty and should eb switched to a
# dedicated return type instead
choice.value = [options[val] for val in value] # type: ignore
else:
choice.value = options[value] # type: ignore
return choice
def preview_text(self, selection: str) -> Optional[str]:
"""
Callback for preview display on profile selection
"""
profile = self.get_profile_by_name(selection)
return profile.preview_text() if profile is not None else None
profile_handler = ProfileHandler()

340
archinstall/lib/profiles.py
View File

@ -1,340 +0,0 @@
from __future__ import annotations
import hashlib
import importlib.util
import json
import os
import re
import ssl
import sys
import urllib.error
import urllib.parse
import urllib.request
from typing import Optional, Dict, Union, TYPE_CHECKING, Any
from types import ModuleType
# https://stackoverflow.com/a/39757388/929999
if TYPE_CHECKING:
from .installer import Installer
_: Any
from .general import multisplit
from .networking import list_interfaces
from .storage import storage
from .exceptions import ProfileNotFound
def grab_url_data(path :str) -> str:
safe_path = path[: path.find(':') + 1] + ''.join([item if item in ('/', '?', '=', '&') else urllib.parse.quote(item) for item in multisplit(path[path.find(':') + 1:], ('/', '?', '=', '&'))])
ssl_context = ssl.create_default_context()
ssl_context.check_hostname = False
ssl_context.verify_mode = ssl.CERT_NONE
response = urllib.request.urlopen(safe_path, context=ssl_context)
return response.read() # bytes?
def is_desktop_profile(profile :str) -> bool:
if str(profile) == 'Profile(desktop)':
return True
desktop_profile = Profile(None, "desktop")
with open(desktop_profile.path, 'r') as source:
source_data = source.read()
if '__name__' in source_data and '__supported__' in source_data:
with desktop_profile.load_instructions(namespace=f"{desktop_profile.namespace}.py") as imported:
if hasattr(imported, '__supported__'):
desktop_profiles = imported.__supported__
return str(profile) in [f"Profile({s})" for s in desktop_profiles]
return False
def list_profiles(
filter_irrelevant_macs :bool = True,
subpath :str = '',
filter_top_level_profiles :bool = False
) -> Dict[str, Dict[str, Union[str, bool]]]:
# TODO: Grab from github page as well, not just local static files
if filter_irrelevant_macs:
local_macs = list_interfaces()
cache = {}
# Grab all local profiles found in PROFILE_PATH
for PATH_ITEM in storage['PROFILE_PATH']:
for root, folders, files in os.walk(os.path.abspath(os.path.expanduser(PATH_ITEM + subpath))):
for file in files:
if file == '__init__.py':
continue
if os.path.splitext(file)[1] == '.py':
tailored = False
if len(mac := re.findall('(([a-zA-z0-9]{2}[-:]){5}([a-zA-z0-9]{2}))', file)):
if filter_irrelevant_macs and mac[0][0].lower() not in local_macs:
continue
tailored = True
description = ''
with open(os.path.join(root, file), 'r') as fh:
first_line = fh.readline()
if len(first_line) and first_line[0] == '#':
description = first_line[1:].strip()
cache[file[:-3]] = {'path': os.path.join(root, file), 'description': description, 'tailored': tailored}
break
# Grab profiles from upstream URL
if storage['PROFILE_DB']:
profiles_url = os.path.join(storage["UPSTREAM_URL"] + subpath, storage['PROFILE_DB'])
try:
profile_list = json.loads(grab_url_data(profiles_url))
except urllib.error.HTTPError as err:
print(_('Error: Listing profiles on URL "{}" resulted in:').format(profiles_url), err)
return cache
except json.decoder.JSONDecodeError as err:
print(_('Error: Could not decode "{}" result as JSON:').format(profiles_url), err)
return cache
for profile in profile_list:
if os.path.splitext(profile)[1] == '.py':
tailored = False
if len(mac := re.findall('(([a-zA-z0-9]{2}[-:]){5}([a-zA-z0-9]{2}))', profile)):
if filter_irrelevant_macs and mac[0][0].lower() not in local_macs:
continue
tailored = True
cache[profile[:-3]] = {'path': os.path.join(storage["UPSTREAM_URL"] + subpath, profile), 'description': profile_list[profile], 'tailored': tailored}
if filter_top_level_profiles:
for profile in list(cache.keys()):
if Profile(None, profile).is_top_level_profile() is False:
del cache[profile]
return cache
class Script:
def __init__(self, profile :str, installer :Optional[Installer] = None):
"""
:param profile: A string representing either a boundled profile, a local python file
or a remote path (URL) to a python script-profile. Three examples:
* profile: https://archlinux.org/some_profile.py
* profile: desktop
* profile: /path/to/profile.py
"""
self.profile = profile
self.installer = installer # TODO: Appears not to be used anymore?
self.converted_path = None
self.spec = None
self.examples = {}
self.namespace = os.path.splitext(os.path.basename(self.path))[0]
self.original_namespace = self.namespace
def __enter__(self, *args :str, **kwargs :str) -> ModuleType:
self.execute()
return sys.modules[self.namespace]
def __exit__(self, *args :str, **kwargs :str) -> None:
# TODO: https://stackoverflow.com/questions/28157929/how-to-safely-handle-an-exception-inside-a-context-manager
if len(args) >= 2 and args[1]:
raise args[1]
if self.original_namespace:
self.namespace = self.original_namespace
def localize_path(self, profile_path :str) -> str:
if (url := urllib.parse.urlparse(profile_path)).scheme and url.scheme in ('https', 'http'):
if not self.converted_path:
self.converted_path = f"/tmp/{os.path.basename(self.profile).replace('.py', '')}_{hashlib.md5(os.urandom(12)).hexdigest()}.py"
with open(self.converted_path, "w") as temp_file:
temp_file.write(urllib.request.urlopen(url.geturl()).read().decode('utf-8'))
return self.converted_path
else:
return profile_path
@property
def path(self) -> str:
parsed_url = urllib.parse.urlparse(self.profile)
# The Profile was not a direct match on a remote URL
if not parsed_url.scheme:
# Try to locate all local or known URL's
if not self.examples:
self.examples = list_profiles()
if f"{self.profile}" in self.examples:
return self.localize_path(self.examples[self.profile]['path'])
# TODO: Redundant, the below block shouldn't be needed as profiles are stripped of their .py, but just in case for now:
elif f"{self.profile}.py" in self.examples:
return self.localize_path(self.examples[f"{self.profile}.py"]['path'])
# Path was not found in any known examples, check if it's an absolute path
if os.path.isfile(self.profile):
return self.profile
raise ProfileNotFound(f"File {self.profile} does not exist in {storage['PROFILE_PATH']}")
elif parsed_url.scheme in ('https', 'http'):
return self.localize_path(self.profile)
else:
raise ProfileNotFound(f"Cannot handle scheme {parsed_url.scheme}")
def load_instructions(self, namespace :Optional[str] = None) -> 'Script':
if namespace:
self.namespace = namespace
self.spec = importlib.util.spec_from_file_location(self.namespace, self.path)
imported = importlib.util.module_from_spec(self.spec)
sys.modules[self.namespace] = imported
return self
def execute(self) -> ModuleType:
if self.namespace not in sys.modules or self.spec is None:
self.load_instructions()
self.spec.loader.exec_module(sys.modules[self.namespace])
return sys.modules[self.namespace]
class Profile(Script):
def __init__(self, installer :Optional[Installer], path :str):
super(Profile, self).__init__(path, installer)
def __dump__(self, *args :str, **kwargs :str) -> Dict[str, str]:
return {'path': self.path}
def __repr__(self, *args :str, **kwargs :str) -> str:
return f'Profile({os.path.basename(self.profile)})'
@property
def name(self) -> str:
return os.path.basename(self.profile)
@property
def is_desktop_profile(self) -> bool:
return is_desktop_profile(repr(self))
def install(self) -> ModuleType:
# Before installing, revert any temporary changes to the namespace.
# This ensures that the namespace during installation is the original initiation namespace.
# (For instance awesome instead of aweosme.py or app-awesome.py)
self.namespace = self.original_namespace
return self.execute()
def has_prep_function(self) -> bool:
with open(self.path, 'r') as source:
source_data = source.read()
# Some crude safety checks, make sure the imported profile has
# a __name__ check and if so, check if it's got a _prep_function()
# we can call to ask for more user input.
#
# If the requirements are met, import with .py in the namespace to not
# trigger a traditional:
# if __name__ == 'moduleName'
if '__name__' in source_data and '_prep_function' in source_data:
with self.load_instructions(namespace=f"{self.namespace}.py") as imported:
if hasattr(imported, '_prep_function'):
return True
return False
def has_post_install(self) -> bool:
with open(self.path, 'r') as source:
source_data = source.read()
# Some crude safety checks, make sure the imported profile has
# a __name__ check and if so, check if it's got a _prep_function()
# we can call to ask for more user input.
#
# If the requirements are met, import with .py in the namespace to not
# trigger a traditional:
# if __name__ == 'moduleName'
if '__name__' in source_data and '_post_install' in source_data:
with self.load_instructions(namespace=f"{self.namespace}.py") as imported:
if hasattr(imported, '_post_install'):
return True
def is_top_level_profile(self) -> bool:
with open(self.path, 'r') as source:
source_data = source.read()
if '__name__' in source_data and 'is_top_level_profile' in source_data:
with self.load_instructions(namespace=f"{self.namespace}.py") as imported:
if hasattr(imported, 'is_top_level_profile'):
return imported.is_top_level_profile
# Default to True if nothing is specified,
# since developers like less code - omitting it should assume they want to present it.
return True
def get_profile_description(self) -> str:
with open(self.path, 'r') as source:
source_data = source.read()
if '__description__' in source_data:
with self.load_instructions(namespace=f"{self.namespace}.py") as imported:
if hasattr(imported, '__description__'):
return imported.__description__
# Default to this string if the profile does not have a description.
return "This profile does not have the __description__ attribute set."
@property
def packages(self) -> Optional[list]:
"""
Returns a list of packages baked into the profile definition.
If no package definition has been done, .packages() will return None.
"""
with open(self.path, 'r') as source:
source_data = source.read()
# Some crude safety checks, make sure the imported profile has
# a __name__ check before importing.
#
# If the requirements are met, import with .py in the namespace to not
# trigger a traditional:
# if __name__ == 'moduleName'
if '__name__' in source_data and '__packages__' in source_data:
with self.load_instructions(namespace=f"{self.namespace}.py") as imported:
if hasattr(imported, '__packages__'):
return imported.__packages__
return None
class Application(Profile):
def __repr__(self, *args :str, **kwargs :str):
return f'Application({os.path.basename(self.profile)})'
@property
def path(self) -> str:
parsed_url = urllib.parse.urlparse(self.profile)
# The Profile was not a direct match on a remote URL
if not parsed_url.scheme:
# Try to locate all local or known URL's
if not self.examples:
self.examples = list_profiles(subpath='/applications')
if f"{self.profile}" in self.examples:
return self.localize_path(self.examples[self.profile]['path'])
# TODO: Redundant, the below block shouldn't be needed as profiles are stripped of their .py, but just in case for now:
elif f"{self.profile}.py" in self.examples:
return self.localize_path(self.examples[f"{self.profile}.py"]['path'])
# Path was not found in any known examples, check if it's an absolute path
if os.path.isfile(self.profile):
return os.path.basename(self.profile)
raise ProfileNotFound(f"Application file {self.profile} does not exist in {storage['PROFILE_PATH']}")
elif parsed_url.scheme in ('https', 'http'):
return self.localize_path(self.profile)
else:
raise ProfileNotFound(f"Application cannot handle scheme {parsed_url.scheme}")
def install(self) -> ModuleType:
# Before installing, revert any temporary changes to the namespace.
# This ensures that the namespace during installation is the original initiation namespace.
# (For instance awesome instead of aweosme.py or app-awesome.py)
self.namespace = self.original_namespace
return self.execute()

19
archinstall/lib/storage.py
View File

@ -1,26 +1,19 @@
import os
# There's a few scenarios of execution:
# 1. In the git repository, where ./profiles/ exist
# 1. In the git repository, where ./profiles_bck/ exist
# 2. When executing from a remote directory, but targeted a script that starts from the git repository
# 3. When executing as a python -m archinstall module where profiles exist one step back for library reasons.
# 3. When executing as a python -m archinstall module where profiles_bck exist one step back for library reasons.
# (4. Added the ~/.config directory as an additional option for future reasons)
#
# And Keeping this in dict ensures that variables are shared across imports.
from typing import Any, Dict
from pathlib import Path
storage: Dict[str, Any] = {
'PROFILE_PATH': [
'./profiles',
'~/.config/archinstall/profiles',
os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), 'profiles'),
# os.path.abspath(f'{os.path.dirname(__file__)}/../examples')
],
'UPSTREAM_URL': 'https://raw.githubusercontent.com/archlinux/archinstall/master/profiles',
'PROFILE_DB': None, # Used in cases when listing profiles is desired, not mandatory for direct profile grabbing.
'PROFILE': Path(__file__).parent.parent.joinpath('default_profiles'),
'LOG_PATH': '/var/log/archinstall',
'LOG_FILE': 'install.log',
'MOUNT_POINT': '/mnt/archinstall',
'MOUNT_POINT': Path('/mnt/archinstall'),
'ENC_IDENTIFIER': 'ainst',
'DISK_TIMEOUTS' : 1, # seconds
'DISK_RETRY_ATTEMPTS' : 5, # RETRY_ATTEMPTS * DISK_TIMEOUTS is used in disk operations

Some files were not shown because too many files have changed in this diff Show More