Commit Graph

491 Commits

Author SHA1 Message Date
Dylan Taylor d76cf18eaa Create symlinks outside of the archinstall folder 2022-02-12 14:26:59 -05:00
Dylan Taylor c41c1c9102 Move example and profiles folders to archinstall 2022-02-12 14:26:37 -05:00
Dylan M. Taylor 1df17eb987
Add a flag to install testing repositories (#967)
* Add a boolean to install testing repos, default to false

* More work on adding structure

* Add logic to enable testing repos.

* Corrections

* Make flake8 happy about regex escapes

* Flake8 doesn't like whitespace around equals

* Fix trailing whitespace character
2022-02-12 18:11:34 +01:00
Daniel 003a35be3d
Fix errors on selection of additional packages (#959)
* Fix errors on selection of additional packages

* Fix flake8

* Added the new /groups/search/json/?name=x endpoint merged today

* Fixed flake8 complaint

* Forgot to do json.loads() on the HTTP request result

* Update package selection

* Fix flake8

Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
Co-authored-by: Anton Hvornum <anton@hvornum.se>
2022-02-12 11:47:51 +01:00
Werner Llácer 5990491292
Some refinements to the Menu infraestructure (#962)
* Correct definition of btrfs standard layout

* Solve issue #936

* make ask_for_a_timezone as synonym to ask_timezone

* Some refining in GeneralMenu
secret is now a general function

* Revert "Some refining in GeneralMenu"

This reverts commit e6e131cb19.

* New version of the FlexibleMenu
* Added new functionality to Selector
* Created a GeneralMenu class
* GlobalMenu is made a child of GeneralMenu

* Some refining in GeneralMenu
secret is now a general function

* log is invoked in GeneralMenu directly

* Materialize again _setup_selection_menu_options. Gives more room to play

* Callbacks converted as methods
Synch() (data area and menu) decoupled from enable()
and made general before any run

* Only_hd gets a new implementation of the menu
flake8 corrections

* New version of the FlexibleMenu
* Added new functionality to Selector
* Created a GeneralMenu class
* GlobalMenu is made a child of GeneralMenu

* changes from the rebase left dangling

* Adapt to PR #874

* Adapted to nationalization framework (PR 893).
String still NOT adapted

* flake8 complains

* Use of archinstall.output_config instead of local copy at swiss.py

* Problems with the last merge

* git complains

* Menu admits now preset values and cursor positioning

* Now GeneralMenu moves to the next entry after each selection

* flake8 complains

* Control of limits for cursor position at GeneralMenu

* Make auto cursor positioning optional at GeneralMenu.
True for GlobalMenu

* Code cleanup after rebase, and flake8 complains
2022-02-11 13:22:35 +01:00
Anton Hvornum d3b6832345
Change usage `lsblk` to `blkid` where possible. And general improvements to disk handling. (#949)
* Renamed all_disks() to all_blockdevices() as it's technically a bit more correct. As it would return partitions as well. And it now distinguishes between BlockDevice() and Partition() when returning values. Also lsblk has been replaced with blkid and glob.glob() on /sys/class/block/

* Added handling of loop devices

* Added device information enrichment

* Removed old code

* Updated the usage of blockdevice.info['type'] to 'DEVTYPE' as per returned by blkid instead of lsblk

* Created a MapperDev() and DMCryptDev() 'disk' classes. This should help differentiate between BlockDevice(), Partition() and crypt-specific devices. Due to some new helper functions (mentioned later) BlockDevice().device_or_backfile has been simplified thanks to the information being pre-existing instead of having to call losetup. BlockDevice().mountpoint has been added as a dummy function. archinstall.enrich_blockdevice_information() has been added to enrich information extracted from blkid. archinstall.uevent() has been created to parse blkdid data when in -o export format, also eats up /sys/class/x/y/uevent data which is of the same format. all_blockdevices() now returns mapper devices found under /dev/mapper/ and properly returns the different types, not just BlockDevice() for everything. archinstall.get_parent_of_partition() has been added, which can translate /dev/sda1 to /dev/sda via strings by using /sys/class/block/ - note here tho that it has to be a blockdevice. Other parents won't return properly. archinstall.find_mountpoint() was created to be able to find mountpoints for a device, rather than the other way around which get_mount_info() handles. find_mountpoint() will convert /dev/sda1 -> /boot for instance, whereas get_mount_info('/boot') will convert /boot -> /dev/sda1. archinstall.get_partitions_in_use() will now properly traverse the entire structure, not just one level deep. This called for the creation of get_all_targets() which will take a dictionary structure and look for 'target': '/path' and return all '/path' values, recursively without being trapped in get_partitions_in_use() which has recursive targets. get_partitions_in_use() now returns a dictionary instead of a list, as it also returns the mountpoint and associated device. It will also prioritize MapperDev over DMCryptDev (as MapperDev is a child of DMCryptDev and can be resolved, other way around is 'trickier').

* Reverted Installer().partitions to return only the partitions. It's a slight speed defecit by not returning the mountpoints directly as {mntpoint: partition}, because the partition.mountpoint is a bit slow. But it avoids breaking legacy code :)

* Fixed a few imports, and added MapperDev.filesystem

* Fixed so that Installer().detect_encryption() uses the new separated logic of MapperDev/DMCryptDev/Partition

* Fixing a translation issue on hostname

* Added DMCryptDev.filesystem

* Added back archinstall.all_disks() for legacy reasons.

* Added a deprecation warning to archinstall.all_disks()

* Added a enrichment check for dm-crypt specific devices, which in turn sets DMCRYPT_NAME on the device information, making it easier to detect and target DMCryptDev when needed. This should avoid issues with loopdevices being detected as DMCryptDev. Also some minor tweaks to variable names on unencrypted setups

* Made debug output hidden without --verbose. Also added get_subvolume_info() which takes a path and returns any 'btrfs subvol show' information on the path

* Partition().subvolumes has been added. Which generates an iterator over potential subvolumes based on mountpoint. Partition().mount_information is now a list of mount-destinations, as one partition should be allowed to be mounted to several places (more commonly used in btrfs subvolumes). BtrfsSubvolume() has been added as a 'device type', which should make it more easy to recognize a BtrfsSubvolume object right out of the gate. Only found from Partition().subvolumes currently.

* Parameter fix for --verbose in log()

* Made sure loopdev devices have a PATH key as well, to make dealing with them as blockdevice as seamless as possible.

* Added backup information grabbing on uninitated devices

* Tweaked backup option if losetup cannot find the blockdev

* looks like losetup doesn't exit with a bad code even when it should. Raising ourselves when information is empty.

* Fixed structure returned from get_blockdevice_uevent()

* Made sure that fallback to blkid and loopdev info returns a PTTYPE (Partition Table Type) that is None, since we can't locate it yet (empty drive). But it's not False either, which is what logic after is looking for

* Deprecated split_bind_name() and added more debugging

* get_partitions_in_use() now only iterates over Partition() objects. And properly checks all mountpoints, not only the first.

* Flipped SysCallError to show the last 500 bytes if data rather than the first, to catch the actual errors.

* Removed anonymous parameters to all_blockdevices()

* Added .mount_information to MapperDev

* Added typing annotations

* Partition().mountpoint is now a @property that will live-report the current mountpoint.

* Removed the need for setting Partition().mountpoint = X when calling .mount() and .unmount(), as it's live-reported instead.

* Added .subvolumes to MapperDev()

* Added debug information

* Muting F841 in helpers.py

* Tweaked debug information in get_mount_info()

* Minimized log output by moving Partition().mountpoint to DEBUG, as partitions are allowed to not be mounted

* Simplified Installer().add_bootloader() to act as a router to the new split function, one for each bootloader type. This since flake8 complained about complexity and I agree that it became to wild.
2022-02-08 23:21:20 +01:00
Werner Llácer ef558fdb45
Solving some issues after merging flexible_menu (#948)
* Correct definition of btrfs standard layout

* Solve issue #936

* make ask_for_a_timezone as synonym to ask_timezone

* first batch of changes due to the last merges
selector.text is a call to selector.menu_text
A number of Generalmenu methods reappeared in their old version at Global

* disk encryption password menu option restored
2022-02-07 01:58:33 +01:00
Werner Llácer 1ea6fea1d8
Flexible menu 2 (#916)
* Correct definition of btrfs standard layout

* New version of the FlexibleMenu
* Added new functionality to Selector
* Created a GeneralMenu class
* GlobalMenu is made a child of GeneralMenu

* Some refining in GeneralMenu
secret is now a general function

* log is invoked in GeneralMenu directly

* Correction at GeneralMenu

* Materialize again _setup_selection_menu_options. Gives more room to play

* Callbacks converted as methods
Synch() (data area and menu) decoupled from enable()
and made general before any run

* script swiss added to the patch set

* Only_hd gets a new implementation of the menu
flake8 corrections

* swiss.py description added

* New version of the FlexibleMenu
* Added new functionality to Selector
* Created a GeneralMenu class
* GlobalMenu is made a child of GeneralMenu

* changes from the rebase left dangling

* Modify order of execution between exec_menu and post_processing.
Added selector_name as parameter for exec_menu

* minor corrections to the scripts

* Adapt to PR #874

* Solve issue #936

* make ask_for_a_timezone as synonym to ask_timezone

* Adapted to nationalization framework (PR 893).
String still NOT adapted

* flake8 complains

* Use of archinstall.output_config instead of local copy at swiss.py

* Problems with the last merge

* more flake8 complains. caused by reverted changes re. ask*timezone

* git complains

Co-authored-by: Anton Hvornum <anton@hvornum.se>
2022-02-06 11:54:13 +01:00
Daniel 9fb8d3164c
Update nationalization (#944)
* Update nationalization

* Update translations

* Finish german translation

* Fix errors #943

* Add remaining translations

* Fix alignment in menu

* Update README

* Update translations:

* Fix flake8

* Update tz function

Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
2022-02-06 11:30:26 +01:00
Werner Llácer ec73bdab4c
Routine to properly print and save config data (#888)
* Created a standard function to show/save the config parameters

* flake8 complains

* Correct definition of btrfs standard layout

* Solve issue #936

* Moved output_configs to lib/configuration.py
2022-02-05 13:52:14 +01:00
Werner Llácer 85f2938df9
Moving load_config from guided to \__init__.py (#887)
* The initializations done at the start of guided.py are now at archinstall/__init__.py
The scripts have been changed accordingly

* Correct definition of btrfs standard layout

Co-authored-by: Anton Hvornum <anton@hvornum.se>
2022-02-04 23:48:41 +01:00
Werner Llácer 3cd7dc24c7
Command locales (second batch) (#886)
* flexibilize the definition of execution locale for OS commands executed via the SysCommand* interface.
Defined a storage argument which holds the default
Added functions to
unset the program own locales
reset to the program default locales
set a specific locale
A decorator to execute functions in the host locale environment

* rename decorator local_environ to host_locale_environ
created a simmetric decorator c_locale_environ, to make a routine work with the C locale whatever is set

* Correct definition of btrfs standard layout

* Added error handling

* Fixed issue where archinstall.Boot() would raise an exception in vain

* Added debugging for SysCommandWorker()

* Added some debugging

* Tweaking debug a bit

* Tweaking debug

* Adding more debug

* Adding more debug

* Removed some debugging

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Adding more debug

* Removed soem debugging

* Removed soem debugging

* Testing a revert

* Adding back the reverted change, adding lofile

* Redirecting stdout to /dev/null for testing (to avoid interrupting the fork)

* Reverted debug changes

* Testing os.system()

Co-authored-by: Anton Hvornum <anton@hvornum.se>
2022-02-03 00:02:30 +01:00
Daniel 37d6da7e4e
Migrate old input to new menu (#874)
* Migrate old input to new menu

* Fix imports

* Remove imports

* Update

* Fixed import by changing 'import archinstall', to 'from ..menu import Menu' and use Menu() directly

* Converted archinstall.<thing> to from ..where import <thing>. This enables us to use archinstall as a module, a git repository in testing and other things without having to install archinstall as an actual module.

Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
Co-authored-by: Anton Hvornum <anton@hvornum.se>
2022-02-02 14:26:09 +01:00
Anton Hvornum c08520f990
SysCommand() to remove ANSII VT100 Esc codes & archlinux-keyring fix (#933)
* Fixed SysCommandWorker() so that it removes ANSII VT100 escape codes. I also moved package.py into it's own folder, as that's something I want to expand on a lot, so package related stuff should go in there. I created a installed_package() function which gets information about the locally installed package. I changed so that find_packages() and find_package() returns a data-model instead for the package information. This should unify and make sure we detect issues down the line.

* Working on structuring .version constructor that works with BaseModel

* Added version contructors to VersionDef(). Also added __eq__ and __lt__ to LocalPackage() and PackageSearchResult().

* removed debug and added a TODO

* Removed whitespace

* Removed mirror-database function from myrepo
2022-02-02 14:22:52 +01:00
Anton Hvornum 364bc23b48
Updated configuration example "custom commands" 2022-02-02 08:22:08 +01:00
Werner Llácer e3adab13e8
Issue923 (#927)
* Correct definition of btrfs standard layout

* Solving issue 923.
Standarize keyword "wipe" to signal the need of formatting a FS

* flake8 complains
2022-02-01 16:44:39 +01:00
Timothy Pidashev 179c103cdd
Synchronize `archlinux-keyring` when running guided.py (#917)
* Update guided.py

* Update guided.py

* Update guided.py

* Fix typo

* Format string properly; elaborate log msg

* Make change consistent to mirror-check

* Update documentation
2022-01-30 11:01:04 +01:00
Dylan M. Taylor 90e320dd11
Allow mirror reachability check to be skipped (#910)
* Allow network check to be skipped

* Technically it is a mirror check - not a check for general connectivity
2022-01-26 07:53:38 +01:00
Thomas 34488d25c5
Disable zswap (#883)
Disable zswap when using zram.

The swap device is now initialised before the bootloader, inline with the
installation guide.
2022-01-20 17:49:18 +00:00
Werner Llácer 3d30a4eb28
Missing arguments after GlobalMenu execution (issue #868) (#871)
* Install.__init__ parameter kernel as a list everywhere.
I've dropped the string check, as it shouldn't happen again, and it's good that it fails, when it happens

* Global Menu now loads in the arguments dict all options enabled with default value and no manually selected (issue #868)
2022-01-15 21:44:13 +01:00
Werner Llácer 31f55e8d6b
Activate file write into only_hd and upgrade to guided.py level (#867) 2022-01-15 14:20:06 +01:00
Anton Hvornum da0c08e79a Added in version history in the configuration 2022-01-14 12:54:22 +01:00
Anton Hvornum f175ea8b7d
Created a save_user_configurations (#856)
* Created a save_user_configurations

This should avoid accidental placement of individual json outputs in a way that some doesn't get saved before installation continues etc.

* Was looking for disk_layouts in the wrong place.
* changed numerous other references to disk_layouts looking in the wrong place.
2022-01-14 11:48:56 +01:00
Daniel 1234261a7a
Global menu (#806)
* Global menu

* Fix flake8

* Refactor code

* Add documentation

* Fix flake8

* Add support for user flow mentioned in #799

* Move import

* Fix flake8 (again)

Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
Co-authored-by: Anton Hvornum <anton.feeds@gmail.com>
2022-01-07 10:48:23 +00:00
HTV04 08d7375e62
Fix partition layout (#794)
* Fix partition layout

* Tweaked the numbers to match the alignment of previous partitions.

Co-authored-by: Anton Hvornum <anton.feeds@gmail.com>
2022-01-07 09:55:51 +00:00
Werner Llácer b63cf7cb15
- No treatement of disk_layouts_file (#844)
- Ordered exit
2022-01-06 23:27:56 +01:00
Ilario Gelmetti 3474176e64
examples/guided: if selected, activate NTP also in installer system (#831)
* Ask for NTP even if timezone is unset
* Ask for NTP before retrieving mirror list
* Activate NTP in the installer as soon as asked
* Removed excessive import
SysCommand is exposed through archinstall
* Removed f-string

Co-authored-by: Anton Hvornum <anton.feeds+github@gmail.com>
2022-01-02 22:45:44 +01:00
Anton Hvornum 240f688cce Adding a Pipewire application profile (#821)
* Adding a Pipewire application profile

This to better manage the pipewire setup process and minimize guided a bit. This also adds the concept of @archinstall.plugin decorators to add a plugin in run-time. Which pipewire uses to detect user creation and enable the pipewire-pulse service for new users.

* Forgot to run .install() on pipewire Application()
* Backwards compatible variable insertion for installation session
2022-01-02 16:38:02 +01:00
Anton Hvornum 93aa4aa6cc
Fix inspired by #804 to tweak the fix in v2.3.1-dev 2022-01-02 14:14:00 +01:00
Anton Hvornum b1af3eda83 Fixed issue #799. Where user prompt wouldn't be shown if a root password was set. 2022-01-02 14:11:00 +01:00
Werner Llácer b89408ab7b
Improved command line argument parsing (#725)
* An update to PR 715, making the handling of the *--mount-point* parameter less error prone.
I added a synomym (accepting the name both with underscore and dash) and ignoring when no value specified
I added it explicitly to the list to accept both the --parm value and --parm=value syntax
DOES NOT check the contents of the parameter
* Explicitly set all the know parameters
* Define explictly all parameters.
Make all non explicitly defined parameters behave as standard parameters, with on exception, names are not changed
Some cleanup of the code
In guided.py the reference to the dry_run parameter is updated to the standard naming convention for parameters
* Linter with flake8. corrections
* Linter with flake8. corrections (II)
* Linter with flake8. corrections (and III)
* Added --disk_layout argument. Was missing
I moved its loading from guided.py to __init__.py as it happens to the other json related arguments
* Better handling of errors during processing of the --disk_layouts parameter.
I define a routine to read an store a JSON file or stream. Tested on disk_layout
* Expand the former commit to all JSON file arguments
* Moved the function we created to read json files/streams to general.py. Add some comments
* flake8. A reference now unneded
* The merge process for the dry-run argument was causing the issue, not solving it
The del is just a cleanup for version upgrade without consequence (I hope)
* flake8 warning
* Correcting the last correction . Worked for old config files, but only for them
* New parameter parsing algorithm. More flexible and accepts multiple arguments (optionallY)
plus some documentation effort
* flake8 warning. For once is significant ( != None to not None)
2022-01-02 12:28:49 +01:00
Werner Llácer e388537bc3
better handling the skip partitioning option. (#778)
* better handling the skip partitioning option. plus sending some warnings to the user
* device configuration options taken out from ask_user_questions.
Forced by flake8
* Revert "device configuration options taken out from ask_user_questions."
This reverts commit 1b3cffb3df.
* Adapted to new selection ui
* Cleanup of obsolete code
* Changed colour of skip messages from red to yellow
* Flake8 comments
2022-01-02 12:01:19 +01:00
Werner Llácer 7f9b7991e9
Btrfs2 (#787)
* All the changes needed to make btrfs subvolumes work. It boils down to two points;
the handling of the addressing of subvolumes re. physical partitions, and the small changes at the bootloader level
* We added a new script only_hd for testing purposes. It only handles hadrd drive management
* restoring an escape hatch during subvolume processing
* hipercommented manage_btrfs_subvolumes
* Ready to be able to select and process options in subvolume mounting
* Separte nodatacow processing
* Solving a flake8 complain
* Use of bind names @ get_filesystem_type
* compress mount option bypass
* Preparations for encryption handling
* Compatibility to master version re. encrypted btrfs volumes
* Now we can create subvolumes and mountpoints inside an encrypted btrfs partition
* changes for entries file generation with systemd-bootctl
* flake8 corrections plus some comments

Co-authored-by: Anton Hvornum <anton@hvornum.se>
2021-12-31 13:47:41 +01:00
Anton Hvornum cad911893d
Bringing in changes from v2.3.1-dev branch. This fixes a selection of things mentioned in #819. 2021-12-31 12:33:01 +01:00
Daniel 908c7b8cc0
Add simple menu for better UX (#660)
* Add simple menu for better UX
* Add remove external dependency
* Fix harddisk return value on skip
* Table output for partitioning process
* Switch partitioning to simple menu
* fixup! Switch partitioning to simple menu
* Ignoring complexity and binary operator issues
Only in simple_menu.py
* Added license text to the MIT licensed file
* Added in versioning information
* Fixed some imports and removed the last generic_select() from user_interaction. Also fixed a revert/merged fork of ask_for_main_filesystem_format()
* Update color scheme to match Arch style better
* Use cyan as default cursor color
* Leave simple menu the same

Co-authored-by: Daniel Girtler <girtler.daniel@gmail.com>
Co-authored-by: Anton Hvornum <anton.feeds+github@gmail.com>
Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2021-12-02 21:17:51 +01:00
Anton Hvornum 5aaa8d9814 Adds version to user_conf.json
The key is never read anywhere, but would be useful in debugging purposes.
2021-11-28 11:31:23 +01:00
Anton Hvornum 87e75a7d09 Updated examples, fix #772 2021-11-26 14:24:47 +00:00
Anton Hvornum 070d088462
Added error handling to guided surrounding disk_layouts 2021-11-24 21:33:53 +01:00
Dylan Taylor 04c2b19dbe Fix error at end of installation from missing method import 2021-11-23 15:27:02 -05:00
Dylan M. Taylor 72d02a391d
Add accessibility packages to installed system if using the on the live ISO (#760)
* Add group of accesibility tools
* Conditionally extend base packages
* Enable it if accessibility in use in guided
* Fix circular import
2021-11-23 21:10:21 +01:00
Anton Hvornum 6b6c9c84be
Adding a warning when /boot is reasonably too small (#738)
* Moved convert_size_to_gb() into disk/helpers.py, Added a Partition().size property meta function. Using the .size value to check if /boot is too small which will raise an exception. The only drawback is that it's done post-formatting. This in order to catch scenarios where formatting isn't used.
* Changed /boot warning from 0.15GB to 0.25GB
* Changed the wording in the warning when /boot is too small.
2021-11-23 11:37:51 +00:00
Dylan Taylor 45ba4a3f1b Rewrite bootloader selection to allow efistub if advanced flag is set 2021-11-22 20:25:43 -05:00
Dylan M. Taylor e532b76158
Make NTFS an advanced option (#753)
* Make NTFS an advanced option
2021-11-22 21:08:41 +01:00
Anton Hvornum 29d0b3d155
Simplified size definition in dict. (#752)
* Simplified size definition in dict. Also changed from MiB to MB and GiB to GB on places where they were used, as BlockDevice().size now returns GB by default, so no math operations needed
* Appended the /boot offset to /root when specifying /home start.
2021-11-22 11:27:49 +00:00
Anton Hvornum 66e23af422
Merge PR #739 - Added @staticmethod decorators
source code improvement
2021-11-20 22:21:00 +00:00
Anton Hvornum 627b6b5207
Incorrect location usage of AVAILABLE_GFX_DRIVERS
#710
2021-11-20 10:32:28 +01:00
Steven Lee 91d030a09c
source code improvement 2021-11-19 14:09:47 +11:00
Anton Hvornum b22500babd Removed assumptions that !user and !super-users exists. 2021-11-11 18:59:53 +01:00
Anton Hvornum ad09c8cfa2 Bad copy paste of code prohibited credentials from being stored. 2021-11-11 09:59:22 +00:00
Anton Hvornum 813afa1507 Only adding disk encryption to user_credentials.json when supplied. 2021-11-11 09:43:06 +00:00