87c3b7e767
amazonImage: make statically sized again
...
For reasons we haven't been able to work out, the aarch64 EC2 image now
regularly exceeds the output image size on hydra.nixos.org. As a
workaround, set this back to being statically sized again.
The other images do seem to build - it's just a case of the EC2 image
now being too large (occasionally non-determinstically).
2021-05-01 02:19:42 +00:00
5aa4273e4f
treewide: use auto diskSize for make-disk-image
...
(cherry picked from commit f3aa040bcb
2021-04-24 14:49:07 -04:00
4fb91cbafe
Revert "treewide: use auto diskSize for make-disk-image"
...
This reverts commit f3aa040bcb
2021-04-24 02:38:36 +00:00
f521b12b0e
Revert "nixos/amazon-image: (temporarily) use fixed disk size again"
...
This reverts commit 6a8359a92a
2021-04-24 02:38:25 +00:00
6a8359a92a
nixos/amazon-image: (temporarily) use fixed disk size again
...
As a temporary workaround for #120473 while the image builder is patched
to correctly look up disk sizes, partially revert
f3aa040bcb
2021-04-24 00:43:47 +00:00
f3aa040bcb
treewide: use auto diskSize for make-disk-image
2021-04-22 19:52:49 +02:00
606b49721f
add new Google Cloud image for the current release
...
update the create-gce.sh script with the ability to create public images
out of a GS object.
2021-03-21 14:04:09 -04:00
7092dd52f8
amazonImage: Upload disks as GP3 for cheaper & faster IO ( #109027 )
...
GP3 is always faster and cheaper than GP2, so sticking to GP2 is
leaving money on the table.
https://cloudwiry.com/ebs-gp3-vs-gp2-pricing-comparison/
2021-01-11 13:54:40 -05:00
38a394bdee
Merge pull request #102174 from grahamc/ami-root-use-gpt
...
AMI root partition table: use GPT to support >2T partitions
2020-10-30 16:14:37 -04:00
d77ddf2a40
nixos.amazonAmi: use legacy+gpt disk images to support partitions >2T
2020-10-30 15:50:25 -04:00
74a577b293
create-amis: improve wording around the service name's IAM role
...
Co-authored-by: Cole Helbling <cole.e.helbling@outlook.com>
2020-10-30 12:40:17 -04:00
2bf1fc0345
create-amis: allow customizing the service role name
...
The complete setup on the AWS end can be configured
with the following Terraform configuration. It generates
a ./credentials.sh which I just copy/pasted in to the
create-amis.sh script near the top. Note: the entire stack
of users and bucket can be destroyed at the end of the
import.
variable "region" {
type = string
}
variable "availability_zone" {
type = string
}
provider "aws" {
region = var.region
}
resource "aws_s3_bucket" "nixos-amis" {
bucket_prefix = "nixos-amis-"
lifecycle_rule {
enabled = true
abort_incomplete_multipart_upload_days = 1
expiration {
days = 7
}
}
}
resource "local_file" "credential-file" {
file_permission = "0700"
filename = "${path.module}/credentials.sh"
sensitive_content = <<SCRIPT
export service_role_name="${aws_iam_role.vmimport.name}"
export bucket="${aws_s3_bucket.nixos-amis.bucket}"
export AWS_ACCESS_KEY_ID="${aws_iam_access_key.uploader.id}"
export AWS_SECRET_ACCESS_KEY="${aws_iam_access_key.uploader.secret}"
SCRIPT
}
# The following resources are for the *uploader*
resource "aws_iam_user" "uploader" {
name = "nixos-amis-uploader"
}
resource "aws_iam_access_key" "uploader" {
user = aws_iam_user.uploader.name
}
resource "aws_iam_user_policy" "upload-to-nixos-amis" {
user = aws_iam_user.uploader.name
policy = data.aws_iam_policy_document.upload-policy-document.json
}
data "aws_iam_policy_document" "upload-policy-document" {
statement {
effect = "Allow"
actions = [
"s3:ListBucket",
"s3:GetBucketLocation",
]
resources = [
aws_s3_bucket.nixos-amis.arn
]
}
statement {
effect = "Allow"
actions = [
"s3:PutObject",
"s3:GetObject",
"s3:DeleteObject",
]
resources = [
"${aws_s3_bucket.nixos-amis.arn}/*"
]
}
statement {
effect = "Allow"
actions = [
"ec2:ImportSnapshot",
"ec2:DescribeImportSnapshotTasks",
"ec2:DescribeImportSnapshotTasks",
"ec2:RegisterImage",
"ec2:DescribeImages"
]
resources = [
"*"
]
}
}
# The following resources are for the *vmimport service user*
# See: https://docs.aws.amazon.com/vm-import/latest/userguide/vmie_prereqs.html#vmimport-role
resource "aws_iam_role" "vmimport" {
assume_role_policy = data.aws_iam_policy_document.vmimport-trust.json
}
resource "aws_iam_role_policy" "vmimport-access" {
role = aws_iam_role.vmimport.id
policy = data.aws_iam_policy_document.vmimport-access.json
}
data "aws_iam_policy_document" "vmimport-access" {
statement {
effect = "Allow"
actions = [
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
]
resources = [
aws_s3_bucket.nixos-amis.arn,
"${aws_s3_bucket.nixos-amis.arn}/*"
]
}
statement {
effect = "Allow"
actions = [
"ec2:ModifySnapshotAttribute",
"ec2:CopySnapshot",
"ec2:RegisterImage",
"ec2:Describe*"
]
resources = [
"*"
]
}
}
data "aws_iam_policy_document" "vmimport-trust" {
statement {
effect = "Allow"
principals {
type = "Service"
identifiers = [ "vmie.amazonaws.com" ]
}
actions = [
"sts:AssumeRole"
]
condition {
test = "StringEquals"
variable = "sts:ExternalId"
values = [ "vmimport" ]
}
}
}
2020-10-30 12:12:08 -04:00
e253de8a77
create-amis.sh: log the full response if describing the import snapshot tasks fails
2020-10-30 12:08:01 -04:00
f92a883ddb
nixos ec2/create-amis.sh: shellcheck: $ is not needed in arithmetic
2020-10-30 12:08:01 -04:00
7dac8470cf
nixos ec2/create-amis.sh: shellcheck: explicitly make the additions to block_device_mappings single strings
2020-10-30 12:08:00 -04:00
a66a22ca54
nixos ec2/create-amis.sh: shellcheck: read without -r mangles backslashes
2020-10-30 12:08:00 -04:00
baf7ed3f24
nixos ec2/create-amis.sh: shellcheck: SC2155: Declare and assign separately to avoid masking return values.
2020-10-30 12:07:59 -04:00
f5994c208d
nixos ec2/create-amis.sh: shellcheck: quote state_dir reference
2020-10-30 12:07:59 -04:00
c76692192a
nixos ec2/create-amis.sh: shellcheck: quote region references
2020-10-30 12:07:49 -04:00
d08b81c3b5
Merge pull request #89116 from wagdav/fix-args-create-amis
...
nixos/maintainers/scripts/ec2/create-amis.sh: fix argument check
2020-08-22 16:47:54 +02:00
0cb79c953d
nixos/ec2: remove dependency on NIX_PATH
...
This is required when migrating to flakes
2020-08-16 10:56:44 +00:00
2b5659c700
nixos/maintainers/*: editorconfig fixes
2020-08-04 00:23:54 +10:00
3b1ed035c3
create-amis: fix argument check
...
Because this script enables `set -u` when no arguments are provided bash
exits with the error:
$1: unbound variable
instead of the helpful usage message.
2020-05-28 17:41:45 +02:00
7c7e76450b
nixos/azure-new: use local nixpkgs
2020-04-27 02:11:10 -07:00
1992768157
nixos/azure: clarify how users work in basic example
2020-03-29 13:56:55 -07:00
627ae7e057
nixos/azure: upload-image.sh cleanup $1 handling
2020-03-29 13:56:55 -07:00
a5de97f21e
nixos/azure: upload-image names the image better
2020-03-29 13:56:55 -07:00
c2b2cc6dbd
nixos/azure: simplify example image
2020-03-29 13:56:55 -07:00
20f981de08
azure: init nixos/maintainers/scripts/azure-new
2020-03-29 13:56:55 -07:00
129176452c
nixos-ami: update nvme_core.io_timeout for linux kernel >= 4.15
...
NixOS 20.03 is built on kernel 5.4 and 19.09 is on 4.19, so we should update
this option to the highest value possible, per linked upstream instructions from
Amazon.
2020-03-22 00:35:56 -04:00
4e5b0571ed
create-amis: Add eu-north-1
2020-03-05 18:00:28 +00:00
65dcd244bc
maintainers/create-azure.sh: run from anywhere
...
I'm not really sure how the line directly after ended up with this,
but this line didn't...
2020-01-09 20:54:28 +00:00
bd61216f55
ec2/create-amis.sh: register root device as /dev/xvda
...
For the case of blkfront drives, there appears to be no difference
between /dev/sda1 and /dev/xvda: the drive always appears as the
kernel device /dev/xvda.
For the case of nvme drives, the root device typically appears as
/dev/nvme0n1. Amazon provides the 'ec2-utils' package for their first
party linux ("Amazon Linux"), which configures udev to create symlinks
from the provided name to the nvme device name. This name is
communicated through nvme "Identify Controller" response, which can be
inspected with:
nvme id-ctrl --raw-binary /dev/nvme0n1 | cut -c3073-3104 | hexdump -C
On Amazon Linux, where the device is attached as "/dev/xvda", this
creates:
- /dev/xvda -> nvme0n1
- /dev/xvda1 -> nvme0n1p1
On NixOS where the device is attach as "/dev/sda1", this creates:
- /dev/sda1 -> nvme0n1
- /dev/sda11 -> nvme0n1p1
This is odd, but not inherently a problem.
NixOS unconditionally configures grub to install to `/dev/xvda`, which
fails on an instance using nvme storage. With the root device name set
to xvda, both blkfront and nvme drives are accessible as /dev/xvda,
either directly or by symlink.
2019-11-02 05:58:58 +09:00
dc13a7f26a
ec2-amis.nix: add 19.09 amis
...
replace /home/deploy -> $HOME to allow running the script from outside
the bastion.
2019-10-28 14:04:20 -04:00
b263e57c57
scripts/gce: make image name configurable
2019-10-25 10:10:42 +02:00
84742e2293
amazon-image.nix: upload prebuilt images
2019-09-05 00:52:21 +09:00
25bee972e9
amazon-image.nix: add hydra-build-products and improve metadata
2019-09-05 00:52:20 +09:00
027e5560bd
amazon-image.nix: default to vpc formatted images
...
These can be imported without converison.
2019-09-05 00:52:20 +09:00
5501274b5f
amazon-image.nix: add EFI support, enable by default for aarch64
2019-09-05 00:52:17 +09:00
0f8596ab3f
mass replace "flip map -> forEach"
...
See `forEach`-introduction commit.
```
rg 'flip map ' --files-with-matches | xargs sed -i 's/flip map /forEach /g'
```
2019-08-05 14:03:38 +03:00
91bb646e98
Revert "mass replace "flip map -> foreach""
...
This reverts commit 3b0534310c
2019-08-05 14:01:45 +03:00
3b0534310c
mass replace "flip map -> foreach"
...
See `foreach`-introduction commit.
```
rg 'flip map ' --files-with-matches | xargs sed -i 's/flip map /foreach /g'
```
2019-07-14 13:46:10 +03:00
786f02f7a4
treewide: Remove usage of isNull
...
isNull "is deprecated; just write e == null instead" says the Nix manual
2019-04-29 14:05:50 +02:00
261372b69c
amazon-image.nix: Resolve failure to include resize2fs
...
Since 34234dcb51
2019-03-15 17:33:45 +01:00
d14f102334
Merge pull request #44573 from vincentbernat/feature/cloudstack
...
nixos/cloudstack-image: initial import
2019-02-24 08:28:42 -08:00
d190b204f0
Rename `novaImage` to `openstackImage`
...
People don't necessary know `nova` is related to Openstack (it is a
component of Openstack). So, it is more explicit to call it
`openstackImage`.
2019-02-11 20:58:44 +01:00
88570538b3
google-compute-image: make it a module and the size tuneable ( #49854 )
...
* move GCE system configuration to `google-compute-config.nix`
* remove `fetch-ssh-keys` service (disabled in comment)
2018-11-26 14:51:00 +00:00
15f98b7192
nixos/cloudstack-image: initial import
...
Cloudstack images are simply using cloud-init. They are not headless
as a user usually have access to a console. Otherwise, the difference
with Openstack are mostly handled by cloud-init.
This is still some minor issues. Notably, there is no non-root user.
Other cloud images usually come with a user named after the
distribution and with sudo. Would it make sense for NixOS?
Cloudstack gives the user the ability to change the password.
Cloud-init support for this is imperfect and the set-passwords module
should be declared as `- [set-passwords, always]` for this to work. I
don't know if there is an easy way to "patch" default cloud-init
configuration. However, without a non-root user, this is of no use.
Similarly, hostname is usually set through cloud-init using
`set_hostname` and `update_hostname` modules. While the patch to
declare nixos to cloud-init contains some code to set hostname, the
previously mentioned modules are not enabled.
2018-11-17 20:40:11 +01:00
b240822cfa
create-amis.sh: Change directory for AMIs
2018-07-24 21:19:14 +02:00
2e979e8ceb
[bot] nixos/*: remove unused arguments in lambdas
2018-07-20 20:56:59 +00:00
Luke Granger-Brown