EC2のサービスアップデート、それも地味なんですが大事なアップデートがありました。
興味もありますが、この渋い機能を触ってみないことにはありがたみを体験できない、 いざ本番でいきなり触らざるを得ない状況になったときに慌てずに済むかと思いました。
>aws ec2 get-serial-console-access-status --region ap-northeast-1
{
"SerialConsoleAccessEnabled": false
}
>aws ec2 get-serial-console-access-status --region ap-northeast-1
{
"SerialConsoleAccessEnabled": true
}
sudo passwd root
(これはrootユーザの例)などを行う/dev/xvda
や /dev/sda
にアタッチしたボリューム以外のボリュームでルートボリュームになる可能性がある${sshd_pid}
).ssh/config
の権限を 誤って 設定してしまった場合sudo ./ec2rl run
)sudo rm -rf /
)を 誤って 実行してしまった場合まずは実験環境を作成するために必要な設定を実施
docker run --rm -it -v %userprofile%\\.aws:/root/.aws amazon/aws-cli
%command%Set-ExecutionPolicy RemoteSigned -Force
Get-ExecutionPolicy
で確認docker -v
で確認Docker version 20.10.5, build 55c4c88
で記事を作成New-Item –type file –force $profile
でPS起動時に読み込むファイルを作成notepad $profile
で編集aws --version
で動作確認$profile
に記載する内容function aws {
$location = Get-Location
docker run --rm -v ${HOME}/.aws:/root/.aws -v ${location}:/aws amazon/aws-cli $Args
}
> aws --version
Unable to find image 'amazon/aws-cli:latest' locally
latest: Pulling from amazon/aws-cli
99a21848b482: Pull complete
f3944f83fc76: Pull complete
faabaca4f97b: Pull complete
e9ba7dc72ebe: Pull complete
40686cdddde2: Pull complete
Digest: sha256:cb3819faeea756e87b273c8eb6d007b607f12282440755794295e75e28fa2659
Status: Downloaded newer image for amazon/aws-cli:latest
aws-cli/2.2.1 Python/3.8.8 Linux/5.4.72-microsoft-standard-WSL2 docker/x86_64.amzn.2 prompt/off
aws-cdk-sample にて公開していますが、 前述の実験するための環境を作成するCDKのサンプルファイルを作成しています。
予め、キーペアを作成して頂いて、リソース作成時にパラメータとして渡すことで作成されます
初めてEC2を起動するリージョンの場合は、しばらく待つとAWSから「Your Request For Accessing AWS Resources Has Been Validated」というメールが届くのでお待ちください
8/16 | 13:57:03 | CREATE_FAILED | AWS::EC2::Instance | tkyEc2f Your request for accessing resources in this region is being validated, and you will not be able
to launch additional resources in this region until the validation is complete. We will notify you by email once your request has been validated. While normally resolved within minutes, please allow up to 4 hours for this process to complete. If the issue still persists, please let us know by writing to aws-verification@amazon.com for further assistance. (Service: AmazonEC2; Status Code: 400; Error Code: PendingVerification; Request ID: 6a88df4b-d83b-4320-8700-5ee9f6c1764f; Proxy: null)
new SrcStack (C:\\Projects\\aws-cdk-sample\\aws-cdk-sample\\src\\lib\\src-stack.ts:152:23)
\\_ Object.<anonymous> (C:\\Projects\\aws-cdk-sample\\aws-cdk-sample\\src\\bin\\src.ts:7:1)
\\_ Module._compile (internal/modules/cjs/loader.js:1063:30)
\\_ Module.m._compile (C:\\Projects\\aws-cdk-sample\\aws-cdk-sample\\src\\node_modules\\ts-node\\src\\index.ts:1056:23)
\\_ Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
\\_ Object.require.extensions.<computed> [as .ts] (C:\\Projects\\aws-cdk-sample\\aws-cdk-sample\\src\\node_modules\\ts-node\\src\\index.ts:1059:12)
\\_ Module.load (internal/modules/cjs/loader.js:928:32)
\\_ Function.Module._load (internal/modules/cjs/loader.js:769:14)
\\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
\\_ main (C:\\Projects\\aws-cdk-sample\\aws-cdk-sample\\src\\node_modules\\ts-node\\src\\bin.ts:198:14)
\\_ Object.<anonymous> (C:\\Projects\\aws-cdk-sample\\aws-cdk-sample\\src\\node_modules\\ts-node\\src\\bin.ts:288:3)
\\_ Module._compile (internal/modules/cjs/loader.js:1063:30)
\\_ Object.Module._extensions..js (internal/modules/cjs/loader.js:1092:10)
\\_ Module.load (internal/modules/cjs/loader.js:928:32)
\\_ Function.Module._load (internal/modules/cjs/loader.js:769:14)
\\_ Function.executeUserEntryPoint [as runMain] (internal/modules/run_main.js:72:12)
\\_ internal/main/run_main_module.js:17:47
8/16 | 13:57:03 | CREATE_FAILED | AWS::EC2::Instance |
aws ec2 describe-instances --query 'Reservations[*].Instances[*].[{Instance:InstanceId}, {State:State.Name},{PublicIpAddress:PublicIpAddress},{VolumeName:BlockDeviceMappings[*].DeviceName},{VolumeId:BlockDeviceMappings[*].Ebs.VolumeId},{ImageId:ImageId}]' --output yaml
- - - Instance: i-0d5ed80e0cf1cf423
- State: running
- PublicIpAddress: 13.231.159.93
- VolumeName:
- /dev/sda1
- VolumeId:
- vol-0fbf23be9d7829fc8
- ImageId: ami-0a8f70a19608d9e8f
- - - Instance: i-027fc9866bcdebea4
- State: running
- PublicIpAddress: 18.179.14.108
- VolumeName:
- /dev/xvda
- VolumeId:
- vol-0cb6295c58b0d24b3
- ImageId: ami-0f037327d64de9e49
- - - Instance: i-0affaf9475a0c08e7
- State: running
- PublicIpAddress: 13.114.175.144
- VolumeName:
- /dev/xvda
- VolumeId:
- vol-024a2457c789b9f49
- ImageId: ami-0f037327d64de9e49
- - - Instance: i-00be31cf6a8086b47
- State: running
- PublicIpAddress: 54.238.231.164
- VolumeName:
- /dev/xvda
- VolumeId:
- vol-06e289e23476d3e55
- ImageId: ami-0f037327d64de9e49
- - - Instance: i-0d646b701c2c8ddf7
- State: running
- PublicIpAddress: 54.250.148.9
- VolumeName:
- /dev/sda1
- VolumeId:
- vol-00ccd542e5aab6e05
- ImageId: ami-0a8f70a19608d9e8f
- - - Instance: i-0cf0e004d715d8d49
- State: running
- PublicIpAddress: 18.183.149.191
- VolumeName:
- /dev/xvda
- VolumeId:
- vol-00829f9b262d558c3
- ImageId: ami-0f037327d64de9e49
aws ec2 describe-images --query 'Images[].[{ImageId:ImageId},{Description:Description}]' --image-ids ami-0a8f70a19608d9e8f ami-0f037327d64de9e49 --output yaml
- - ImageId: ami-0a8f70a19608d9e8f
- Description: Microsoft Windows Server 2019 with Desktop Experience Locale Japanese
AMI provided by Amazon
- - ImageId: ami-0f037327d64de9e49
- Description: Amazon Linux 2 AMI 2.0.20210421.0 x86_64 HVM gp2
sudo systemctl status sshd
Active: active (running)
sudo systemctl stop sshd
Active: inactive (dead)
.ssh/authorized_keys
を 誤って 設定してしまった場合 # sudo chown ec2-user:ec2-user /home/ec2-user/.ssh/authoriz
#
#
# ls -l /home/ec2-user/.ssh/authorized_keys
-rw------- 1 root root 388 May 1 07:05 /home/ec2-user/.ssh/authorized_keys
#
# sudo chown ec2-user:ec2-user /home/ec2-user/.ssh/authoriz
# ls -l /home/ec2-user/.ssh/authorized_keys
-rw------- 1 ec2-user ec2-user 388 May 1 07:05 /home/ec2-user/.ssh/authorized_keys
#
t3.micro
だと起動処理の途中で止まったので t3.medium
へ変更後起動した様子sudo ./ec2rl run
)$ sudo ./ec2rl run
-----------[Backup Creation]-----------
No backup option selected. Please consider backing up your volumes or instance
----------[Configuration File]----------
Configuration file saved:
/var/tmp/ec2rl/2021-05-01T07_51_32.356031/configuration.cfg
-------------[Output Logs]-------------
The output logs are located in:
/var/tmp/ec2rl/2021-05-01T07_51_32.356031
--------------[Module Run]--------------
Running Modules:
arptable, blkid, cgroups, clocksource, cpuinfo, date, ethtool, ethtoolg, ethtooli, ethtoolk, ethtools, ifconfig, iomem, iproute, ipslink, iptablesrules, kernelcmdline, kernelversion, kpti, last, localtime, lsblk, lsmod, lspci, mdstat, meminfo, mounts, osrelease, pagetypeinfo, partitions, procstat, retpoline, rpmpackages, scheddebug, slabinfo, sysctl, vmallocinfo, xenfeatures, zoneinfo, arpcache, arpignore, asymmetricroute, conntrackfull, consoleoverload, duplicatefslabels, duplicatefsuuid, duplicatepartuuid, hungtasks, ixgbevfversion, kernelbug, kerneldereference, kernelpanic, oomkiller, openssh, softlockup, tcprecycle, xennetrocket, cloudinitlog, dhclientleases, dmesg, environment, fstab, hosts, httpdlogs, inittab, kerberosconfig, kernelconfig, libtirpcnetconfig, lvmconf, messages, mysqldlog, nginxlogs, nsswitch, ntpconf, profile, resolvconf, sarhistory, sysctlconf, systemsmanager, udev, workspacelogs, yumlog
----------[Diagnostic Results]----------
module run/openssh [FAILURE] Improper configuration of one or more OpenSSH components.
-- SSH may deny access to users when improperly configured.
-- FAILURE Permission mode includes permissions for groups and/or other users: /etc/ssh/ssh_host_rsa_key
-- Adjust permissions: sudo chmod 600 /etc/ssh/ssh_host_rsa_key
-- FAILURE Permission mode includes permissions for groups and/or other users: /etc/ssh/ssh_host_ecdsa_key
-- Adjust permissions: sudo chmod 600 /etc/ssh/ssh_host_ecdsa_key
-- FAILURE Permission mode includes permissions for groups and/or other users: /etc/ssh/ssh_host_ed25519_key
-- Adjust permissions: sudo chmod 600 /etc/ssh/ssh_host_ed25519_key
module run/arpcache [SUCCESS] Aggressive arp caching is disabled.
module run/arpignore [SUCCESS] arp ignore is disabled for all interfaces.
module run/asymmetricroute [SUCCESS] No duplicate subnets found.
module run/conntrackfull [SUCCESS] No conntrack table full errors found.
module run/consoleoverload [SUCCESS] No serial console overload found
module run/duplicatefslabels [SUCCESS] No duplicate filesystem labels found.
module run/duplicatefsuuid [SUCCESS] No duplicate filesystem UUIDs found.
module run/duplicatepartuuid [SUCCESS] No duplicate partition UUIDs found.
module run/hungtasks [SUCCESS] No hung tasks found
module run/ixgbevfversion [SUCCESS] Not using ixgbevf driver.
module run/kernelbug [SUCCESS] No kernel bug occurrences found
module run/kerneldereference [SUCCESS] No kernel null pointer dereference ocurrences found
module run/kernelpanic [SUCCESS] No kernel panic ocurrences found
module run/oomkiller [SUCCESS] No oom-killer invocations found
module run/softlockup [SUCCESS] No CPU soft lockup ocurrences found
module run/tcprecycle [SUCCESS] Aggressive TCP recycling is disabled.
module run/xennetrocket [SUCCESS] Not using xen_netfront driver.
--------------[Run Stats]--------------
Total modules run: 82
'diagnose' modules run: 18
successes: 17
failures: 1
warnings: 0
unknown: 0
'collect' modules run: 39
'gather' modules run: 25
Modules not run due to missing: sudo | software | parameters | perf-impact
0 | 7 | 63 | 7
----------------[NOTICE]----------------
Please note, this directory could contain sensitive data depending on modules run! Please review its contents!
----------------[Upload]----------------
You can upload results to AWS Support with the following, or run 'help upload' for details on using an S3 presigned URL:
sudo ./ec2rl upload --upload-directory=/var/tmp/ec2rl/2021-05-01T07_51_32.356031 --support-url="URLProvidedByAWSSupport"
The quotation marks are required, and if you ran the tool with sudo, you will also need to upload with sudo.
---------------[Feedback]---------------
We appreciate your feedback. If you have any to give, please visit:
<https://aws.au1.qualtrics.com/jfe1/form/SV_3KrcrMZ2quIDzjn?InstanceID=i-00be31cf6a8086b47&Version=1.1.4>
$ pwd
/var/tmp/ec2rl
$
$ tree
.
tqq 2021-05-01T07_50_44.679097
x?? tqq gathered_out
x?? tqq Main.log
x?? mqq mod_out
x?? tqq postdiagnostic
x?? tqq prediagnostic
x?? mqq run
mqq 2021-05-01T07_51_32.356031 [error opening dir]
7 directories, 1 file
$
Invoke-WebRequest <https://s3.amazonaws.com/ec2rescue/windows/EC2Rescue_latest.zip> -OutFile $env:USERPROFILE\\Desktop\\EC2Rescue_latest.zip
sudo rm -rf /
)を 誤って 実行してしまった場合最近のEC2でサービスアップデートされている内容が一通り実施できてよかったです。 どうしても新しいサービスに飛びつきですが、こういったトラブルシューティング関連の内容は、 実際の現場で初めて触って、ただでさえ1分1秒の復旧が望まれるような状況で、 そもそもの使い方をドキュメントから読み解きながら実行するという二重苦からは少し開放されるかなと思います。
日頃から訓練って大事ですね。